2024-04-28T13:17:19.921 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-28T13:17:19.921 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/quiesce} -- /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911/orig.config.yaml 2024-04-28T13:17:19.966 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-28T13:17:20.053 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911 branch: wip-rishabh-testing-20240426.111959 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/quiesce} email: null first_in_suite: false job_id: '7676911' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow request sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI7L8TE5So2DQv96t6FnSiAvpZaOROivbIf2fyRlxXKTgmq3DTpY0WPzfNamw+B3wfidZAzU0uOrvF6H1I2X2g0= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHYTvuYQNVm4jN95gDka/WmqvB4XshGdGOq29jZP/IqzYmL0Ee4E/59D4XGQeznrW/74dP6qqCG/Y7eMwSIaH5w= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_quiesce teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-28_11:41:23 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-28T13:17:20.053 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-28T13:17:20.054 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-28T13:17:20.054 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-28T13:17:20.055 INFO:teuthology.task.internal:Checking packages... 2024-04-28T13:17:20.077 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-28T13:17:20.077 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-28T13:17:20.077 INFO:teuthology.packaging:ref: None 2024-04-28T13:17:20.077 INFO:teuthology.packaging:tag: None 2024-04-28T13:17:20.077 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-28T13:17:20.077 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T13:17:20.077 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-28T13:17:20.212 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-28T13:17:20.214 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-28T13:17:20.220 INFO:teuthology.task.internal:no buildpackages task found 2024-04-28T13:17:20.221 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-28T13:17:20.232 INFO:teuthology.task.internal:Saving configuration 2024-04-28T13:17:20.248 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-28T13:17:20.254 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-28T13:17:20.282 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-28 13:11:16.106168', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI7L8TE5So2DQv96t6FnSiAvpZaOROivbIf2fyRlxXKTgmq3DTpY0WPzfNamw+B3wfidZAzU0uOrvF6H1I2X2g0='} 2024-04-28T13:17:20.304 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-28 13:11:16.107648', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHYTvuYQNVm4jN95gDka/WmqvB4XshGdGOq29jZP/IqzYmL0Ee4E/59D4XGQeznrW/74dP6qqCG/Y7eMwSIaH5w='} 2024-04-28T13:17:20.304 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-28T13:17:20.310 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-28T13:17:20.310 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-28T13:17:20.311 INFO:teuthology.run_tasks:Running task console_log... 2024-04-28T13:17:20.379 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f66bd2c4160>, signals=[15]) 2024-04-28T13:17:20.380 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-28T13:17:20.394 INFO:teuthology.task.internal:Opening connections... 2024-04-28T13:17:20.394 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:17:20.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T13:17:20.470 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:17:20.471 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T13:17:20.547 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-28T13:17:20.553 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-04-28T13:17:20.572 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-04-28T13:17:20.572 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-04-28T13:17:20.630 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-04-28T13:17:20.630 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-04-28T13:17:20.630 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-04-28T13:17:20.630 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-04-28T13:17:20.630 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-04-28T13:17:20.630 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-04-28T13:17:20.630 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-28T13:17:20.630 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-04-28T13:17:20.630 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-04-28T13:17:20.630 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-28T13:17:20.630 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-04-28T13:17:20.631 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-28T13:17:20.631 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-28T13:17:20.631 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-28T13:17:20.631 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-04-28T13:17:20.650 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2024-04-28T13:17:20.668 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2024-04-28T13:17:20.668 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2024-04-28T13:17:20.725 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream" 2024-04-28T13:17:20.725 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9" 2024-04-28T13:17:20.725 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos" 2024-04-28T13:17:20.725 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora" 2024-04-28T13:17:20.725 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9" 2024-04-28T13:17:20.725 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9" 2024-04-28T13:17:20.725 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-28T13:17:20.725 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2024-04-28T13:17:20.725 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon" 2024-04-28T13:17:20.725 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-28T13:17:20.725 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/" 2024-04-28T13:17:20.725 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-28T13:17:20.725 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-28T13:17:20.725 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-28T13:17:20.726 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2024-04-28T13:17:20.750 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-28T13:17:20.760 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-28T13:17:20.766 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-28T13:17:20.766 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-04-28T13:17:20.768 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2024-04-28T13:17:20.784 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-28T13:17:20.791 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-28T13:17:20.791 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-04-28T13:17:20.826 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2024-04-28T13:17:20.928 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-28T13:17:20.979 INFO:teuthology.run_tasks:Running task kernel... 2024-04-28T13:17:20.994 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-28T13:17:20.994 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-28T13:17:20.994 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-28T13:17:20.995 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-28T13:17:20.995 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-28T13:17:21.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:17:21.011 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-28T13:17:21.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:17:21.015 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-28T13:17:21.167 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T13:17:21.167 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-28T13:17:21.167 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-04-28T13:17:21.185 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-437.el9.x86_64 2024-04-28T13:17:21.185 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T13:17:21.185 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-28T13:17:21.185 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum upgrade -y linux-firmware 2024-04-28T13:17:21.254 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T13:17:21.254 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-28T13:17:21.255 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2024-04-28T13:17:21.272 INFO:teuthology.orchestra.run.smithi203.stdout:5.14.0-437.el9.x86_64 2024-04-28T13:17:21.272 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T13:17:21.272 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-28T13:17:21.272 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum upgrade -y linux-firmware 2024-04-28T13:17:24.527 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.0 MB 00:01 2024-04-28T13:17:24.818 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-04-28T13:17:26.395 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-28T13:17:32.271 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.2 MB 00:02 2024-04-28T13:17:33.922 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-28T13:17:48.165 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 874 kB/s | 19 MB 00:22 2024-04-28T13:17:53.315 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-28T13:17:55.006 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 6.2 MB 00:03 2024-04-28T13:17:56.640 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-28T13:17:58.413 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-04-28T13:17:58.608 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-28T13:18:00.393 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-04-28T13:18:00.394 INFO:teuthology.orchestra.run.smithi203.stdout:Nothing to do. 2024-04-28T13:18:00.395 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-04-28T13:18:00.495 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-28T13:18:00.495 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T13:18:00.637 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-28T13:18:00.637 DEBUG:teuthology.orchestra.run.smithi203:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-28T13:18:00.640 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi203.front.sepia.ceph.com to complete... 2024-04-28T13:18:03.724 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-04-28T13:18:05.686 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-04-28T13:18:05.687 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-04-28T13:18:05.687 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-04-28T13:18:05.798 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-28T13:18:05.798 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T13:18:05.944 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-28T13:18:05.945 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-28T13:18:05.947 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi196.front.sepia.ceph.com to complete... 2024-04-28T13:19:12.316 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28 13:19:12 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-28T13:19:12.316 INFO:teuthology.orchestra.run.smithi203.stderr:FINISHED --2024-04-28 13:19:12-- 2024-04-28T13:19:12.316 INFO:teuthology.orchestra.run.smithi203.stderr:Total wall clock time: 1m 12s 2024-04-28T13:19:12.316 INFO:teuthology.orchestra.run.smithi203.stderr:Downloaded: 1 files, 341M in 1m 11s (4.79 MB/s) 2024-04-28T13:19:12.320 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-28T13:19:12.320 DEBUG:teuthology.orchestra.run.smithi203:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-28T13:19:14.409 INFO:teuthology.orchestra.run.smithi203.stdout:Verifying... ######################################## 2024-04-28T13:19:14.581 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing... ######################################## 2024-04-28T13:19:14.687 INFO:teuthology.orchestra.run.smithi203.stdout:Updating / installing... 2024-04-28T13:19:22.696 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-28T13:19:56.662 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28 13:19:56 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-28T13:19:56.662 INFO:teuthology.orchestra.run.smithi196.stderr:FINISHED --2024-04-28 13:19:56-- 2024-04-28T13:19:56.662 INFO:teuthology.orchestra.run.smithi196.stderr:Total wall clock time: 1m 51s 2024-04-28T13:19:56.662 INFO:teuthology.orchestra.run.smithi196.stderr:Downloaded: 1 files, 341M in 1m 50s (3.10 MB/s) 2024-04-28T13:19:56.665 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-28T13:19:56.665 DEBUG:teuthology.orchestra.run.smithi196:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-28T13:19:58.799 INFO:teuthology.orchestra.run.smithi196.stdout:Verifying... ######################################## 2024-04-28T13:19:58.988 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing... ######################################## 2024-04-28T13:19:59.097 INFO:teuthology.orchestra.run.smithi196.stdout:Updating / installing... 2024-04-28T13:20:00.788 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi203.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-28T13:20:00.788 DEBUG:teuthology.orchestra.run.smithi203:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-28T13:20:00.875 INFO:teuthology.orchestra.run.smithi203.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:00.875 INFO:teuthology.orchestra.run.smithi203.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:00.875 INFO:teuthology.orchestra.run.smithi203.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:00.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-28T13:20:00.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-28T13:20:00.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-28T13:20:00.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-28T13:20:00.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-28T13:20:00.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-28T13:20:00.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-28T13:20:00.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-28T13:20:00.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-28T13:20:00.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-28T13:20:00.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-28T13:20:00.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-28T13:20:00.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-28T13:20:00.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-28T13:20:00.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-28T13:20:00.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-28T13:20:00.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-28T13:20:00.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-28T13:20:00.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-28T13:20:00.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-28T13:20:00.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-28T13:20:00.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-28T13:20:00.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-28T13:20:00.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-28T13:20:00.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-28T13:20:00.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-28T13:20:00.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-28T13:20:00.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-28T13:20:00.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-28T13:20:00.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-28T13:20:00.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-28T13:20:00.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-28T13:20:00.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-28T13:20:00.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-28T13:20:00.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-28T13:20:00.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-28T13:20:00.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-28T13:20:00.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-28T13:20:00.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-28T13:20:00.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-28T13:20:00.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-28T13:20:00.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-28T13:20:00.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-28T13:20:00.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-28T13:20:00.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-28T13:20:00.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-28T13:20:00.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-28T13:20:00.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-28T13:20:00.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-28T13:20:00.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-28T13:20:00.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-28T13:20:00.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-28T13:20:00.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-28T13:20:00.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-28T13:20:00.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-28T13:20:00.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-28T13:20:00.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-28T13:20:00.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-28T13:20:00.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-28T13:20:00.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-28T13:20:00.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-28T13:20:00.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-28T13:20:00.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-28T13:20:00.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-28T13:20:00.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-28T13:20:00.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-28T13:20:00.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-28T13:20:00.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-28T13:20:00.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-28T13:20:00.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-28T13:20:00.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-28T13:20:00.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-28T13:20:00.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-28T13:20:00.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-28T13:20:00.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-28T13:20:00.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-28T13:20:00.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-28T13:20:00.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-28T13:20:00.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-28T13:20:00.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-28T13:20:00.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-28T13:20:00.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-28T13:20:00.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-28T13:20:00.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-28T13:20:00.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-28T13:20:00.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-28T13:20:00.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-28T13:20:00.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-28T13:20:00.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-28T13:20:00.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-28T13:20:00.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-28T13:20:00.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-28T13:20:00.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-28T13:20:00.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-28T13:20:00.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-28T13:20:00.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-28T13:20:00.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-28T13:20:00.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-28T13:20:00.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-28T13:20:00.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-28T13:20:00.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-28T13:20:00.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-28T13:20:00.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-28T13:20:00.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-28T13:20:00.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-28T13:20:00.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-28T13:20:00.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-28T13:20:00.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-28T13:20:00.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-28T13:20:00.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-28T13:20:00.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-28T13:20:00.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-28T13:20:00.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-28T13:20:00.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-28T13:20:00.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-28T13:20:00.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-28T13:20:00.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-28T13:20:00.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-28T13:20:00.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-28T13:20:00.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-28T13:20:00.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-28T13:20:00.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-28T13:20:00.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-28T13:20:00.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-28T13:20:00.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-28T13:20:00.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-28T13:20:00.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-28T13:20:00.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-28T13:20:00.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-28T13:20:00.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-28T13:20:00.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-28T13:20:00.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-28T13:20:00.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-28T13:20:00.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-28T13:20:00.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-28T13:20:00.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-28T13:20:00.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-28T13:20:00.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-28T13:20:00.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-28T13:20:00.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-28T13:20:00.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-28T13:20:00.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-28T13:20:00.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-28T13:20:00.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-28T13:20:00.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-28T13:20:00.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-28T13:20:00.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-28T13:20:00.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-28T13:20:00.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-28T13:20:00.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-28T13:20:00.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-28T13:20:00.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-28T13:20:00.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-28T13:20:00.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-28T13:20:00.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-28T13:20:00.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-28T13:20:00.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-28T13:20:00.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-28T13:20:00.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-28T13:20:00.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-28T13:20:00.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-28T13:20:00.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-28T13:20:00.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-28T13:20:00.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-28T13:20:00.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-28T13:20:00.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-28T13:20:00.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-28T13:20:00.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-28T13:20:00.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-28T13:20:00.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-28T13:20:00.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-28T13:20:00.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-28T13:20:00.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-28T13:20:00.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-28T13:20:00.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-28T13:20:00.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-28T13:20:00.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-28T13:20:00.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-28T13:20:00.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-28T13:20:00.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-28T13:20:00.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-28T13:20:00.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-28T13:20:00.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-28T13:20:00.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-28T13:20:00.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-28T13:20:00.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-28T13:20:00.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-28T13:20:00.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-28T13:20:00.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-28T13:20:00.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-28T13:20:00.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-28T13:20:00.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-28T13:20:00.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-28T13:20:00.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-28T13:20:00.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-28T13:20:00.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-28T13:20:00.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-28T13:20:00.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-28T13:20:00.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-28T13:20:00.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-28T13:20:00.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-28T13:20:00.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-28T13:20:00.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-28T13:20:00.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-28T13:20:00.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-28T13:20:00.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-28T13:20:00.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-28T13:20:00.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-28T13:20:00.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-28T13:20:00.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-28T13:20:00.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-28T13:20:00.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-28T13:20:00.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-28T13:20:00.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-28T13:20:00.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-28T13:20:00.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-28T13:20:00.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-28T13:20:00.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-28T13:20:00.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-28T13:20:00.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-28T13:20:00.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-28T13:20:00.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-28T13:20:00.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-28T13:20:00.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-28T13:20:00.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-28T13:20:00.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-28T13:20:00.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-28T13:20:00.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-28T13:20:00.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-28T13:20:00.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-28T13:20:00.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-28T13:20:00.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-28T13:20:00.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-28T13:20:00.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-28T13:20:00.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-28T13:20:00.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-28T13:20:00.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-28T13:20:00.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-28T13:20:00.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-28T13:20:00.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-28T13:20:00.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-28T13:20:00.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-28T13:20:00.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-28T13:20:00.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-28T13:20:00.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-28T13:20:00.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-28T13:20:00.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-28T13:20:00.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-28T13:20:00.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-28T13:20:00.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-28T13:20:00.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-28T13:20:00.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-28T13:20:00.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-28T13:20:00.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-28T13:20:00.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-28T13:20:00.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-28T13:20:00.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-28T13:20:00.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-28T13:20:00.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-28T13:20:00.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-28T13:20:00.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-28T13:20:00.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-28T13:20:00.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-28T13:20:00.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-28T13:20:00.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-28T13:20:00.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-28T13:20:00.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-28T13:20:00.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-28T13:20:00.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-28T13:20:00.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-28T13:20:00.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-28T13:20:00.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-28T13:20:00.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-28T13:20:00.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-28T13:20:00.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-28T13:20:00.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-28T13:20:00.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-28T13:20:00.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-28T13:20:00.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-28T13:20:00.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-28T13:20:00.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-28T13:20:00.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-28T13:20:00.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-28T13:20:00.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-28T13:20:00.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-28T13:20:00.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-28T13:20:00.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-28T13:20:00.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-28T13:20:00.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-28T13:20:00.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-28T13:20:00.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-28T13:20:00.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-28T13:20:00.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-28T13:20:00.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-28T13:20:00.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-28T13:20:00.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-28T13:20:00.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-28T13:20:00.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-28T13:20:00.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-28T13:20:00.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-28T13:20:00.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-28T13:20:00.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-28T13:20:00.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-28T13:20:00.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-28T13:20:00.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-28T13:20:00.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-28T13:20:00.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-28T13:20:00.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-28T13:20:00.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-28T13:20:00.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-28T13:20:00.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-28T13:20:00.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-28T13:20:00.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-28T13:20:00.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-28T13:20:00.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-28T13:20:00.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-28T13:20:00.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-28T13:20:00.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-28T13:20:00.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-28T13:20:00.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-28T13:20:00.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-28T13:20:00.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-28T13:20:00.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-28T13:20:00.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-28T13:20:00.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-28T13:20:00.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-28T13:20:00.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-28T13:20:00.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-28T13:20:00.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-28T13:20:00.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-28T13:20:00.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-28T13:20:00.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-28T13:20:00.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-28T13:20:00.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-28T13:20:00.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-28T13:20:00.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-28T13:20:00.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-28T13:20:00.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-28T13:20:00.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-28T13:20:00.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-28T13:20:00.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-28T13:20:00.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-28T13:20:00.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-28T13:20:00.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-28T13:20:00.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-28T13:20:00.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-28T13:20:00.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-28T13:20:00.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-28T13:20:00.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-28T13:20:00.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-28T13:20:00.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-28T13:20:00.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-28T13:20:00.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-28T13:20:00.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-28T13:20:00.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-28T13:20:00.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-28T13:20:00.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-28T13:20:00.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-28T13:20:00.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-28T13:20:00.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-28T13:20:00.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-28T13:20:00.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-28T13:20:00.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-28T13:20:00.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-28T13:20:00.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-28T13:20:00.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-28T13:20:00.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-28T13:20:00.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-28T13:20:00.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-28T13:20:00.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-28T13:20:00.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-28T13:20:00.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-28T13:20:00.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-28T13:20:00.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-28T13:20:00.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-28T13:20:00.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-28T13:20:00.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-28T13:20:00.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-28T13:20:00.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-28T13:20:00.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-28T13:20:00.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-28T13:20:00.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-28T13:20:00.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-28T13:20:00.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-28T13:20:00.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-28T13:20:00.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-28T13:20:00.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-28T13:20:00.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-28T13:20:00.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-28T13:20:00.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-28T13:20:00.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-28T13:20:00.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-28T13:20:00.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-28T13:20:00.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-28T13:20:00.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-28T13:20:00.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-28T13:20:00.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-28T13:20:00.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-28T13:20:00.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-28T13:20:00.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-28T13:20:00.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-28T13:20:00.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-28T13:20:00.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-28T13:20:00.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-28T13:20:00.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-28T13:20:00.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-28T13:20:00.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-28T13:20:00.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-28T13:20:00.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-28T13:20:00.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-28T13:20:00.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-28T13:20:00.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-28T13:20:00.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-28T13:20:00.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-28T13:20:00.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-28T13:20:00.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-28T13:20:00.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-28T13:20:00.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-28T13:20:00.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-28T13:20:00.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-28T13:20:00.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-28T13:20:00.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-28T13:20:00.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-28T13:20:00.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-28T13:20:00.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-28T13:20:00.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-28T13:20:00.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-28T13:20:00.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-28T13:20:00.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-28T13:20:00.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-28T13:20:00.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-28T13:20:00.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-28T13:20:00.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-28T13:20:00.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-28T13:20:00.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-28T13:20:00.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-28T13:20:00.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-28T13:20:00.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-28T13:20:00.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-28T13:20:00.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-28T13:20:00.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-28T13:20:00.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-28T13:20:00.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-28T13:20:00.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-28T13:20:00.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-28T13:20:00.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-28T13:20:00.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-28T13:20:00.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-28T13:20:00.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-28T13:20:00.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-28T13:20:00.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-28T13:20:00.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-28T13:20:00.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-28T13:20:00.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-28T13:20:00.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-28T13:20:00.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-28T13:20:00.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-28T13:20:00.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-28T13:20:00.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-28T13:20:00.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-28T13:20:00.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-28T13:20:00.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-28T13:20:00.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-28T13:20:00.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-28T13:20:00.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-28T13:20:00.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-28T13:20:00.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-28T13:20:00.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-28T13:20:00.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-28T13:20:00.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-28T13:20:00.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-28T13:20:00.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-28T13:20:00.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-28T13:20:00.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-28T13:20:00.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-28T13:20:00.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-28T13:20:00.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-28T13:20:00.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-28T13:20:00.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-28T13:20:00.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-28T13:20:00.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-28T13:20:00.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-28T13:20:00.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-28T13:20:00.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-28T13:20:00.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-28T13:20:00.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-28T13:20:00.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-28T13:20:00.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-28T13:20:00.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-28T13:20:00.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-28T13:20:00.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-28T13:20:00.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-28T13:20:00.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-28T13:20:00.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-28T13:20:00.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-28T13:20:00.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-28T13:20:00.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-28T13:20:00.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-28T13:20:00.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-28T13:20:00.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-28T13:20:00.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-28T13:20:00.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-28T13:20:00.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-28T13:20:00.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-28T13:20:00.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-28T13:20:00.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-28T13:20:00.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-28T13:20:00.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-28T13:20:00.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-28T13:20:00.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-28T13:20:00.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-28T13:20:00.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-28T13:20:00.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-28T13:20:00.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-28T13:20:00.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-28T13:20:00.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-28T13:20:00.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-28T13:20:00.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-28T13:20:00.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-28T13:20:00.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-28T13:20:00.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-28T13:20:00.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-28T13:20:00.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-28T13:20:00.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-28T13:20:00.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-28T13:20:00.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-28T13:20:00.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-28T13:20:00.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-28T13:20:00.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-28T13:20:00.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-28T13:20:00.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-28T13:20:00.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-28T13:20:00.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-28T13:20:00.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-28T13:20:00.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-28T13:20:00.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-28T13:20:00.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-28T13:20:00.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-28T13:20:00.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-28T13:20:00.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-28T13:20:00.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-28T13:20:00.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-28T13:20:00.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-28T13:20:00.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-28T13:20:00.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-28T13:20:00.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-28T13:20:00.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-28T13:20:00.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-28T13:20:00.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-28T13:20:00.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-28T13:20:00.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-28T13:20:00.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-28T13:20:00.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-28T13:20:00.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-28T13:20:00.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-28T13:20:00.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-28T13:20:00.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-28T13:20:00.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-28T13:20:00.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-28T13:20:00.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-28T13:20:00.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-28T13:20:00.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-28T13:20:00.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-28T13:20:00.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-28T13:20:00.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-28T13:20:00.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-28T13:20:00.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-28T13:20:00.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-28T13:20:00.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-28T13:20:00.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-28T13:20:00.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-28T13:20:00.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-28T13:20:00.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-28T13:20:00.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-28T13:20:00.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-28T13:20:00.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-28T13:20:00.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-28T13:20:00.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-28T13:20:00.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-28T13:20:00.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-28T13:20:00.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-28T13:20:00.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-28T13:20:00.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-28T13:20:00.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-28T13:20:00.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-28T13:20:00.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-28T13:20:00.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-28T13:20:00.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-28T13:20:00.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-28T13:20:00.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-28T13:20:00.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-28T13:20:00.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-28T13:20:00.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-28T13:20:00.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-28T13:20:00.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-28T13:20:00.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-28T13:20:00.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-28T13:20:00.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-28T13:20:00.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-28T13:20:00.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-28T13:20:00.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-28T13:20:00.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-28T13:20:00.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-28T13:20:00.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-28T13:20:00.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-28T13:20:00.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-28T13:20:00.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-28T13:20:00.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-28T13:20:00.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-28T13:20:00.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-28T13:20:00.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-28T13:20:00.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-28T13:20:00.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-28T13:20:00.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-28T13:20:00.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-28T13:20:00.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-28T13:20:00.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-28T13:20:00.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-28T13:20:00.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-28T13:20:00.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-28T13:20:00.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-28T13:20:00.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-28T13:20:00.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-28T13:20:00.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-28T13:20:00.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-28T13:20:00.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-28T13:20:00.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-28T13:20:00.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-28T13:20:00.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-28T13:20:00.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-28T13:20:00.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-28T13:20:00.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-28T13:20:00.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-28T13:20:00.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-28T13:20:00.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-28T13:20:00.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-28T13:20:00.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-28T13:20:00.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-28T13:20:00.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-28T13:20:00.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-28T13:20:00.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-28T13:20:00.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-28T13:20:00.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-28T13:20:00.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-28T13:20:00.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-28T13:20:00.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-28T13:20:00.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-28T13:20:00.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-28T13:20:00.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-28T13:20:00.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-28T13:20:00.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-28T13:20:00.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-28T13:20:00.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-28T13:20:00.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-28T13:20:00.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-28T13:20:00.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-28T13:20:00.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-28T13:20:00.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-28T13:20:00.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-28T13:20:00.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-28T13:20:00.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-28T13:20:00.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-28T13:20:00.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-28T13:20:00.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-28T13:20:00.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-28T13:20:00.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-28T13:20:00.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-28T13:20:00.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-28T13:20:00.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-28T13:20:00.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-28T13:20:00.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-28T13:20:00.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-28T13:20:00.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-28T13:20:00.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-28T13:20:00.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-28T13:20:00.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-28T13:20:00.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-28T13:20:00.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-28T13:20:00.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-28T13:20:00.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-28T13:20:00.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-28T13:20:00.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-28T13:20:00.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-28T13:20:00.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-28T13:20:00.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-28T13:20:00.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-28T13:20:00.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-28T13:20:00.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-28T13:20:00.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-28T13:20:00.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-28T13:20:00.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-28T13:20:00.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-28T13:20:00.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-28T13:20:00.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-28T13:20:00.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-28T13:20:00.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-28T13:20:00.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-28T13:20:00.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-28T13:20:00.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-28T13:20:00.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-28T13:20:00.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-28T13:20:00.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-28T13:20:00.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-28T13:20:00.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-28T13:20:00.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-28T13:20:00.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-28T13:20:00.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-28T13:20:00.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-28T13:20:00.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-28T13:20:00.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-28T13:20:00.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-28T13:20:00.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-28T13:20:00.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-28T13:20:00.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-28T13:20:00.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-28T13:20:00.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-28T13:20:00.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-28T13:20:00.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-28T13:20:00.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-28T13:20:00.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-28T13:20:00.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-28T13:20:00.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-28T13:20:00.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-28T13:20:00.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-28T13:20:00.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-28T13:20:00.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-28T13:20:00.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-28T13:20:00.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-28T13:20:00.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-28T13:20:00.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-28T13:20:00.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-28T13:20:00.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-28T13:20:00.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-28T13:20:00.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-28T13:20:00.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-28T13:20:00.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-28T13:20:00.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-28T13:20:00.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-28T13:20:00.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-28T13:20:00.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-28T13:20:00.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-28T13:20:00.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-28T13:20:00.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-28T13:20:00.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-28T13:20:00.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-28T13:20:00.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-28T13:20:00.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-28T13:20:00.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-28T13:20:00.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-28T13:20:00.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-28T13:20:00.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-28T13:20:00.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-28T13:20:00.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-28T13:20:00.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-28T13:20:00.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-28T13:20:00.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-28T13:20:00.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-28T13:20:00.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-28T13:20:00.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-28T13:20:00.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-28T13:20:00.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-28T13:20:00.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-28T13:20:00.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-28T13:20:00.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-28T13:20:00.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-28T13:20:00.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-28T13:20:00.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-28T13:20:00.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-28T13:20:00.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-28T13:20:00.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-28T13:20:00.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-28T13:20:00.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-28T13:20:00.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-28T13:20:00.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-28T13:20:00.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-28T13:20:00.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-28T13:20:00.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-28T13:20:00.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-28T13:20:00.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-28T13:20:00.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-28T13:20:00.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-28T13:20:00.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-28T13:20:00.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-28T13:20:00.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-28T13:20:00.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-28T13:20:00.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-28T13:20:00.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-28T13:20:00.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-28T13:20:00.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-28T13:20:00.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-28T13:20:00.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-28T13:20:00.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-28T13:20:00.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-28T13:20:00.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-28T13:20:00.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-28T13:20:00.940 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-28T13:20:00.940 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-28T13:20:00.940 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-28T13:20:00.940 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-28T13:20:00.940 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-28T13:20:00.940 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-28T13:20:00.940 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-28T13:20:00.940 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-28T13:20:00.940 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-28T13:20:00.940 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-28T13:20:00.940 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-28T13:20:00.940 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-28T13:20:00.941 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-28T13:20:00.941 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-28T13:20:00.941 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-28T13:20:00.941 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-28T13:20:00.941 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-28T13:20:00.941 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-28T13:20:00.941 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-28T13:20:00.941 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-28T13:20:00.941 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-28T13:20:00.941 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-28T13:20:00.941 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-28T13:20:00.941 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-28T13:20:00.942 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-28T13:20:00.942 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-28T13:20:00.942 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-28T13:20:00.942 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-28T13:20:00.942 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-28T13:20:00.942 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-28T13:20:00.942 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-28T13:20:00.942 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-28T13:20:00.942 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-28T13:20:00.942 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-28T13:20:00.942 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-28T13:20:00.942 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-28T13:20:00.942 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-28T13:20:00.942 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-28T13:20:00.943 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-28T13:20:00.943 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-28T13:20:00.943 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-28T13:20:00.943 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-28T13:20:00.943 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-28T13:20:00.943 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-28T13:20:00.943 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-28T13:20:00.943 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-28T13:20:00.943 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-28T13:20:00.943 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-28T13:20:00.943 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-28T13:20:00.943 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-28T13:20:00.943 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-28T13:20:00.943 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-28T13:20:00.943 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-28T13:20:00.944 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-28T13:20:00.944 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-28T13:20:00.944 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-28T13:20:00.944 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-28T13:20:00.944 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-28T13:20:00.944 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-28T13:20:00.944 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-28T13:20:00.944 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-28T13:20:00.944 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-28T13:20:00.944 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-28T13:20:00.944 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-28T13:20:00.944 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-28T13:20:00.944 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-28T13:20:00.945 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-28T13:20:00.945 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-28T13:20:00.945 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-28T13:20:00.945 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-28T13:20:00.945 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-28T13:20:00.945 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-28T13:20:00.945 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-28T13:20:00.945 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-28T13:20:00.945 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-28T13:20:00.945 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-28T13:20:00.945 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-28T13:20:00.945 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-28T13:20:00.945 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-28T13:20:00.946 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-28T13:20:00.946 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-28T13:20:00.946 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-28T13:20:00.946 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-28T13:20:00.946 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-28T13:20:00.946 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-28T13:20:00.946 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-28T13:20:00.946 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-28T13:20:00.946 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-28T13:20:00.946 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-28T13:20:00.946 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-28T13:20:00.946 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-28T13:20:00.947 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-28T13:20:00.947 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-28T13:20:00.947 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-28T13:20:00.947 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-28T13:20:00.947 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-28T13:20:00.947 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-28T13:20:00.947 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-28T13:20:00.947 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-28T13:20:00.947 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-28T13:20:00.947 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-28T13:20:00.947 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-28T13:20:00.947 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-28T13:20:00.947 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-28T13:20:00.948 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-28T13:20:00.948 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-28T13:20:00.948 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-28T13:20:00.948 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-28T13:20:00.948 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-28T13:20:00.948 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-28T13:20:00.948 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-28T13:20:00.948 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-28T13:20:00.948 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-28T13:20:00.948 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-28T13:20:00.948 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-28T13:20:00.948 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-28T13:20:00.948 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-28T13:20:00.949 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-28T13:20:00.949 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-28T13:20:00.949 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-28T13:20:00.949 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-28T13:20:00.949 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-28T13:20:00.949 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-28T13:20:00.949 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-28T13:20:00.949 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-28T13:20:00.949 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-28T13:20:00.949 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-28T13:20:00.949 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-28T13:20:00.949 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-28T13:20:00.950 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-28T13:20:00.950 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-28T13:20:00.950 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-28T13:20:00.950 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-28T13:20:00.950 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-28T13:20:00.950 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-28T13:20:00.950 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-28T13:20:00.950 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-28T13:20:00.950 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-28T13:20:00.950 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-28T13:20:00.950 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-28T13:20:00.951 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-28T13:20:00.951 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-28T13:20:00.951 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-28T13:20:00.951 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-28T13:20:00.951 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-28T13:20:00.951 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-28T13:20:00.951 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-28T13:20:00.951 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-28T13:20:00.951 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-28T13:20:00.951 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-28T13:20:00.951 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-28T13:20:00.951 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-28T13:20:00.952 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-28T13:20:00.952 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-28T13:20:00.952 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-28T13:20:00.952 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-28T13:20:00.952 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-28T13:20:00.952 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-28T13:20:00.952 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-28T13:20:00.952 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-28T13:20:00.952 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-28T13:20:00.952 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-28T13:20:00.952 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-28T13:20:00.952 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-28T13:20:00.953 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-28T13:20:00.953 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-28T13:20:00.953 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-28T13:20:00.953 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-28T13:20:00.953 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-28T13:20:00.953 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-28T13:20:00.953 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-28T13:20:00.953 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-28T13:20:00.953 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-28T13:20:00.953 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-28T13:20:00.953 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-28T13:20:00.954 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-28T13:20:00.954 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-28T13:20:00.954 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-28T13:20:00.954 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-28T13:20:00.954 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-28T13:20:00.954 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-28T13:20:00.954 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-28T13:20:00.954 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-28T13:20:00.954 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-28T13:20:00.954 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-28T13:20:00.954 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-28T13:20:00.954 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-28T13:20:00.955 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-28T13:20:00.955 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-28T13:20:00.955 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-28T13:20:00.955 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-28T13:20:00.955 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-28T13:20:00.955 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-28T13:20:00.955 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-28T13:20:00.955 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-28T13:20:00.955 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-28T13:20:00.955 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-28T13:20:00.955 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-28T13:20:00.955 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-28T13:20:00.956 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-28T13:20:00.956 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-28T13:20:00.956 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-28T13:20:00.956 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-28T13:20:00.956 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-28T13:20:00.956 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-28T13:20:00.956 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-28T13:20:00.956 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-28T13:20:00.956 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-28T13:20:00.956 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-28T13:20:00.956 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-28T13:20:00.956 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-28T13:20:00.957 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-28T13:20:00.957 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-28T13:20:00.957 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-28T13:20:00.957 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-28T13:20:00.957 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-28T13:20:00.957 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-28T13:20:00.957 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-28T13:20:00.957 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-28T13:20:00.957 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-28T13:20:00.957 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-28T13:20:00.957 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-28T13:20:00.957 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-28T13:20:00.957 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-28T13:20:00.957 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-28T13:20:00.957 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-28T13:20:00.958 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-28T13:20:00.958 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-28T13:20:00.958 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-28T13:20:00.958 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-28T13:20:00.958 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-28T13:20:00.958 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-28T13:20:00.958 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-28T13:20:00.958 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-28T13:20:00.958 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-28T13:20:00.958 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-28T13:20:00.958 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-28T13:20:00.958 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-28T13:20:00.958 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-28T13:20:00.958 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-28T13:20:00.959 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-28T13:20:00.959 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-28T13:20:00.959 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-28T13:20:00.959 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-28T13:20:00.959 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-28T13:20:00.959 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-28T13:20:00.959 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-28T13:20:00.959 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-28T13:20:00.959 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-28T13:20:00.959 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-28T13:20:00.959 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-28T13:20:00.959 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-28T13:20:00.959 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-28T13:20:00.959 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-28T13:20:00.960 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-28T13:20:00.960 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-28T13:20:00.960 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-28T13:20:00.960 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-28T13:20:00.960 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-28T13:20:00.960 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-28T13:20:00.960 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-28T13:20:00.960 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-28T13:20:00.960 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-28T13:20:00.960 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-28T13:20:00.960 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-28T13:20:00.960 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-28T13:20:00.960 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-28T13:20:00.961 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-28T13:20:00.961 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-28T13:20:00.961 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-28T13:20:00.961 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-28T13:20:00.961 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-28T13:20:00.961 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-28T13:20:00.961 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-28T13:20:00.961 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-28T13:20:00.961 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-28T13:20:00.961 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-28T13:20:00.961 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-28T13:20:00.961 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-28T13:20:00.961 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-28T13:20:00.961 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-28T13:20:00.962 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-28T13:20:00.962 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-28T13:20:00.962 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-28T13:20:00.962 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-28T13:20:00.962 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-28T13:20:00.962 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-28T13:20:00.962 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-28T13:20:00.962 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-28T13:20:00.962 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-28T13:20:00.962 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-28T13:20:00.962 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-28T13:20:00.962 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-28T13:20:00.962 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-28T13:20:00.963 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-28T13:20:00.963 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-28T13:20:00.963 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-28T13:20:00.963 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-28T13:20:00.963 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-28T13:20:00.963 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-28T13:20:00.963 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-28T13:20:00.963 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-28T13:20:00.963 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-28T13:20:00.963 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-28T13:20:00.963 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-28T13:20:00.963 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-28T13:20:00.963 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-28T13:20:00.963 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-28T13:20:00.963 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-28T13:20:00.964 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-28T13:20:00.964 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-28T13:20:00.964 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-28T13:20:00.964 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-28T13:20:00.964 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-28T13:20:00.964 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-28T13:20:00.964 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-28T13:20:00.964 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-28T13:20:00.964 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-28T13:20:00.964 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-28T13:20:00.964 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-28T13:20:00.964 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-28T13:20:00.964 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-28T13:20:00.964 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-28T13:20:00.964 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-28T13:20:00.965 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-28T13:20:00.965 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-28T13:20:00.965 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-28T13:20:00.965 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-28T13:20:00.965 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-28T13:20:00.965 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-28T13:20:00.965 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-28T13:20:00.965 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-28T13:20:00.965 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-28T13:20:00.965 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-28T13:20:00.965 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-28T13:20:00.965 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-28T13:20:00.965 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-28T13:20:00.965 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-28T13:20:00.966 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-28T13:20:00.966 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-28T13:20:00.966 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-28T13:20:00.966 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-28T13:20:00.966 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-28T13:20:00.966 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-28T13:20:00.966 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-28T13:20:00.966 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-28T13:20:00.966 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-28T13:20:00.966 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-28T13:20:00.966 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-28T13:20:00.967 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-28T13:20:00.967 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-28T13:20:00.967 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-28T13:20:00.967 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-28T13:20:00.967 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-28T13:20:00.967 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-28T13:20:00.967 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-28T13:20:00.967 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-28T13:20:00.967 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-28T13:20:00.967 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-28T13:20:00.967 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-28T13:20:00.967 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-28T13:20:00.968 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-28T13:20:00.968 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-28T13:20:00.968 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-28T13:20:00.968 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-28T13:20:00.968 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-28T13:20:00.968 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-28T13:20:00.968 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-28T13:20:00.968 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-28T13:20:00.968 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-28T13:20:00.968 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-28T13:20:00.968 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-28T13:20:00.968 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-28T13:20:00.969 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-28T13:20:00.969 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-28T13:20:00.969 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-28T13:20:00.969 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-28T13:20:00.969 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-28T13:20:00.969 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-28T13:20:00.969 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-28T13:20:00.969 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-28T13:20:00.969 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-28T13:20:00.969 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-28T13:20:00.969 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-28T13:20:00.969 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-28T13:20:00.969 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-28T13:20:00.970 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-28T13:20:00.970 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-28T13:20:00.970 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-28T13:20:00.970 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-28T13:20:00.970 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-28T13:20:00.970 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-28T13:20:00.970 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-28T13:20:00.970 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-28T13:20:00.970 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-28T13:20:00.970 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-28T13:20:00.970 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-28T13:20:00.970 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-28T13:20:00.970 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-28T13:20:00.971 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-28T13:20:00.971 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-28T13:20:00.971 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-28T13:20:00.971 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-28T13:20:00.971 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-28T13:20:00.971 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-28T13:20:00.971 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-28T13:20:00.971 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-28T13:20:00.971 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-28T13:20:00.971 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-28T13:20:00.971 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-28T13:20:00.972 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-28T13:20:00.972 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-28T13:20:00.972 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-28T13:20:00.972 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-28T13:20:00.972 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-28T13:20:00.972 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-28T13:20:00.972 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-28T13:20:00.972 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-28T13:20:00.972 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-28T13:20:00.972 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-28T13:20:00.972 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-28T13:20:00.972 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-28T13:20:00.973 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-28T13:20:00.973 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-28T13:20:00.973 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-28T13:20:00.973 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-28T13:20:00.973 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-28T13:20:00.973 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-28T13:20:00.973 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-28T13:20:00.973 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-28T13:20:00.973 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-28T13:20:00.973 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-28T13:20:00.973 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-28T13:20:00.973 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-28T13:20:00.974 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-28T13:20:00.974 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-28T13:20:00.974 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-28T13:20:00.974 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-28T13:20:00.974 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-28T13:20:00.974 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-28T13:20:00.974 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-28T13:20:00.974 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-28T13:20:00.974 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-28T13:20:00.974 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-28T13:20:00.974 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-28T13:20:00.975 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-28T13:20:00.975 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-28T13:20:00.975 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-28T13:20:00.975 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-28T13:20:00.975 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-28T13:20:00.975 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-28T13:20:00.975 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-28T13:20:00.975 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-28T13:20:00.975 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-28T13:20:00.975 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-28T13:20:00.975 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-28T13:20:00.975 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-28T13:20:00.976 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-28T13:20:00.976 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-28T13:20:00.976 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-28T13:20:00.976 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-28T13:20:00.976 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-28T13:20:00.976 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-28T13:20:00.976 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-28T13:20:00.976 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-28T13:20:00.976 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-28T13:20:00.976 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-28T13:20:00.976 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-28T13:20:00.977 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-28T13:20:00.977 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-28T13:20:00.977 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-28T13:20:00.977 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-28T13:20:00.977 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-28T13:20:00.977 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-28T13:20:00.977 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-28T13:20:00.977 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-28T13:20:00.977 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-28T13:20:00.977 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-28T13:20:00.977 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-28T13:20:00.977 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-28T13:20:00.978 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-28T13:20:00.978 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-28T13:20:00.978 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-28T13:20:00.978 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-28T13:20:00.978 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-28T13:20:00.978 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-28T13:20:00.978 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-28T13:20:00.978 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-28T13:20:00.978 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-28T13:20:00.978 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-28T13:20:00.978 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-28T13:20:00.978 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-28T13:20:00.979 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-28T13:20:00.979 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-28T13:20:00.979 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-28T13:20:00.979 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-28T13:20:00.979 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-28T13:20:00.979 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-28T13:20:00.979 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-28T13:20:00.979 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-28T13:20:00.979 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-28T13:20:00.979 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-28T13:20:00.979 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-28T13:20:00.979 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-28T13:20:00.980 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-28T13:20:00.980 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-28T13:20:00.980 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-28T13:20:00.980 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-28T13:20:00.980 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-28T13:20:00.980 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-28T13:20:00.980 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-28T13:20:00.980 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-28T13:20:00.980 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-28T13:20:00.980 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-28T13:20:00.980 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-28T13:20:00.981 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-28T13:20:00.981 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-28T13:20:00.981 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-28T13:20:00.981 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-28T13:20:00.981 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-28T13:20:00.981 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-28T13:20:00.981 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-28T13:20:00.981 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-28T13:20:00.981 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-28T13:20:00.981 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-28T13:20:00.981 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-28T13:20:00.981 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-28T13:20:00.981 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-28T13:20:00.982 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-28T13:20:00.982 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-28T13:20:00.982 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-28T13:20:00.982 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-28T13:20:00.982 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-28T13:20:00.982 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-28T13:20:00.982 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-28T13:20:00.982 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-28T13:20:00.982 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-28T13:20:00.982 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-28T13:20:00.982 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-28T13:20:00.982 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-28T13:20:00.983 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-28T13:20:00.983 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-28T13:20:00.983 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-28T13:20:00.983 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-28T13:20:00.983 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-28T13:20:00.983 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-28T13:20:00.983 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-28T13:20:00.983 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-28T13:20:00.983 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-28T13:20:00.983 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-28T13:20:00.983 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-28T13:20:00.983 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-28T13:20:00.983 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-28T13:20:00.984 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-28T13:20:00.984 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-28T13:20:00.984 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-28T13:20:00.984 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-28T13:20:00.984 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-28T13:20:00.984 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-28T13:20:00.984 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-28T13:20:00.984 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-28T13:20:00.984 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-28T13:20:00.984 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-28T13:20:00.984 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-28T13:20:00.984 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-28T13:20:00.985 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-28T13:20:00.986 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-28T13:20:00.986 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-28T13:20:00.986 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-28T13:20:00.986 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-28T13:20:00.986 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-28T13:20:00.986 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-28T13:20:00.986 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-28T13:20:00.986 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-28T13:20:00.986 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-28T13:20:00.986 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-28T13:20:00.986 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-28T13:20:00.986 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-28T13:20:00.986 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-28T13:20:00.986 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-28T13:20:00.987 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-28T13:20:00.987 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-28T13:20:00.987 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-28T13:20:00.987 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-28T13:20:00.987 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-28T13:20:00.987 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-28T13:20:00.987 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-28T13:20:00.987 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-28T13:20:00.987 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-28T13:20:00.987 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-28T13:20:00.987 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-28T13:20:00.987 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-28T13:20:00.987 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-28T13:20:00.987 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-28T13:20:00.987 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-28T13:20:00.988 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-28T13:20:00.988 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-28T13:20:00.988 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-28T13:20:00.988 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-28T13:20:00.988 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-28T13:20:00.988 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-28T13:20:00.988 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-28T13:20:00.988 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-28T13:20:00.988 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-28T13:20:00.988 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-28T13:20:00.988 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-28T13:20:00.988 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-28T13:20:00.988 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-28T13:20:00.988 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-28T13:20:00.988 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-28T13:20:00.989 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-28T13:20:00.990 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-28T13:20:00.991 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-28T13:20:00.991 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-28T13:20:00.991 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-28T13:20:00.991 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-28T13:20:00.991 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-28T13:20:00.991 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-28T13:20:00.991 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-28T13:20:00.991 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-28T13:20:00.991 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-28T13:20:00.991 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-28T13:20:00.991 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-28T13:20:00.991 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-28T13:20:00.991 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-28T13:20:00.993 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-28T13:20:00.993 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-28T13:20:00.993 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-28T13:20:00.993 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-28T13:20:00.993 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-28T13:20:00.993 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-28T13:20:00.993 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-28T13:20:00.993 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-28T13:20:00.993 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-28T13:20:00.993 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-28T13:20:00.993 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-28T13:20:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-28T13:20:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-28T13:20:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-28T13:20:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-28T13:20:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-28T13:20:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-28T13:20:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-28T13:20:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-28T13:20:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-28T13:20:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-28T13:20:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-28T13:20:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-28T13:20:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-28T13:20:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-28T13:20:00.994 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-28T13:20:00.995 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-28T13:20:00.995 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-28T13:20:00.995 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-28T13:20:00.995 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-28T13:20:00.995 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-28T13:20:00.995 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-28T13:20:00.995 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-28T13:20:00.995 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-28T13:20:00.995 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-28T13:20:00.995 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-28T13:20:00.995 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-28T13:20:00.995 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-28T13:20:00.995 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-28T13:20:00.996 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-28T13:20:00.996 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-28T13:20:00.996 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-28T13:20:00.996 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-28T13:20:00.996 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-28T13:20:00.996 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-28T13:20:00.996 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-28T13:20:00.996 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-28T13:20:00.996 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-28T13:20:00.996 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-28T13:20:00.996 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-28T13:20:00.996 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-28T13:20:00.996 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-28T13:20:00.997 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-28T13:20:00.997 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-28T13:20:00.997 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-28T13:20:00.997 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-28T13:20:00.997 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-28T13:20:00.997 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-28T13:20:00.997 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-28T13:20:00.997 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-28T13:20:00.997 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-28T13:20:00.997 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-28T13:20:00.997 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-28T13:20:00.997 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-28T13:20:00.997 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-28T13:20:00.998 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-28T13:20:00.998 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-28T13:20:00.998 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-28T13:20:00.998 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-28T13:20:00.998 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-28T13:20:00.998 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-28T13:20:00.998 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-28T13:20:00.998 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-28T13:20:00.998 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-28T13:20:00.998 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-28T13:20:00.998 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-28T13:20:00.999 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-28T13:20:00.999 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-28T13:20:00.999 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-28T13:20:00.999 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-28T13:20:00.999 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-28T13:20:00.999 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-28T13:20:00.999 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-28T13:20:00.999 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-28T13:20:00.999 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-28T13:20:00.999 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-28T13:20:00.999 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-28T13:20:00.999 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-28T13:20:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-28T13:20:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-28T13:20:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-28T13:20:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-28T13:20:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-28T13:20:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-28T13:20:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-28T13:20:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-28T13:20:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-28T13:20:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-28T13:20:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-28T13:20:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-28T13:20:01.001 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-28T13:20:01.001 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-28T13:20:01.001 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-28T13:20:01.001 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-28T13:20:01.001 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-28T13:20:01.001 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-28T13:20:01.001 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-28T13:20:01.001 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-28T13:20:01.001 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-28T13:20:01.001 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-28T13:20:01.001 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-28T13:20:01.001 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-28T13:20:01.002 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-28T13:20:01.002 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-28T13:20:01.002 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-28T13:20:01.002 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-28T13:20:01.002 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-28T13:20:01.002 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-28T13:20:01.002 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-28T13:20:01.002 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-28T13:20:01.002 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-28T13:20:01.002 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-28T13:20:01.002 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-28T13:20:01.002 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-28T13:20:01.003 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-28T13:20:01.003 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-28T13:20:01.003 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-28T13:20:01.003 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-28T13:20:01.003 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-28T13:20:01.003 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-28T13:20:01.003 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-28T13:20:01.003 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-28T13:20:01.003 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-28T13:20:01.003 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-28T13:20:01.003 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-28T13:20:01.003 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-28T13:20:01.003 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-28T13:20:01.003 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-28T13:20:01.004 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-28T13:20:01.004 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-28T13:20:01.004 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-28T13:20:01.004 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-28T13:20:01.004 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-28T13:20:01.004 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-28T13:20:01.004 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-28T13:20:01.004 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-28T13:20:01.004 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-28T13:20:01.004 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-28T13:20:01.004 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-28T13:20:01.004 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-28T13:20:01.004 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-28T13:20:01.004 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-28T13:20:01.004 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-28T13:20:01.005 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-28T13:20:01.005 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-28T13:20:01.005 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-28T13:20:01.005 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-28T13:20:01.005 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-28T13:20:01.005 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-28T13:20:01.005 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-28T13:20:01.005 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-28T13:20:01.005 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-28T13:20:01.005 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-28T13:20:01.005 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-28T13:20:01.005 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-28T13:20:01.006 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-28T13:20:01.006 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-28T13:20:01.006 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-28T13:20:01.006 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-28T13:20:01.006 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-28T13:20:01.006 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-28T13:20:01.006 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-28T13:20:01.006 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-28T13:20:01.006 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-28T13:20:01.006 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-28T13:20:01.006 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-28T13:20:01.006 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-28T13:20:01.006 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-28T13:20:01.007 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-28T13:20:01.007 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-28T13:20:01.007 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-28T13:20:01.007 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-28T13:20:01.007 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-28T13:20:01.007 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-28T13:20:01.007 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-28T13:20:01.007 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-28T13:20:01.007 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-28T13:20:01.007 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-28T13:20:01.007 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-28T13:20:01.007 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-28T13:20:01.008 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-28T13:20:01.008 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-28T13:20:01.008 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-28T13:20:01.008 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-28T13:20:01.008 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-28T13:20:01.008 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-28T13:20:01.008 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-28T13:20:01.008 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-28T13:20:01.008 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-28T13:20:01.008 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-28T13:20:01.008 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-28T13:20:01.008 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-28T13:20:01.008 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-28T13:20:01.009 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-28T13:20:01.009 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-28T13:20:01.009 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-28T13:20:01.009 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-28T13:20:01.009 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-28T13:20:01.009 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-28T13:20:01.009 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-28T13:20:01.009 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-28T13:20:01.009 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-28T13:20:01.009 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-28T13:20:01.009 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-28T13:20:01.010 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-28T13:20:01.010 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-28T13:20:01.010 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-28T13:20:01.010 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-28T13:20:01.010 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-28T13:20:01.010 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-28T13:20:01.010 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-28T13:20:01.010 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-28T13:20:01.010 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-28T13:20:01.010 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-28T13:20:01.010 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-28T13:20:01.011 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-28T13:20:01.011 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-28T13:20:01.011 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-28T13:20:01.011 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-28T13:20:01.011 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-28T13:20:01.011 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-28T13:20:01.011 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-28T13:20:01.011 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-28T13:20:01.011 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-28T13:20:01.011 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-28T13:20:01.011 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-28T13:20:01.011 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-28T13:20:01.012 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-28T13:20:01.012 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-28T13:20:01.012 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-28T13:20:01.012 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-28T13:20:01.012 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-28T13:20:01.012 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-28T13:20:01.012 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-28T13:20:01.012 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-28T13:20:01.012 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-28T13:20:01.012 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-28T13:20:01.012 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-28T13:20:01.012 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-28T13:20:01.013 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-28T13:20:01.013 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-28T13:20:01.013 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-28T13:20:01.013 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-28T13:20:01.013 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-28T13:20:01.013 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-28T13:20:01.013 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-28T13:20:01.013 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-28T13:20:01.013 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-28T13:20:01.013 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-28T13:20:01.014 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-28T13:20:01.014 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-28T13:20:01.014 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-28T13:20:01.014 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-28T13:20:01.014 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-28T13:20:01.014 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-28T13:20:01.014 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-28T13:20:01.014 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-28T13:20:01.014 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-28T13:20:01.014 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-28T13:20:01.014 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-28T13:20:01.015 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-28T13:20:01.015 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-28T13:20:01.015 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-28T13:20:01.015 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-28T13:20:01.015 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-28T13:20:01.015 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-28T13:20:01.015 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-28T13:20:01.015 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-28T13:20:01.015 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-28T13:20:01.015 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-28T13:20:01.015 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-28T13:20:01.015 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-28T13:20:01.015 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-28T13:20:01.016 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-28T13:20:01.016 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-28T13:20:01.016 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-28T13:20:01.016 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-28T13:20:01.016 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-28T13:20:01.016 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-28T13:20:01.016 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-28T13:20:01.016 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-28T13:20:01.016 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-28T13:20:01.016 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-28T13:20:01.016 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-28T13:20:01.016 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-28T13:20:01.016 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-28T13:20:01.017 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-28T13:20:01.017 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-28T13:20:01.017 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-28T13:20:01.017 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-28T13:20:01.017 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-28T13:20:01.017 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-28T13:20:01.017 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-28T13:20:01.017 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-28T13:20:01.017 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-28T13:20:01.017 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-28T13:20:01.017 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-28T13:20:01.017 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-28T13:20:01.017 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-28T13:20:01.017 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-28T13:20:01.017 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-28T13:20:01.018 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-28T13:20:01.018 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-28T13:20:01.018 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-28T13:20:01.018 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-28T13:20:01.018 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-28T13:20:01.018 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-28T13:20:01.018 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-28T13:20:01.018 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-28T13:20:01.018 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-28T13:20:01.018 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-28T13:20:01.018 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-28T13:20:01.018 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-28T13:20:01.018 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-28T13:20:01.018 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-28T13:20:01.019 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-28T13:20:01.019 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-28T13:20:01.019 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-28T13:20:01.019 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-28T13:20:01.019 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-28T13:20:01.019 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-28T13:20:01.019 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-28T13:20:01.019 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-28T13:20:01.019 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-28T13:20:01.019 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-28T13:20:01.019 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-28T13:20:01.019 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-28T13:20:01.019 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-28T13:20:01.019 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-28T13:20:01.020 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-28T13:20:01.020 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-28T13:20:01.020 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-28T13:20:01.020 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-28T13:20:01.020 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-28T13:20:01.020 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-28T13:20:01.020 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-28T13:20:01.020 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-28T13:20:01.020 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-28T13:20:01.020 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-28T13:20:01.020 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-28T13:20:01.020 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-28T13:20:01.020 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-28T13:20:01.020 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-28T13:20:01.020 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-28T13:20:01.021 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-28T13:20:01.021 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-28T13:20:01.021 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-28T13:20:01.021 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-28T13:20:01.021 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-28T13:20:01.021 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-28T13:20:01.021 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-28T13:20:01.021 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-28T13:20:01.021 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-28T13:20:01.021 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-28T13:20:01.021 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-28T13:20:01.021 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-28T13:20:01.021 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-28T13:20:01.021 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-28T13:20:01.022 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-28T13:20:01.022 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-28T13:20:01.022 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-28T13:20:01.022 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-28T13:20:01.022 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-28T13:20:01.022 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-28T13:20:01.022 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-28T13:20:01.022 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-28T13:20:01.022 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-28T13:20:01.022 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-28T13:20:01.022 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-28T13:20:01.022 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-28T13:20:01.022 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-28T13:20:01.022 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-28T13:20:01.023 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-28T13:20:01.023 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-28T13:20:01.023 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-28T13:20:01.023 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-28T13:20:01.023 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-28T13:20:01.023 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-28T13:20:01.023 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-28T13:20:01.023 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-28T13:20:01.023 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-28T13:20:01.023 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-28T13:20:01.023 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-28T13:20:01.023 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-28T13:20:01.023 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-28T13:20:01.023 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-28T13:20:01.023 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-28T13:20:01.024 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-28T13:20:01.024 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-28T13:20:01.024 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-28T13:20:01.024 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-28T13:20:01.024 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-28T13:20:01.024 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-28T13:20:01.024 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-28T13:20:01.024 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-28T13:20:01.024 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-28T13:20:01.024 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-28T13:20:01.024 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-28T13:20:01.024 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-28T13:20:01.024 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-28T13:20:01.024 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-28T13:20:01.025 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-28T13:20:01.025 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-28T13:20:01.025 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-28T13:20:01.025 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-28T13:20:01.025 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-28T13:20:01.025 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-28T13:20:01.025 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-28T13:20:01.025 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-28T13:20:01.025 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-28T13:20:01.025 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-28T13:20:01.025 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-28T13:20:01.025 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-28T13:20:01.025 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-28T13:20:01.025 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-28T13:20:01.026 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-28T13:20:01.026 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-28T13:20:01.026 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-28T13:20:01.026 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-28T13:20:01.026 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-28T13:20:01.026 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-28T13:20:01.026 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-28T13:20:01.026 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-28T13:20:01.026 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-28T13:20:01.026 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-28T13:20:01.026 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-28T13:20:01.026 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-28T13:20:01.027 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-28T13:20:01.027 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-28T13:20:01.027 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-28T13:20:01.027 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-28T13:20:01.027 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-28T13:20:01.027 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-28T13:20:01.027 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-28T13:20:01.027 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-28T13:20:01.027 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-28T13:20:01.027 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-28T13:20:01.027 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-28T13:20:01.027 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-28T13:20:01.028 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-28T13:20:01.028 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-28T13:20:01.028 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-28T13:20:01.028 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-28T13:20:01.028 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-28T13:20:01.028 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-28T13:20:01.028 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-28T13:20:01.028 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-28T13:20:01.028 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-28T13:20:01.028 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-28T13:20:01.028 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-28T13:20:01.028 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-28T13:20:01.029 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-28T13:20:01.029 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-28T13:20:01.029 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-28T13:20:01.029 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-28T13:20:01.029 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-28T13:20:01.029 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-28T13:20:01.029 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-28T13:20:01.029 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-28T13:20:01.029 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-28T13:20:01.029 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-28T13:20:01.029 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-28T13:20:01.029 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-28T13:20:01.030 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-28T13:20:01.030 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-28T13:20:01.030 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-28T13:20:01.030 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-28T13:20:01.030 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-28T13:20:01.030 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-28T13:20:01.034 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:01.035 DEBUG:teuthology.orchestra.run.smithi203:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-28T13:20:01.058 INFO:teuthology.orchestra.run.smithi203.stdout:postinstall scriptlet (using /bin/sh): 2024-04-28T13:20:01.058 INFO:teuthology.orchestra.run.smithi203.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-28T13:20:01.058 INFO:teuthology.orchestra.run.smithi203.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-28T13:20:01.058 INFO:teuthology.orchestra.run.smithi203.stdout:fi 2024-04-28T13:20:01.058 INFO:teuthology.orchestra.run.smithi203.stdout:for file in vmlinuz System.map config; do 2024-04-28T13:20:01.058 INFO:teuthology.orchestra.run.smithi203.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-28T13:20:01.058 INFO:teuthology.orchestra.run.smithi203.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-28T13:20:01.059 INFO:teuthology.orchestra.run.smithi203.stdout: fi 2024-04-28T13:20:01.059 INFO:teuthology.orchestra.run.smithi203.stdout:done 2024-04-28T13:20:01.059 INFO:teuthology.orchestra.run.smithi203.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-28T13:20:01.059 INFO:teuthology.orchestra.run.smithi203.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-28T13:20:01.059 INFO:teuthology.orchestra.run.smithi203.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-28T13:20:01.059 INFO:teuthology.orchestra.run.smithi203.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-28T13:20:01.059 INFO:teuthology.orchestra.run.smithi203.stdout:fi 2024-04-28T13:20:01.059 INFO:teuthology.orchestra.run.smithi203.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-28T13:20:01.059 INFO:teuthology.orchestra.run.smithi203.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:01.059 INFO:teuthology.orchestra.run.smithi203.stdout:fi 2024-04-28T13:20:01.059 INFO:teuthology.orchestra.run.smithi203.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-28T13:20:01.060 INFO:teuthology.orchestra.run.smithi203.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-28T13:20:01.060 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:01.060 INFO:teuthology.orchestra.run.smithi203.stdout:fi 2024-04-28T13:20:01.061 DEBUG:teuthology.orchestra.run.smithi203:> sudo rpm -qi grub2-tools 2024-04-28T13:20:01.142 INFO:teuthology.orchestra.run.smithi203.stdout:Name : grub2-tools 2024-04-28T13:20:01.143 INFO:teuthology.orchestra.run.smithi203.stdout:Epoch : 1 2024-04-28T13:20:01.143 INFO:teuthology.orchestra.run.smithi203.stdout:Version : 2.06 2024-04-28T13:20:01.143 INFO:teuthology.orchestra.run.smithi203.stdout:Release : 68.el9 2024-04-28T13:20:01.143 INFO:teuthology.orchestra.run.smithi203.stdout:Architecture: x86_64 2024-04-28T13:20:01.143 INFO:teuthology.orchestra.run.smithi203.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-28T13:20:01.143 INFO:teuthology.orchestra.run.smithi203.stdout:Group : Unspecified 2024-04-28T13:20:01.143 INFO:teuthology.orchestra.run.smithi203.stdout:Size : 8264254 2024-04-28T13:20:01.143 INFO:teuthology.orchestra.run.smithi203.stdout:License : GPLv3+ 2024-04-28T13:20:01.143 INFO:teuthology.orchestra.run.smithi203.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-28T13:20:01.143 INFO:teuthology.orchestra.run.smithi203.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-28T13:20:01.144 INFO:teuthology.orchestra.run.smithi203.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-28T13:20:01.144 INFO:teuthology.orchestra.run.smithi203.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-28T13:20:01.144 INFO:teuthology.orchestra.run.smithi203.stdout:Packager : builder@centos.org 2024-04-28T13:20:01.144 INFO:teuthology.orchestra.run.smithi203.stdout:Vendor : CentOS 2024-04-28T13:20:01.144 INFO:teuthology.orchestra.run.smithi203.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-28T13:20:01.144 INFO:teuthology.orchestra.run.smithi203.stdout:Summary : Support tools for GRUB. 2024-04-28T13:20:01.144 INFO:teuthology.orchestra.run.smithi203.stdout:Description : 2024-04-28T13:20:01.144 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:20:01.144 INFO:teuthology.orchestra.run.smithi203.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-28T13:20:01.144 INFO:teuthology.orchestra.run.smithi203.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-28T13:20:01.145 INFO:teuthology.orchestra.run.smithi203.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-28T13:20:01.145 INFO:teuthology.orchestra.run.smithi203.stdout:hardware devices. 2024-04-28T13:20:01.145 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:20:01.145 INFO:teuthology.orchestra.run.smithi203.stdout:This subpackage provides tools for support of all platforms. 2024-04-28T13:20:01.146 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-28T13:20:01.146 INFO:teuthology.task.kernel:Updating grub on smithi203 to boot 6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:01.146 DEBUG:teuthology.orchestra.run.smithi203:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-28T13:20:01.712 INFO:teuthology.orchestra.run.smithi203.stderr:Generating grub configuration file ... 2024-04-28T13:20:03.053 INFO:teuthology.orchestra.run.smithi203.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-28T13:20:03.084 INFO:teuthology.orchestra.run.smithi203.stderr:done 2024-04-28T13:20:03.087 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-04-28T13:20:03.101 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.qopHhjtQgJ 2024-04-28T13:20:03.102 DEBUG:teuthology.orchestra.run.smithi203:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.qopHhjtQgJ 2024-04-28T13:20:03.175 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.qopHhjtQgJ 2024-04-28T13:20:03.313 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.qopHhjtQgJ is 6KB 2024-04-28T13:20:03.361 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.qopHhjtQgJ 2024-04-28T13:20:03.375 DEBUG:teuthology.orchestra.run.smithi203:> sudo /bin/ls /boot/loader/entries || true 2024-04-28T13:20:03.437 INFO:teuthology.orchestra.run.smithi203.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-28T13:20:03.437 INFO:teuthology.orchestra.run.smithi203.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-28T13:20:03.438 INFO:teuthology.orchestra.run.smithi203.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-28T13:20:03.438 INFO:teuthology.orchestra.run.smithi203.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-28T13:20:03.439 DEBUG:teuthology.orchestra.run.smithi203:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:03.580 DEBUG:teuthology.orchestra.run.smithi203:> sudo shutdown -r now 2024-04-28T13:20:07.048 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-28T13:20:33.583 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-28T13:20:33.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-28T13:20:33.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T13:20:44.102 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi196.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-28T13:20:44.102 DEBUG:teuthology.orchestra.run.smithi196:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-28T13:20:44.181 INFO:teuthology.orchestra.run.smithi196.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:44.181 INFO:teuthology.orchestra.run.smithi196.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:44.182 INFO:teuthology.orchestra.run.smithi196.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:44.182 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-28T13:20:44.182 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-28T13:20:44.182 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-28T13:20:44.182 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-28T13:20:44.182 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-28T13:20:44.182 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-28T13:20:44.182 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-28T13:20:44.182 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-28T13:20:44.182 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-28T13:20:44.182 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-28T13:20:44.182 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-28T13:20:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-28T13:20:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-28T13:20:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-28T13:20:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-28T13:20:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-28T13:20:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-28T13:20:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-28T13:20:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-28T13:20:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-28T13:20:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-28T13:20:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-28T13:20:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-28T13:20:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-28T13:20:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-28T13:20:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-28T13:20:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-28T13:20:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-28T13:20:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-28T13:20:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-28T13:20:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-28T13:20:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-28T13:20:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-28T13:20:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-28T13:20:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-28T13:20:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-28T13:20:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-28T13:20:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-28T13:20:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-28T13:20:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-28T13:20:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-28T13:20:44.185 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-28T13:20:44.185 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-28T13:20:44.185 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-28T13:20:44.185 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-28T13:20:44.185 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-28T13:20:44.185 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-28T13:20:44.185 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-28T13:20:44.185 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-28T13:20:44.185 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-28T13:20:44.185 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-28T13:20:44.185 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-28T13:20:44.185 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-28T13:20:44.185 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-28T13:20:44.186 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-28T13:20:44.186 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-28T13:20:44.186 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-28T13:20:44.186 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-28T13:20:44.186 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-28T13:20:44.186 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-28T13:20:44.186 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-28T13:20:44.186 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-28T13:20:44.186 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-28T13:20:44.186 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-28T13:20:44.186 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-28T13:20:44.186 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-28T13:20:44.186 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-28T13:20:44.186 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-28T13:20:44.187 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-28T13:20:44.188 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-28T13:20:44.188 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-28T13:20:44.188 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-28T13:20:44.188 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-28T13:20:44.188 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-28T13:20:44.188 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-28T13:20:44.188 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-28T13:20:44.188 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-28T13:20:44.188 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-28T13:20:44.188 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-28T13:20:44.188 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-28T13:20:44.188 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-28T13:20:44.188 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-28T13:20:44.188 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-28T13:20:44.188 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-28T13:20:44.189 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-28T13:20:44.189 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-28T13:20:44.189 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-28T13:20:44.189 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-28T13:20:44.189 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-28T13:20:44.189 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-28T13:20:44.189 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-28T13:20:44.189 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-28T13:20:44.189 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-28T13:20:44.189 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-28T13:20:44.189 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-28T13:20:44.189 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-28T13:20:44.189 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-28T13:20:44.189 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-28T13:20:44.189 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-28T13:20:44.190 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-28T13:20:44.190 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-28T13:20:44.190 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-28T13:20:44.190 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-28T13:20:44.190 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-28T13:20:44.190 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-28T13:20:44.190 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-28T13:20:44.190 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-28T13:20:44.190 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-28T13:20:44.190 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-28T13:20:44.190 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-28T13:20:44.190 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-28T13:20:44.190 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-28T13:20:44.190 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-28T13:20:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-28T13:20:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-28T13:20:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-28T13:20:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-28T13:20:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-28T13:20:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-28T13:20:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-28T13:20:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-28T13:20:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-28T13:20:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-28T13:20:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-28T13:20:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-28T13:20:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-28T13:20:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-28T13:20:44.191 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-28T13:20:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-28T13:20:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-28T13:20:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-28T13:20:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-28T13:20:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-28T13:20:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-28T13:20:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-28T13:20:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-28T13:20:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-28T13:20:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-28T13:20:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-28T13:20:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-28T13:20:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-28T13:20:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-28T13:20:44.192 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-28T13:20:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-28T13:20:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-28T13:20:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-28T13:20:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-28T13:20:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-28T13:20:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-28T13:20:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-28T13:20:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-28T13:20:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-28T13:20:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-28T13:20:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-28T13:20:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-28T13:20:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-28T13:20:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-28T13:20:44.193 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-28T13:20:44.194 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-28T13:20:44.194 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-28T13:20:44.194 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-28T13:20:44.194 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-28T13:20:44.194 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-28T13:20:44.194 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-28T13:20:44.194 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-28T13:20:44.194 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-28T13:20:44.194 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-28T13:20:44.194 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-28T13:20:44.194 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-28T13:20:44.194 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-28T13:20:44.194 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-28T13:20:44.194 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-28T13:20:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-28T13:20:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-28T13:20:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-28T13:20:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-28T13:20:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-28T13:20:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-28T13:20:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-28T13:20:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-28T13:20:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-28T13:20:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-28T13:20:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-28T13:20:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-28T13:20:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-28T13:20:44.195 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-28T13:20:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-28T13:20:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-28T13:20:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-28T13:20:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-28T13:20:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-28T13:20:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-28T13:20:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-28T13:20:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-28T13:20:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-28T13:20:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-28T13:20:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-28T13:20:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-28T13:20:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-28T13:20:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-28T13:20:44.196 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-28T13:20:44.197 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-28T13:20:44.197 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-28T13:20:44.197 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-28T13:20:44.197 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-28T13:20:44.197 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-28T13:20:44.197 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-28T13:20:44.197 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-28T13:20:44.197 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-28T13:20:44.197 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-28T13:20:44.197 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-28T13:20:44.197 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-28T13:20:44.197 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-28T13:20:44.197 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-28T13:20:44.197 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-28T13:20:44.197 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-28T13:20:44.198 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-28T13:20:44.198 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-28T13:20:44.198 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-28T13:20:44.198 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-28T13:20:44.198 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-28T13:20:44.198 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-28T13:20:44.198 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-28T13:20:44.198 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-28T13:20:44.198 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-28T13:20:44.198 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-28T13:20:44.198 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-28T13:20:44.198 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-28T13:20:44.198 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-28T13:20:44.198 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-28T13:20:44.198 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-28T13:20:44.199 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-28T13:20:44.199 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-28T13:20:44.199 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-28T13:20:44.199 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-28T13:20:44.199 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-28T13:20:44.199 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-28T13:20:44.199 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-28T13:20:44.199 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-28T13:20:44.199 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-28T13:20:44.199 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-28T13:20:44.199 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-28T13:20:44.199 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-28T13:20:44.199 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-28T13:20:44.199 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-28T13:20:44.200 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-28T13:20:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-28T13:20:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-28T13:20:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-28T13:20:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-28T13:20:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-28T13:20:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-28T13:20:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-28T13:20:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-28T13:20:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-28T13:20:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-28T13:20:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-28T13:20:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-28T13:20:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-28T13:20:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-28T13:20:44.201 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-28T13:20:44.202 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-28T13:20:44.202 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-28T13:20:44.202 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-28T13:20:44.202 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-28T13:20:44.202 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-28T13:20:44.202 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-28T13:20:44.202 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-28T13:20:44.202 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-28T13:20:44.202 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-28T13:20:44.202 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-28T13:20:44.202 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-28T13:20:44.202 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-28T13:20:44.202 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-28T13:20:44.202 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-28T13:20:44.202 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-28T13:20:44.203 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-28T13:20:44.203 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-28T13:20:44.203 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-28T13:20:44.203 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-28T13:20:44.203 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-28T13:20:44.203 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-28T13:20:44.203 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-28T13:20:44.203 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-28T13:20:44.203 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-28T13:20:44.203 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-28T13:20:44.203 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-28T13:20:44.203 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-28T13:20:44.203 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-28T13:20:44.203 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-28T13:20:44.203 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-28T13:20:44.204 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-28T13:20:44.204 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-28T13:20:44.204 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-28T13:20:44.204 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-28T13:20:44.204 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-28T13:20:44.204 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-28T13:20:44.204 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-28T13:20:44.204 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-28T13:20:44.204 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-28T13:20:44.204 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-28T13:20:44.204 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-28T13:20:44.204 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-28T13:20:44.204 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-28T13:20:44.204 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-28T13:20:44.204 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-28T13:20:44.205 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-28T13:20:44.205 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-28T13:20:44.205 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-28T13:20:44.205 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-28T13:20:44.205 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-28T13:20:44.205 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-28T13:20:44.205 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-28T13:20:44.205 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-28T13:20:44.205 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-28T13:20:44.205 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-28T13:20:44.205 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-28T13:20:44.205 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-28T13:20:44.205 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-28T13:20:44.205 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-28T13:20:44.205 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-28T13:20:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-28T13:20:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-28T13:20:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-28T13:20:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-28T13:20:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-28T13:20:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-28T13:20:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-28T13:20:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-28T13:20:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-28T13:20:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-28T13:20:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-28T13:20:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-28T13:20:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-28T13:20:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-28T13:20:44.206 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-28T13:20:44.207 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-28T13:20:44.207 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-28T13:20:44.207 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-28T13:20:44.207 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-28T13:20:44.207 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-28T13:20:44.207 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-28T13:20:44.207 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-28T13:20:44.207 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-28T13:20:44.207 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-28T13:20:44.207 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-28T13:20:44.207 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-28T13:20:44.207 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-28T13:20:44.207 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-28T13:20:44.207 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-28T13:20:44.207 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-28T13:20:44.208 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-28T13:20:44.209 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-28T13:20:44.209 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-28T13:20:44.209 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-28T13:20:44.209 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-28T13:20:44.209 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-28T13:20:44.209 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-28T13:20:44.209 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-28T13:20:44.209 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-28T13:20:44.209 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-28T13:20:44.209 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-28T13:20:44.209 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-28T13:20:44.209 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-28T13:20:44.209 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-28T13:20:44.209 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-28T13:20:44.209 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-28T13:20:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-28T13:20:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-28T13:20:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-28T13:20:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-28T13:20:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-28T13:20:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-28T13:20:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-28T13:20:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-28T13:20:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-28T13:20:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-28T13:20:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-28T13:20:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-28T13:20:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-28T13:20:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-28T13:20:44.210 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-28T13:20:44.211 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-28T13:20:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-28T13:20:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-28T13:20:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-28T13:20:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-28T13:20:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-28T13:20:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-28T13:20:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-28T13:20:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-28T13:20:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-28T13:20:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-28T13:20:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-28T13:20:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-28T13:20:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-28T13:20:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-28T13:20:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-28T13:20:44.213 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-28T13:20:44.213 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-28T13:20:44.213 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-28T13:20:44.213 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-28T13:20:44.213 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-28T13:20:44.213 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-28T13:20:44.213 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-28T13:20:44.213 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-28T13:20:44.213 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-28T13:20:44.213 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-28T13:20:44.213 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-28T13:20:44.213 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-28T13:20:44.213 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-28T13:20:44.213 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-28T13:20:44.213 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-28T13:20:44.214 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-28T13:20:44.214 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-28T13:20:44.214 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-28T13:20:44.214 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-28T13:20:44.214 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-28T13:20:44.214 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-28T13:20:44.214 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-28T13:20:44.214 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-28T13:20:44.214 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-28T13:20:44.214 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-28T13:20:44.214 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-28T13:20:44.214 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-28T13:20:44.214 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-28T13:20:44.214 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-28T13:20:44.214 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-28T13:20:44.215 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-28T13:20:44.215 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-28T13:20:44.215 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-28T13:20:44.215 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-28T13:20:44.215 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-28T13:20:44.215 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-28T13:20:44.215 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-28T13:20:44.215 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-28T13:20:44.215 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-28T13:20:44.215 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-28T13:20:44.215 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-28T13:20:44.215 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-28T13:20:44.215 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-28T13:20:44.215 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-28T13:20:44.215 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-28T13:20:44.216 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-28T13:20:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-28T13:20:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-28T13:20:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-28T13:20:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-28T13:20:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-28T13:20:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-28T13:20:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-28T13:20:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-28T13:20:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-28T13:20:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-28T13:20:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-28T13:20:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-28T13:20:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-28T13:20:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-28T13:20:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-28T13:20:44.218 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-28T13:20:44.218 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-28T13:20:44.218 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-28T13:20:44.218 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-28T13:20:44.218 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-28T13:20:44.218 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-28T13:20:44.218 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-28T13:20:44.218 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-28T13:20:44.218 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-28T13:20:44.218 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-28T13:20:44.218 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-28T13:20:44.218 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-28T13:20:44.218 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-28T13:20:44.218 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-28T13:20:44.218 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-28T13:20:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-28T13:20:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-28T13:20:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-28T13:20:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-28T13:20:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-28T13:20:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-28T13:20:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-28T13:20:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-28T13:20:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-28T13:20:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-28T13:20:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-28T13:20:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-28T13:20:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-28T13:20:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-28T13:20:44.219 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-28T13:20:44.220 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-28T13:20:44.220 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-28T13:20:44.220 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-28T13:20:44.220 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-28T13:20:44.220 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-28T13:20:44.220 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-28T13:20:44.220 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-28T13:20:44.220 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-28T13:20:44.220 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-28T13:20:44.220 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-28T13:20:44.220 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-28T13:20:44.220 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-28T13:20:44.220 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-28T13:20:44.220 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-28T13:20:44.220 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-28T13:20:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-28T13:20:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-28T13:20:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-28T13:20:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-28T13:20:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-28T13:20:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-28T13:20:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-28T13:20:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-28T13:20:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-28T13:20:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-28T13:20:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-28T13:20:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-28T13:20:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-28T13:20:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-28T13:20:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-28T13:20:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-28T13:20:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-28T13:20:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-28T13:20:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-28T13:20:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-28T13:20:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-28T13:20:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-28T13:20:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-28T13:20:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-28T13:20:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-28T13:20:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-28T13:20:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-28T13:20:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-28T13:20:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-28T13:20:44.222 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-28T13:20:44.223 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-28T13:20:44.223 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-28T13:20:44.223 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-28T13:20:44.223 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-28T13:20:44.223 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-28T13:20:44.223 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-28T13:20:44.223 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-28T13:20:44.223 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-28T13:20:44.223 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-28T13:20:44.223 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-28T13:20:44.223 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-28T13:20:44.223 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-28T13:20:44.223 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-28T13:20:44.223 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-28T13:20:44.223 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-28T13:20:44.224 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-28T13:20:44.224 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-28T13:20:44.224 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-28T13:20:44.224 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-28T13:20:44.224 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-28T13:20:44.224 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-28T13:20:44.224 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-28T13:20:44.224 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-28T13:20:44.224 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-28T13:20:44.224 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-28T13:20:44.224 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-28T13:20:44.224 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-28T13:20:44.224 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-28T13:20:44.224 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-28T13:20:44.224 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-28T13:20:44.225 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-28T13:20:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-28T13:20:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-28T13:20:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-28T13:20:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-28T13:20:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-28T13:20:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-28T13:20:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-28T13:20:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-28T13:20:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-28T13:20:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-28T13:20:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-28T13:20:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-28T13:20:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-28T13:20:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-28T13:20:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-28T13:20:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-28T13:20:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-28T13:20:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-28T13:20:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-28T13:20:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-28T13:20:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-28T13:20:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-28T13:20:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-28T13:20:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-28T13:20:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-28T13:20:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-28T13:20:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-28T13:20:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-28T13:20:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-28T13:20:44.227 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-28T13:20:44.228 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-28T13:20:44.228 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-28T13:20:44.228 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-28T13:20:44.228 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-28T13:20:44.228 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-28T13:20:44.228 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-28T13:20:44.228 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-28T13:20:44.228 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-28T13:20:44.228 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-28T13:20:44.228 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-28T13:20:44.228 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-28T13:20:44.228 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-28T13:20:44.228 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-28T13:20:44.228 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-28T13:20:44.228 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-28T13:20:44.229 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-28T13:20:44.230 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-28T13:20:44.230 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-28T13:20:44.230 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-28T13:20:44.230 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-28T13:20:44.230 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-28T13:20:44.230 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-28T13:20:44.230 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-28T13:20:44.230 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-28T13:20:44.230 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-28T13:20:44.230 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-28T13:20:44.230 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-28T13:20:44.230 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-28T13:20:44.230 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-28T13:20:44.230 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-28T13:20:44.230 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-28T13:20:44.231 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-28T13:20:44.231 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-28T13:20:44.231 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-28T13:20:44.231 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-28T13:20:44.231 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-28T13:20:44.231 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-28T13:20:44.231 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-28T13:20:44.231 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-28T13:20:44.231 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-28T13:20:44.231 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-28T13:20:44.231 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-28T13:20:44.231 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-28T13:20:44.231 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-28T13:20:44.231 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-28T13:20:44.231 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-28T13:20:44.232 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-28T13:20:44.232 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-28T13:20:44.232 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-28T13:20:44.232 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-28T13:20:44.232 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-28T13:20:44.232 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-28T13:20:44.232 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-28T13:20:44.232 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-28T13:20:44.232 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-28T13:20:44.232 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-28T13:20:44.232 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-28T13:20:44.232 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-28T13:20:44.232 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-28T13:20:44.232 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-28T13:20:44.232 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-28T13:20:44.233 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-28T13:20:44.234 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-28T13:20:44.234 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-28T13:20:44.234 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-28T13:20:44.234 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-28T13:20:44.234 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-28T13:20:44.234 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-28T13:20:44.234 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-28T13:20:44.234 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-28T13:20:44.234 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-28T13:20:44.234 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-28T13:20:44.234 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-28T13:20:44.234 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-28T13:20:44.234 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-28T13:20:44.234 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-28T13:20:44.234 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-28T13:20:44.235 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-28T13:20:44.235 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-28T13:20:44.235 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-28T13:20:44.235 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-28T13:20:44.235 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-28T13:20:44.235 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-28T13:20:44.235 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-28T13:20:44.235 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-28T13:20:44.235 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-28T13:20:44.235 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-28T13:20:44.235 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-28T13:20:44.235 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-28T13:20:44.235 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-28T13:20:44.235 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-28T13:20:44.235 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-28T13:20:44.236 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-28T13:20:44.236 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-28T13:20:44.236 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-28T13:20:44.236 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-28T13:20:44.236 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-28T13:20:44.236 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-28T13:20:44.236 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-28T13:20:44.236 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-28T13:20:44.236 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-28T13:20:44.236 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-28T13:20:44.236 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-28T13:20:44.236 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-28T13:20:44.236 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-28T13:20:44.236 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-28T13:20:44.236 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-28T13:20:44.237 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-28T13:20:44.237 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-28T13:20:44.237 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-28T13:20:44.237 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-28T13:20:44.237 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-28T13:20:44.237 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-28T13:20:44.237 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-28T13:20:44.237 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-28T13:20:44.237 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-28T13:20:44.237 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-28T13:20:44.237 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-28T13:20:44.237 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-28T13:20:44.237 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-28T13:20:44.237 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-28T13:20:44.237 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-28T13:20:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-28T13:20:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-28T13:20:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-28T13:20:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-28T13:20:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-28T13:20:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-28T13:20:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-28T13:20:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-28T13:20:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-28T13:20:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-28T13:20:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-28T13:20:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-28T13:20:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-28T13:20:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-28T13:20:44.238 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-28T13:20:44.239 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-28T13:20:44.239 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-28T13:20:44.239 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-28T13:20:44.239 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-28T13:20:44.239 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-28T13:20:44.239 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-28T13:20:44.239 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-28T13:20:44.239 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-28T13:20:44.239 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-28T13:20:44.239 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-28T13:20:44.239 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-28T13:20:44.239 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-28T13:20:44.239 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-28T13:20:44.239 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-28T13:20:44.239 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-28T13:20:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-28T13:20:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-28T13:20:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-28T13:20:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-28T13:20:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-28T13:20:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-28T13:20:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-28T13:20:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-28T13:20:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-28T13:20:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-28T13:20:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-28T13:20:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-28T13:20:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-28T13:20:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-28T13:20:44.240 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-28T13:20:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-28T13:20:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-28T13:20:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-28T13:20:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-28T13:20:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-28T13:20:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-28T13:20:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-28T13:20:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-28T13:20:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-28T13:20:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-28T13:20:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-28T13:20:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-28T13:20:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-28T13:20:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-28T13:20:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-28T13:20:44.242 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-28T13:20:44.242 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-28T13:20:44.242 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-28T13:20:44.242 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-28T13:20:44.242 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-28T13:20:44.242 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-28T13:20:44.242 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-28T13:20:44.242 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-28T13:20:44.242 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-28T13:20:44.242 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-28T13:20:44.242 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-28T13:20:44.242 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-28T13:20:44.242 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-28T13:20:44.242 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-28T13:20:44.243 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-28T13:20:44.243 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-28T13:20:44.243 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-28T13:20:44.243 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-28T13:20:44.243 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-28T13:20:44.243 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-28T13:20:44.243 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-28T13:20:44.243 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-28T13:20:44.243 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-28T13:20:44.243 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-28T13:20:44.243 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-28T13:20:44.243 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-28T13:20:44.243 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-28T13:20:44.243 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-28T13:20:44.243 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-28T13:20:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-28T13:20:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-28T13:20:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-28T13:20:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-28T13:20:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-28T13:20:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-28T13:20:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-28T13:20:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-28T13:20:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-28T13:20:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-28T13:20:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-28T13:20:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-28T13:20:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-28T13:20:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-28T13:20:44.244 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-28T13:20:44.245 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-28T13:20:44.246 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-28T13:20:44.246 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-28T13:20:44.246 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-28T13:20:44.246 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-28T13:20:44.246 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-28T13:20:44.246 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-28T13:20:44.246 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-28T13:20:44.246 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-28T13:20:44.246 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-28T13:20:44.246 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-28T13:20:44.246 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-28T13:20:44.246 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-28T13:20:44.246 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-28T13:20:44.246 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-28T13:20:44.247 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-28T13:20:44.248 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-28T13:20:44.249 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-28T13:20:44.249 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-28T13:20:44.249 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-28T13:20:44.249 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-28T13:20:44.249 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-28T13:20:44.249 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-28T13:20:44.249 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-28T13:20:44.249 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-28T13:20:44.249 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-28T13:20:44.249 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-28T13:20:44.249 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-28T13:20:44.249 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-28T13:20:44.249 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-28T13:20:44.249 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-28T13:20:44.249 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-28T13:20:44.250 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-28T13:20:44.250 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-28T13:20:44.250 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-28T13:20:44.250 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-28T13:20:44.250 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-28T13:20:44.250 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-28T13:20:44.250 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-28T13:20:44.250 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-28T13:20:44.250 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-28T13:20:44.250 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-28T13:20:44.250 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-28T13:20:44.250 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-28T13:20:44.250 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-28T13:20:44.250 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-28T13:20:44.250 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-28T13:20:44.251 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-28T13:20:44.251 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-28T13:20:44.251 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-28T13:20:44.251 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-28T13:20:44.251 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-28T13:20:44.251 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-28T13:20:44.251 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-28T13:20:44.251 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-28T13:20:44.251 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-28T13:20:44.251 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-28T13:20:44.251 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-28T13:20:44.251 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-28T13:20:44.251 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-28T13:20:44.251 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-28T13:20:44.251 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-28T13:20:44.252 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-28T13:20:44.252 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-28T13:20:44.252 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-28T13:20:44.252 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-28T13:20:44.252 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-28T13:20:44.252 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-28T13:20:44.252 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-28T13:20:44.252 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-28T13:20:44.252 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-28T13:20:44.252 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-28T13:20:44.252 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-28T13:20:44.252 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-28T13:20:44.252 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-28T13:20:44.252 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-28T13:20:44.252 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-28T13:20:44.253 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-28T13:20:44.253 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-28T13:20:44.253 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-28T13:20:44.253 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-28T13:20:44.253 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-28T13:20:44.253 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-28T13:20:44.253 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-28T13:20:44.253 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-28T13:20:44.253 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-28T13:20:44.253 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-28T13:20:44.253 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-28T13:20:44.253 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-28T13:20:44.253 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-28T13:20:44.253 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-28T13:20:44.253 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-28T13:20:44.254 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-28T13:20:44.254 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-28T13:20:44.254 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-28T13:20:44.254 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-28T13:20:44.254 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-28T13:20:44.254 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-28T13:20:44.254 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-28T13:20:44.254 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-28T13:20:44.254 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-28T13:20:44.254 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-28T13:20:44.254 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-28T13:20:44.254 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-28T13:20:44.254 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-28T13:20:44.254 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-28T13:20:44.254 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-28T13:20:44.255 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-28T13:20:44.255 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-28T13:20:44.255 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-28T13:20:44.255 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-28T13:20:44.255 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-28T13:20:44.255 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-28T13:20:44.255 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-28T13:20:44.255 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-28T13:20:44.255 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-28T13:20:44.255 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-28T13:20:44.255 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-28T13:20:44.255 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-28T13:20:44.255 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-28T13:20:44.255 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-28T13:20:44.255 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-28T13:20:44.256 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-28T13:20:44.256 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-28T13:20:44.256 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-28T13:20:44.256 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-28T13:20:44.256 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-28T13:20:44.256 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-28T13:20:44.256 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-28T13:20:44.256 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-28T13:20:44.256 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-28T13:20:44.256 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-28T13:20:44.256 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-28T13:20:44.256 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-28T13:20:44.256 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-28T13:20:44.256 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-28T13:20:44.256 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-28T13:20:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-28T13:20:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-28T13:20:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-28T13:20:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-28T13:20:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-28T13:20:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-28T13:20:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-28T13:20:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-28T13:20:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-28T13:20:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-28T13:20:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-28T13:20:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-28T13:20:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-28T13:20:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-28T13:20:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-28T13:20:44.258 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-28T13:20:44.258 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-28T13:20:44.258 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-28T13:20:44.258 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-28T13:20:44.258 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-28T13:20:44.258 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-28T13:20:44.258 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-28T13:20:44.258 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-28T13:20:44.258 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-28T13:20:44.258 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-28T13:20:44.258 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-28T13:20:44.258 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-28T13:20:44.258 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-28T13:20:44.258 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-28T13:20:44.258 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-28T13:20:44.259 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-28T13:20:44.259 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-28T13:20:44.259 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-28T13:20:44.259 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-28T13:20:44.259 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-28T13:20:44.259 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-28T13:20:44.259 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-28T13:20:44.259 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-28T13:20:44.259 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-28T13:20:44.259 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-28T13:20:44.259 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-28T13:20:44.259 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-28T13:20:44.259 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-28T13:20:44.259 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-28T13:20:44.259 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-28T13:20:44.260 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-28T13:20:44.260 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-28T13:20:44.260 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-28T13:20:44.260 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-28T13:20:44.260 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-28T13:20:44.260 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-28T13:20:44.260 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-28T13:20:44.260 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-28T13:20:44.260 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-28T13:20:44.260 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-28T13:20:44.260 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-28T13:20:44.260 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-28T13:20:44.260 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-28T13:20:44.260 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-28T13:20:44.260 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-28T13:20:44.261 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-28T13:20:44.261 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-28T13:20:44.261 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-28T13:20:44.261 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-28T13:20:44.261 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-28T13:20:44.261 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-28T13:20:44.261 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-28T13:20:44.261 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-28T13:20:44.261 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-28T13:20:44.261 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-28T13:20:44.261 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-28T13:20:44.261 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-28T13:20:44.261 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-28T13:20:44.261 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-28T13:20:44.261 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-28T13:20:44.262 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-28T13:20:44.262 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-28T13:20:44.262 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-28T13:20:44.262 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-28T13:20:44.262 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-28T13:20:44.262 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-28T13:20:44.262 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-28T13:20:44.262 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-28T13:20:44.262 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-28T13:20:44.262 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-28T13:20:44.262 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-28T13:20:44.262 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-28T13:20:44.262 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-28T13:20:44.262 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-28T13:20:44.262 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-28T13:20:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-28T13:20:44.264 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-28T13:20:44.264 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-28T13:20:44.264 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-28T13:20:44.264 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-28T13:20:44.264 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-28T13:20:44.264 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-28T13:20:44.264 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-28T13:20:44.264 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-28T13:20:44.264 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-28T13:20:44.264 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-28T13:20:44.264 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-28T13:20:44.264 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-28T13:20:44.264 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-28T13:20:44.264 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-28T13:20:44.264 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-28T13:20:44.265 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-28T13:20:44.266 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-28T13:20:44.266 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-28T13:20:44.266 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-28T13:20:44.266 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-28T13:20:44.266 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-28T13:20:44.266 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-28T13:20:44.266 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-28T13:20:44.266 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-28T13:20:44.266 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-28T13:20:44.266 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-28T13:20:44.266 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-28T13:20:44.266 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-28T13:20:44.266 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-28T13:20:44.266 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-28T13:20:44.266 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-28T13:20:44.267 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-28T13:20:44.267 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-28T13:20:44.267 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-28T13:20:44.267 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-28T13:20:44.267 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-28T13:20:44.267 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-28T13:20:44.267 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-28T13:20:44.267 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-28T13:20:44.267 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-28T13:20:44.267 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-28T13:20:44.267 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-28T13:20:44.267 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-28T13:20:44.267 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-28T13:20:44.267 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-28T13:20:44.267 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-28T13:20:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-28T13:20:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-28T13:20:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-28T13:20:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-28T13:20:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-28T13:20:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-28T13:20:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-28T13:20:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-28T13:20:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-28T13:20:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-28T13:20:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-28T13:20:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-28T13:20:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-28T13:20:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-28T13:20:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-28T13:20:44.269 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-28T13:20:44.269 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-28T13:20:44.269 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-28T13:20:44.269 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-28T13:20:44.269 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-28T13:20:44.269 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-28T13:20:44.269 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-28T13:20:44.269 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-28T13:20:44.269 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-28T13:20:44.269 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-28T13:20:44.269 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-28T13:20:44.269 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-28T13:20:44.269 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-28T13:20:44.269 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-28T13:20:44.269 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-28T13:20:44.270 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-28T13:20:44.270 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-28T13:20:44.270 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-28T13:20:44.270 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-28T13:20:44.270 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-28T13:20:44.270 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-28T13:20:44.270 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-28T13:20:44.270 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-28T13:20:44.270 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-28T13:20:44.270 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-28T13:20:44.270 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-28T13:20:44.270 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-28T13:20:44.270 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-28T13:20:44.270 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-28T13:20:44.271 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-28T13:20:44.272 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-28T13:20:44.272 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-28T13:20:44.272 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-28T13:20:44.272 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-28T13:20:44.272 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-28T13:20:44.272 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-28T13:20:44.272 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-28T13:20:44.272 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-28T13:20:44.272 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-28T13:20:44.272 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-28T13:20:44.272 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-28T13:20:44.272 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-28T13:20:44.272 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-28T13:20:44.272 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-28T13:20:44.273 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-28T13:20:44.273 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-28T13:20:44.273 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-28T13:20:44.273 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-28T13:20:44.273 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-28T13:20:44.273 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-28T13:20:44.273 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-28T13:20:44.273 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-28T13:20:44.273 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-28T13:20:44.273 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-28T13:20:44.273 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-28T13:20:44.273 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-28T13:20:44.273 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-28T13:20:44.273 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-28T13:20:44.273 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-28T13:20:44.274 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-28T13:20:44.274 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-28T13:20:44.274 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-28T13:20:44.274 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-28T13:20:44.274 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-28T13:20:44.274 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-28T13:20:44.274 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-28T13:20:44.274 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-28T13:20:44.274 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-28T13:20:44.274 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-28T13:20:44.274 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-28T13:20:44.274 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-28T13:20:44.274 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-28T13:20:44.274 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-28T13:20:44.274 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-28T13:20:44.275 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-28T13:20:44.275 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-28T13:20:44.275 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-28T13:20:44.275 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-28T13:20:44.275 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-28T13:20:44.275 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-28T13:20:44.275 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-28T13:20:44.275 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-28T13:20:44.275 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-28T13:20:44.275 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-28T13:20:44.275 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-28T13:20:44.275 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-28T13:20:44.275 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-28T13:20:44.275 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-28T13:20:44.275 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-28T13:20:44.276 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-28T13:20:44.276 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-28T13:20:44.276 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-28T13:20:44.276 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-28T13:20:44.276 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-28T13:20:44.276 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-28T13:20:44.276 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-28T13:20:44.276 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-28T13:20:44.276 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-28T13:20:44.276 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-28T13:20:44.276 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-28T13:20:44.276 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-28T13:20:44.276 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-28T13:20:44.276 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-28T13:20:44.276 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-28T13:20:44.277 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-28T13:20:44.277 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-28T13:20:44.277 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-28T13:20:44.277 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-28T13:20:44.277 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-28T13:20:44.277 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-28T13:20:44.277 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-28T13:20:44.277 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-28T13:20:44.277 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-28T13:20:44.277 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-28T13:20:44.277 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-28T13:20:44.277 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-28T13:20:44.277 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-28T13:20:44.277 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-28T13:20:44.277 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-28T13:20:44.278 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-28T13:20:44.278 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-28T13:20:44.278 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-28T13:20:44.278 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-28T13:20:44.278 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-28T13:20:44.278 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-28T13:20:44.278 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-28T13:20:44.278 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-28T13:20:44.278 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-28T13:20:44.278 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-28T13:20:44.278 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-28T13:20:44.278 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-28T13:20:44.278 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-28T13:20:44.278 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-28T13:20:44.278 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-28T13:20:44.279 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-28T13:20:44.279 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-28T13:20:44.279 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-28T13:20:44.279 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-28T13:20:44.279 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-28T13:20:44.279 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-28T13:20:44.279 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-28T13:20:44.279 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-28T13:20:44.279 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-28T13:20:44.279 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-28T13:20:44.279 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-28T13:20:44.279 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-28T13:20:44.279 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-28T13:20:44.279 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-28T13:20:44.279 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-28T13:20:44.280 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-28T13:20:44.280 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-28T13:20:44.280 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-28T13:20:44.280 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-28T13:20:44.280 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-28T13:20:44.280 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-28T13:20:44.280 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-28T13:20:44.280 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-28T13:20:44.280 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-28T13:20:44.280 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-28T13:20:44.280 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-28T13:20:44.280 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-28T13:20:44.280 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-28T13:20:44.280 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-28T13:20:44.280 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-28T13:20:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-28T13:20:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-28T13:20:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-28T13:20:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-28T13:20:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-28T13:20:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-28T13:20:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-28T13:20:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-28T13:20:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-28T13:20:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-28T13:20:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-28T13:20:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-28T13:20:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-28T13:20:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-28T13:20:44.281 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-28T13:20:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-28T13:20:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-28T13:20:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-28T13:20:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-28T13:20:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-28T13:20:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-28T13:20:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-28T13:20:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-28T13:20:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-28T13:20:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-28T13:20:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-28T13:20:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-28T13:20:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-28T13:20:44.282 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-28T13:20:44.283 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-28T13:20:44.284 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-28T13:20:44.285 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-28T13:20:44.285 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-28T13:20:44.285 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-28T13:20:44.285 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-28T13:20:44.285 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-28T13:20:44.285 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-28T13:20:44.285 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-28T13:20:44.285 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-28T13:20:44.285 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-28T13:20:44.285 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-28T13:20:44.285 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-28T13:20:44.285 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-28T13:20:44.285 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-28T13:20:44.285 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-28T13:20:44.285 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-28T13:20:44.286 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-28T13:20:44.286 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-28T13:20:44.286 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-28T13:20:44.286 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-28T13:20:44.286 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-28T13:20:44.286 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-28T13:20:44.286 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-28T13:20:44.286 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-28T13:20:44.286 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-28T13:20:44.286 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-28T13:20:44.286 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-28T13:20:44.286 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-28T13:20:44.286 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-28T13:20:44.286 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-28T13:20:44.287 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-28T13:20:44.287 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-28T13:20:44.287 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-28T13:20:44.287 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-28T13:20:44.287 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-28T13:20:44.287 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-28T13:20:44.287 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-28T13:20:44.287 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-28T13:20:44.287 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-28T13:20:44.287 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-28T13:20:44.287 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-28T13:20:44.287 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-28T13:20:44.287 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-28T13:20:44.287 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-28T13:20:44.287 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-28T13:20:44.288 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-28T13:20:44.288 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-28T13:20:44.288 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-28T13:20:44.288 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-28T13:20:44.288 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-28T13:20:44.288 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-28T13:20:44.288 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-28T13:20:44.288 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-28T13:20:44.288 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-28T13:20:44.288 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-28T13:20:44.288 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-28T13:20:44.288 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-28T13:20:44.288 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-28T13:20:44.288 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-28T13:20:44.288 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-28T13:20:44.289 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-28T13:20:44.289 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-28T13:20:44.289 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-28T13:20:44.289 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-28T13:20:44.289 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-28T13:20:44.289 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-28T13:20:44.289 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-28T13:20:44.289 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-28T13:20:44.289 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-28T13:20:44.289 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-28T13:20:44.289 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-28T13:20:44.289 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-28T13:20:44.289 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-28T13:20:44.289 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-28T13:20:44.290 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-28T13:20:44.290 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-28T13:20:44.290 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-28T13:20:44.290 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-28T13:20:44.290 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-28T13:20:44.290 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-28T13:20:44.290 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-28T13:20:44.290 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-28T13:20:44.290 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-28T13:20:44.290 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-28T13:20:44.290 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-28T13:20:44.290 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-28T13:20:44.290 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-28T13:20:44.290 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-28T13:20:44.290 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-28T13:20:44.291 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-28T13:20:44.291 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-28T13:20:44.291 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-28T13:20:44.291 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-28T13:20:44.291 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-28T13:20:44.291 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-28T13:20:44.291 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-28T13:20:44.291 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-28T13:20:44.291 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-28T13:20:44.291 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-28T13:20:44.291 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-28T13:20:44.291 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-28T13:20:44.291 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-28T13:20:44.291 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-28T13:20:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-28T13:20:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-28T13:20:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-28T13:20:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-28T13:20:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-28T13:20:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-28T13:20:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-28T13:20:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-28T13:20:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-28T13:20:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-28T13:20:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-28T13:20:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-28T13:20:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-28T13:20:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-28T13:20:44.292 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-28T13:20:44.293 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-28T13:20:44.293 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-28T13:20:44.293 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-28T13:20:44.293 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-28T13:20:44.293 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-28T13:20:44.293 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-28T13:20:44.293 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-28T13:20:44.293 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-28T13:20:44.293 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-28T13:20:44.293 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-28T13:20:44.293 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-28T13:20:44.293 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-28T13:20:44.293 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-28T13:20:44.293 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-28T13:20:44.293 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-28T13:20:44.294 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-28T13:20:44.294 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-28T13:20:44.294 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-28T13:20:44.294 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-28T13:20:44.294 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-28T13:20:44.294 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-28T13:20:44.294 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-28T13:20:44.294 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-28T13:20:44.294 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-28T13:20:44.294 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-28T13:20:44.294 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-28T13:20:44.294 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-28T13:20:44.294 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-28T13:20:44.294 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-28T13:20:44.294 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-28T13:20:44.295 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-28T13:20:44.295 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-28T13:20:44.295 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-28T13:20:44.295 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-28T13:20:44.295 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-28T13:20:44.295 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-28T13:20:44.295 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-28T13:20:44.295 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-28T13:20:44.295 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-28T13:20:44.295 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-28T13:20:44.295 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-28T13:20:44.295 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-28T13:20:44.295 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-28T13:20:44.295 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-28T13:20:44.296 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-28T13:20:44.296 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-28T13:20:44.296 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-28T13:20:44.296 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-28T13:20:44.296 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-28T13:20:44.296 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-28T13:20:44.296 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-28T13:20:44.296 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-28T13:20:44.296 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-28T13:20:44.296 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-28T13:20:44.296 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-28T13:20:44.296 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-28T13:20:44.296 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-28T13:20:44.296 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-28T13:20:44.296 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-28T13:20:44.297 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-28T13:20:44.297 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-28T13:20:44.297 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-28T13:20:44.297 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-28T13:20:44.297 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-28T13:20:44.297 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-28T13:20:44.297 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-28T13:20:44.297 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-28T13:20:44.297 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-28T13:20:44.297 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-28T13:20:44.297 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-28T13:20:44.297 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-28T13:20:44.297 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-28T13:20:44.297 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-28T13:20:44.298 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-28T13:20:44.298 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-28T13:20:44.298 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-28T13:20:44.298 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-28T13:20:44.298 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-28T13:20:44.298 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-28T13:20:44.298 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-28T13:20:44.298 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-28T13:20:44.298 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-28T13:20:44.298 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-28T13:20:44.298 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-28T13:20:44.298 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-28T13:20:44.298 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-28T13:20:44.298 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-28T13:20:44.298 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-28T13:20:44.299 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-28T13:20:44.299 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-28T13:20:44.299 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-28T13:20:44.299 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-28T13:20:44.299 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-28T13:20:44.299 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-28T13:20:44.299 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-28T13:20:44.299 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-28T13:20:44.299 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-28T13:20:44.299 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-28T13:20:44.299 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-28T13:20:44.299 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-28T13:20:44.299 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-28T13:20:44.299 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-28T13:20:44.299 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-28T13:20:44.300 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-28T13:20:44.300 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-28T13:20:44.300 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-28T13:20:44.300 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-28T13:20:44.300 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-28T13:20:44.300 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-28T13:20:44.300 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-28T13:20:44.300 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-28T13:20:44.300 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-28T13:20:44.300 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-28T13:20:44.300 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-28T13:20:44.300 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-28T13:20:44.300 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-28T13:20:44.300 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-28T13:20:44.300 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-28T13:20:44.301 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-28T13:20:44.302 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-28T13:20:44.302 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-28T13:20:44.302 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-28T13:20:44.302 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-28T13:20:44.302 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-28T13:20:44.302 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-28T13:20:44.302 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-28T13:20:44.302 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-28T13:20:44.302 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-28T13:20:44.302 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-28T13:20:44.302 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-28T13:20:44.302 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-28T13:20:44.302 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-28T13:20:44.302 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-28T13:20:44.303 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-28T13:20:44.303 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-28T13:20:44.303 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-28T13:20:44.303 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-28T13:20:44.303 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-28T13:20:44.303 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-28T13:20:44.303 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-28T13:20:44.303 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-28T13:20:44.303 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-28T13:20:44.303 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-28T13:20:44.303 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-28T13:20:44.303 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-28T13:20:44.303 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-28T13:20:44.303 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-28T13:20:44.303 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-28T13:20:44.304 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-28T13:20:44.304 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-28T13:20:44.304 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-28T13:20:44.304 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-28T13:20:44.304 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-28T13:20:44.304 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-28T13:20:44.304 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-28T13:20:44.304 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-28T13:20:44.304 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-28T13:20:44.304 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-28T13:20:44.304 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-28T13:20:44.304 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-28T13:20:44.304 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-28T13:20:44.304 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-28T13:20:44.304 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-28T13:20:44.305 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-28T13:20:44.305 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-28T13:20:44.305 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-28T13:20:44.305 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-28T13:20:44.305 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-28T13:20:44.305 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-28T13:20:44.305 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-28T13:20:44.305 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-28T13:20:44.305 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-28T13:20:44.305 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-28T13:20:44.305 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-28T13:20:44.305 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-28T13:20:44.305 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-28T13:20:44.305 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-28T13:20:44.306 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-28T13:20:44.306 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-28T13:20:44.306 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-28T13:20:44.306 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-28T13:20:44.306 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-28T13:20:44.306 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-28T13:20:44.306 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-28T13:20:44.306 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-28T13:20:44.306 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-28T13:20:44.306 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-28T13:20:44.306 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-28T13:20:44.306 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-28T13:20:44.306 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-28T13:20:44.306 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-28T13:20:44.306 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-28T13:20:44.307 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-28T13:20:44.307 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-28T13:20:44.307 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-28T13:20:44.307 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-28T13:20:44.307 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-28T13:20:44.307 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-28T13:20:44.307 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-28T13:20:44.307 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-28T13:20:44.307 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-28T13:20:44.307 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-28T13:20:44.307 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-28T13:20:44.307 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-28T13:20:44.307 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-28T13:20:44.307 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-28T13:20:44.307 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-28T13:20:44.308 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-28T13:20:44.308 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-28T13:20:44.308 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-28T13:20:44.308 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-28T13:20:44.308 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-28T13:20:44.308 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-28T13:20:44.308 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-28T13:20:44.308 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-28T13:20:44.308 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-28T13:20:44.308 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-28T13:20:44.308 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-28T13:20:44.308 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-28T13:20:44.308 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-28T13:20:44.308 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-28T13:20:44.309 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-28T13:20:44.309 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-28T13:20:44.309 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-28T13:20:44.309 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-28T13:20:44.309 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-28T13:20:44.309 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-28T13:20:44.309 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-28T13:20:44.309 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-28T13:20:44.309 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-28T13:20:44.309 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-28T13:20:44.309 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-28T13:20:44.309 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-28T13:20:44.309 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-28T13:20:44.309 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-28T13:20:44.309 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-28T13:20:44.310 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-28T13:20:44.310 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-28T13:20:44.310 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-28T13:20:44.310 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-28T13:20:44.310 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-28T13:20:44.310 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-28T13:20:44.310 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-28T13:20:44.310 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-28T13:20:44.310 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-28T13:20:44.310 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-28T13:20:44.310 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-28T13:20:44.310 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-28T13:20:44.310 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-28T13:20:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-28T13:20:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-28T13:20:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-28T13:20:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-28T13:20:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-28T13:20:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-28T13:20:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-28T13:20:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-28T13:20:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-28T13:20:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-28T13:20:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-28T13:20:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-28T13:20:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-28T13:20:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-28T13:20:44.311 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-28T13:20:44.312 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-28T13:20:44.312 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-28T13:20:44.312 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-28T13:20:44.315 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:44.316 DEBUG:teuthology.orchestra.run.smithi196:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-28T13:20:44.341 INFO:teuthology.orchestra.run.smithi196.stdout:postinstall scriptlet (using /bin/sh): 2024-04-28T13:20:44.341 INFO:teuthology.orchestra.run.smithi196.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-28T13:20:44.342 INFO:teuthology.orchestra.run.smithi196.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-28T13:20:44.342 INFO:teuthology.orchestra.run.smithi196.stdout:fi 2024-04-28T13:20:44.342 INFO:teuthology.orchestra.run.smithi196.stdout:for file in vmlinuz System.map config; do 2024-04-28T13:20:44.342 INFO:teuthology.orchestra.run.smithi196.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-28T13:20:44.342 INFO:teuthology.orchestra.run.smithi196.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-28T13:20:44.342 INFO:teuthology.orchestra.run.smithi196.stdout: fi 2024-04-28T13:20:44.342 INFO:teuthology.orchestra.run.smithi196.stdout:done 2024-04-28T13:20:44.342 INFO:teuthology.orchestra.run.smithi196.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-28T13:20:44.342 INFO:teuthology.orchestra.run.smithi196.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-28T13:20:44.342 INFO:teuthology.orchestra.run.smithi196.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-28T13:20:44.342 INFO:teuthology.orchestra.run.smithi196.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-28T13:20:44.342 INFO:teuthology.orchestra.run.smithi196.stdout:fi 2024-04-28T13:20:44.342 INFO:teuthology.orchestra.run.smithi196.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-28T13:20:44.343 INFO:teuthology.orchestra.run.smithi196.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:44.343 INFO:teuthology.orchestra.run.smithi196.stdout:fi 2024-04-28T13:20:44.343 INFO:teuthology.orchestra.run.smithi196.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-28T13:20:44.343 INFO:teuthology.orchestra.run.smithi196.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-28T13:20:44.343 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:44.343 INFO:teuthology.orchestra.run.smithi196.stdout:fi 2024-04-28T13:20:44.343 DEBUG:teuthology.orchestra.run.smithi196:> sudo rpm -qi grub2-tools 2024-04-28T13:20:44.416 INFO:teuthology.orchestra.run.smithi196.stdout:Name : grub2-tools 2024-04-28T13:20:44.416 INFO:teuthology.orchestra.run.smithi196.stdout:Epoch : 1 2024-04-28T13:20:44.416 INFO:teuthology.orchestra.run.smithi196.stdout:Version : 2.06 2024-04-28T13:20:44.416 INFO:teuthology.orchestra.run.smithi196.stdout:Release : 68.el9 2024-04-28T13:20:44.416 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: x86_64 2024-04-28T13:20:44.416 INFO:teuthology.orchestra.run.smithi196.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-28T13:20:44.416 INFO:teuthology.orchestra.run.smithi196.stdout:Group : Unspecified 2024-04-28T13:20:44.417 INFO:teuthology.orchestra.run.smithi196.stdout:Size : 8264254 2024-04-28T13:20:44.417 INFO:teuthology.orchestra.run.smithi196.stdout:License : GPLv3+ 2024-04-28T13:20:44.417 INFO:teuthology.orchestra.run.smithi196.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-28T13:20:44.417 INFO:teuthology.orchestra.run.smithi196.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-28T13:20:44.417 INFO:teuthology.orchestra.run.smithi196.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-28T13:20:44.417 INFO:teuthology.orchestra.run.smithi196.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-28T13:20:44.417 INFO:teuthology.orchestra.run.smithi196.stdout:Packager : builder@centos.org 2024-04-28T13:20:44.417 INFO:teuthology.orchestra.run.smithi196.stdout:Vendor : CentOS 2024-04-28T13:20:44.417 INFO:teuthology.orchestra.run.smithi196.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-28T13:20:44.417 INFO:teuthology.orchestra.run.smithi196.stdout:Summary : Support tools for GRUB. 2024-04-28T13:20:44.417 INFO:teuthology.orchestra.run.smithi196.stdout:Description : 2024-04-28T13:20:44.417 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:20:44.417 INFO:teuthology.orchestra.run.smithi196.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-28T13:20:44.417 INFO:teuthology.orchestra.run.smithi196.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-28T13:20:44.418 INFO:teuthology.orchestra.run.smithi196.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-28T13:20:44.418 INFO:teuthology.orchestra.run.smithi196.stdout:hardware devices. 2024-04-28T13:20:44.418 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:20:44.418 INFO:teuthology.orchestra.run.smithi196.stdout:This subpackage provides tools for support of all platforms. 2024-04-28T13:20:44.419 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-28T13:20:44.419 INFO:teuthology.task.kernel:Updating grub on smithi196 to boot 6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:44.419 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-28T13:20:44.989 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-04-28T13:20:46.401 INFO:teuthology.orchestra.run.smithi196.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-28T13:20:46.431 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-04-28T13:20:46.433 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-04-28T13:20:46.463 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.cNkdB1Sn7K 2024-04-28T13:20:46.464 DEBUG:teuthology.orchestra.run.smithi196:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.cNkdB1Sn7K 2024-04-28T13:20:46.531 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.cNkdB1Sn7K 2024-04-28T13:20:46.668 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.cNkdB1Sn7K is 6KB 2024-04-28T13:20:46.717 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.cNkdB1Sn7K 2024-04-28T13:20:46.732 DEBUG:teuthology.orchestra.run.smithi196:> sudo /bin/ls /boot/loader/entries || true 2024-04-28T13:20:46.793 INFO:teuthology.orchestra.run.smithi196.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-28T13:20:46.793 INFO:teuthology.orchestra.run.smithi196.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-28T13:20:46.793 INFO:teuthology.orchestra.run.smithi196.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-28T13:20:46.793 INFO:teuthology.orchestra.run.smithi196.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-28T13:20:46.794 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-28T13:20:46.968 DEBUG:teuthology.orchestra.run.smithi196:> sudo shutdown -r now 2024-04-28T13:20:51.998 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-04-28T13:21:01.006 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-28T13:21:01.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T13:21:04.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-04-28T13:21:16.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-28T13:21:16.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T13:21:16.971 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-28T13:21:16.972 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-28T13:21:16.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T13:21:35.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-28T13:21:44.524 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-28T13:21:44.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T13:21:47.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-28T13:21:59.594 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-28T13:21:59.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T13:22:16.158 DEBUG:teuthology.orchestra.remote:timed out 2024-04-28T13:22:31.197 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-28T13:22:31.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T13:22:31.390 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-04-28T13:22:31.688 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-04-28T13:22:32.107 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-28T13:22:32.107 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-28T13:22:32.107 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-04-28T13:22:32.158 INFO:teuthology.orchestra.run.smithi196.stdout:6.9.0-rc3-g806ec36b2112 2024-04-28T13:22:32.158 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-28T13:22:32.159 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-28T13:22:32.159 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-28T13:22:32.319 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-28T13:22:32.320 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-04-28T13:22:32.320 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2024-04-28T13:22:32.372 INFO:teuthology.orchestra.run.smithi203.stdout:6.9.0-rc3-g806ec36b2112 2024-04-28T13:22:32.373 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-28T13:22:32.373 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-28T13:22:32.373 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-28T13:22:33.160 DEBUG:teuthology.parallel:result is None 2024-04-28T13:22:33.373 DEBUG:teuthology.parallel:result is None 2024-04-28T13:22:33.374 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-28T13:22:33.381 INFO:teuthology.task.internal:Creating test directory... 2024-04-28T13:22:33.381 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-28T13:22:33.388 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-28T13:22:33.445 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-28T13:22:33.452 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-28T13:22:33.458 INFO:teuthology.task.internal:Creating archive directory... 2024-04-28T13:22:33.458 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-28T13:22:33.463 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-28T13:22:33.531 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-28T13:22:33.537 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-28T13:22:33.537 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-28T13:22:33.541 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-28T13:22:33.656 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-28T13:22:33.663 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-28T13:22:33.708 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-28T13:22:33.717 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-28T13:22:33.721 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-28T13:22:33.728 INFO:teuthology.task.internal:Configuring sudo... 2024-04-28T13:22:33.728 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-28T13:22:33.732 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-28T13:22:33.817 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-28T13:22:33.826 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-28T13:22:33.827 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-28T13:22:33.834 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-28T13:22:33.889 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-28T13:22:33.984 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-28T13:22:34.061 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-28T13:22:34.130 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-28T13:22:34.216 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:22:34.216 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-28T13:22:34.284 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-28T13:22:34.371 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-28T13:22:34.468 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-28T13:22:34.553 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-28T13:22:34.641 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:22:34.641 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-28T13:22:34.727 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-04-28T13:22:34.731 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2024-04-28T13:22:34.838 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-28T13:22:34.879 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-28T13:22:35.347 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-28T13:22:35.355 INFO:teuthology.task.internal:Starting timer... 2024-04-28T13:22:35.355 INFO:teuthology.run_tasks:Running task pcp... 2024-04-28T13:22:35.364 INFO:teuthology.run_tasks:Running task selinux... 2024-04-28T13:22:35.372 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-28T13:22:35.372 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-04-28T13:22:35.476 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-04-28T13:22:35.482 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2024-04-28T13:22:35.599 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2024-04-28T13:22:35.603 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-28T13:22:35.603 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-04-28T13:22:35.651 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-04-28T13:22:35.651 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2024-04-28T13:22:35.703 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2024-04-28T13:22:35.703 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi196.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2024-04-28T13:22:35.703 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-28T13:22:35.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:22:35.768 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-28T13:22:35.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:22:35.844 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-28T13:22:35.844 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-04-28T13:22:35.930 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2024-04-28T13:22:36.006 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-28T13:22:36.015 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-28T13:22:36.031 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-28T13:22:36.032 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi196.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-28T13:27:43.941 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi196.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2024-04-28T13:27:43.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-28T13:27:43.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T13:27:44.050 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-04-28T13:27:44.215 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-28T13:27:44.215 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-28T13:27:44.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T13:27:44.337 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-04-28T13:27:44.488 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-28T13:27:44.488 INFO:teuthology.run_tasks:Running task clock... 2024-04-28T13:27:44.498 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-28T13:27:44.499 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-28T13:27:44.499 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-28T13:27:44.506 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-28T13:27:44.507 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-28T13:27:44.593 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-28T13:27:44.597 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-28T13:27:44.636 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-28T13:27:44.639 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-28T13:27:44.709 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-04-28T13:27:44.710 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2024-04-28T13:27:44.739 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2024-04-28T13:27:44.743 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-04-28T13:27:44.776 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-28T13:27:44.787 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-28T13:27:44.813 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-28T13:27:44.829 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-28T13:27:44.949 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-04-28T13:27:44.957 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-28T13:27:44.958 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-04-28T13:27:44.958 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T13:27:44.958 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T13:27:44.958 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T13:27:44.958 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T13:27:44.977 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-04-28T13:27:44.984 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-28T13:27:44.984 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-04-28T13:27:44.985 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T13:27:44.985 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T13:27:44.985 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T13:27:44.985 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T13:27:44.986 INFO:teuthology.run_tasks:Running task install... 2024-04-28T13:27:44.996 DEBUG:teuthology.task.install:project ceph 2024-04-28T13:27:44.996 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-28T13:27:44.996 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-28T13:27:44.996 INFO:teuthology.task.install:Using flavor: default 2024-04-28T13:27:45.002 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-28T13:27:45.002 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-28T13:27:45.003 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-28T13:27:45.003 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T13:27:45.005 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-28T13:27:45.005 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T13:27:45.152 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-28T13:27:45.153 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-28T13:27:45.228 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-28T13:27:45.228 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-28T13:27:45.325 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-28T13:27:45.325 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:27:45.326 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-28T13:27:45.336 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-28T13:27:45.337 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:27:45.337 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-28T13:27:45.416 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-28T13:27:45.416 DEBUG:teuthology.orchestra.run.smithi203:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-28T13:27:45.423 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-28T13:27:45.423 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-28T13:27:45.545 DEBUG:teuthology.orchestra.run.smithi203:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-28T13:27:45.555 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-28T13:27:45.654 DEBUG:teuthology.orchestra.run.smithi203:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-28T13:27:45.664 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-28T13:27:45.732 INFO:teuthology.orchestra.run.smithi203.stdout:check_obsoletes = 1 2024-04-28T13:27:45.736 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum clean all 2024-04-28T13:27:45.745 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-04-28T13:27:45.749 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-04-28T13:27:46.060 INFO:teuthology.orchestra.run.smithi203.stdout:48 files removed 2024-04-28T13:27:46.065 INFO:teuthology.orchestra.run.smithi196.stdout:48 files removed 2024-04-28T13:27:46.100 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-28T13:27:46.104 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-28T13:27:46.894 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 236 kB/s | 84 kB 00:00 2024-04-28T13:27:46.956 INFO:teuthology.orchestra.run.smithi203.stdout:ceph packages for x86_64 202 kB/s | 84 kB 00:00 2024-04-28T13:27:47.478 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-04-28T13:27:47.524 INFO:teuthology.orchestra.run.smithi203.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-04-28T13:27:48.012 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-04-28T13:27:48.024 INFO:teuthology.orchestra.run.smithi203.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-04-28T13:27:48.648 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-28T13:27:50.924 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-28T13:27:55.311 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-28T13:27:57.219 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-28T13:27:57.300 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 895 kB/s | 8.0 MB 00:09 2024-04-28T13:27:59.038 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-28T13:28:00.493 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 19 MB 00:01 2024-04-28T13:28:05.096 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-28T13:28:05.340 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 6.2 MB 00:00 2024-04-28T13:28:07.110 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-04-28T13:28:07.556 INFO:teuthology.orchestra.run.smithi203.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-28T13:28:07.557 INFO:teuthology.orchestra.run.smithi203.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-28T13:28:07.578 INFO:teuthology.orchestra.run.smithi203.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-28T13:28:07.579 INFO:teuthology.orchestra.run.smithi203.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-28T13:28:07.580 INFO:teuthology.orchestra.run.smithi203.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-28T13:28:07.581 INFO:teuthology.orchestra.run.smithi203.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-28T13:28:07.584 INFO:teuthology.orchestra.run.smithi203.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-28T13:28:07.587 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-28T13:28:07.588 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-28T13:28:07.658 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-04-28T13:28:07.679 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-04-28T13:28:07.679 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-04-28T13:28:07.679 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-04-28T13:28:07.679 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-04-28T13:28:07.679 INFO:teuthology.orchestra.run.smithi203.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-28T13:28:07.679 INFO:teuthology.orchestra.run.smithi203.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-28T13:28:07.679 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-28T13:28:07.679 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-28T13:28:07.679 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-28T13:28:07.679 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-28T13:28:07.679 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-28T13:28:07.679 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-28T13:28:07.679 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-28T13:28:07.680 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-28T13:28:07.680 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-28T13:28:07.680 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-28T13:28:07.680 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-28T13:28:07.680 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-28T13:28:07.680 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-28T13:28:07.680 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-28T13:28:07.680 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-28T13:28:07.680 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-28T13:28:07.680 INFO:teuthology.orchestra.run.smithi203.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-28T13:28:07.680 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-28T13:28:07.680 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-28T13:28:07.680 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-28T13:28:07.680 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-28T13:28:07.681 INFO:teuthology.orchestra.run.smithi203.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-28T13:28:07.681 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-28T13:28:07.681 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-28T13:28:07.681 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-28T13:28:07.681 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-28T13:28:07.681 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-28T13:28:07.681 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-28T13:28:07.681 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-28T13:28:07.681 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-28T13:28:07.681 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-28T13:28:07.681 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-28T13:28:07.681 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-28T13:28:07.681 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-28T13:28:07.682 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-28T13:28:07.682 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2024-04-28T13:28:07.682 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-28T13:28:07.682 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-28T13:28:07.682 INFO:teuthology.orchestra.run.smithi203.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-28T13:28:07.682 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-28T13:28:07.682 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-04-28T13:28:07.682 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-28T13:28:07.682 INFO:teuthology.orchestra.run.smithi203.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-28T13:28:07.682 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-28T13:28:07.682 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-28T13:28:07.682 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-28T13:28:07.682 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-28T13:28:07.683 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-28T13:28:07.683 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-28T13:28:07.683 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-28T13:28:07.683 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-28T13:28:07.683 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-28T13:28:07.683 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-28T13:28:07.683 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-28T13:28:07.683 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-28T13:28:07.683 INFO:teuthology.orchestra.run.smithi203.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-28T13:28:07.683 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-28T13:28:07.683 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-28T13:28:07.683 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-28T13:28:07.683 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-28T13:28:07.683 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-28T13:28:07.684 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-28T13:28:07.684 INFO:teuthology.orchestra.run.smithi203.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-28T13:28:07.684 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-28T13:28:07.684 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-28T13:28:07.684 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-28T13:28:07.684 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-28T13:28:07.684 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-28T13:28:07.684 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-28T13:28:07.684 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-28T13:28:07.684 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-28T13:28:07.684 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-28T13:28:07.684 INFO:teuthology.orchestra.run.smithi203.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-28T13:28:07.684 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-28T13:28:07.684 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-28T13:28:07.685 INFO:teuthology.orchestra.run.smithi203.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-28T13:28:07.685 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-28T13:28:07.685 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-28T13:28:07.685 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-28T13:28:07.685 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-28T13:28:07.685 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-28T13:28:07.685 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-28T13:28:07.685 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-28T13:28:07.685 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-28T13:28:07.685 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-28T13:28:07.685 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-28T13:28:07.685 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-28T13:28:07.686 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-28T13:28:07.686 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-28T13:28:07.686 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-28T13:28:07.686 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-28T13:28:07.686 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-28T13:28:07.686 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-28T13:28:07.686 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-28T13:28:07.686 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-28T13:28:07.686 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-28T13:28:07.686 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-28T13:28:07.686 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-28T13:28:07.686 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-28T13:28:07.687 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-28T13:28:07.688 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-28T13:28:07.688 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-28T13:28:07.688 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-28T13:28:07.688 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-28T13:28:07.688 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-28T13:28:07.688 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-28T13:28:07.688 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-28T13:28:07.688 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-28T13:28:07.688 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-28T13:28:07.688 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-28T13:28:07.688 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-28T13:28:07.688 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-28T13:28:07.688 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-28T13:28:07.688 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-28T13:28:07.688 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-28T13:28:07.689 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-28T13:28:07.689 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-28T13:28:07.689 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-28T13:28:07.689 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-28T13:28:07.689 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-28T13:28:07.689 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-28T13:28:07.689 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-28T13:28:07.689 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-28T13:28:07.689 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-28T13:28:07.689 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-28T13:28:07.689 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-28T13:28:07.689 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-28T13:28:07.689 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-28T13:28:07.689 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-28T13:28:07.690 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-28T13:28:07.690 INFO:teuthology.orchestra.run.smithi203.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-28T13:28:07.690 INFO:teuthology.orchestra.run.smithi203.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-28T13:28:07.690 INFO:teuthology.orchestra.run.smithi203.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-28T13:28:07.690 INFO:teuthology.orchestra.run.smithi203.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-28T13:28:07.690 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-28T13:28:07.690 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-28T13:28:07.690 INFO:teuthology.orchestra.run.smithi203.stdout:Installing weak dependencies: 2024-04-28T13:28:07.690 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-28T13:28:07.690 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:28:07.690 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-04-28T13:28:07.690 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-04-28T13:28:07.690 INFO:teuthology.orchestra.run.smithi203.stdout:Install 148 Packages 2024-04-28T13:28:07.690 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 4 Packages 2024-04-28T13:28:07.690 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:28:07.691 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 222 M 2024-04-28T13:28:07.691 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-04-28T13:28:08.512 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-28T13:28:08.615 INFO:teuthology.orchestra.run.smithi203.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-04-28T13:28:09.431 INFO:teuthology.orchestra.run.smithi203.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.1 MB/s | 889 kB 00:00 2024-04-28T13:28:09.707 INFO:teuthology.orchestra.run.smithi203.stdout:(3/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 15 MB/s | 21 MB 00:01 2024-04-28T13:28:09.740 INFO:teuthology.orchestra.run.smithi203.stdout:(4/152): ceph-immutable-object-cache-19.0.0-330 471 kB/s | 145 kB 00:00 2024-04-28T13:28:09.882 INFO:teuthology.orchestra.run.smithi203.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 14 MB/s | 2.4 MB 00:00 2024-04-28T13:28:10.183 INFO:teuthology.orchestra.run.smithi203.stdout:(6/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 16 MB/s | 4.7 MB 00:00 2024-04-28T13:28:10.608 INFO:teuthology.orchestra.run.smithi203.stdout:(7/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-04-28T13:28:10.717 INFO:teuthology.orchestra.run.smithi203.stdout:(8/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-04-28T13:28:10.792 INFO:teuthology.orchestra.run.smithi203.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 340 kB/s | 25 kB 00:00 2024-04-28T13:28:11.142 INFO:teuthology.orchestra.run.smithi203.stdout:(10/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 18 MB/s | 17 MB 00:00 2024-04-28T13:28:11.201 INFO:teuthology.orchestra.run.smithi203.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 3.8 MB/s | 225 kB 00:00 2024-04-28T13:28:11.251 INFO:teuthology.orchestra.run.smithi203.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 551 kB/s | 27 kB 00:00 2024-04-28T13:28:11.301 INFO:teuthology.orchestra.run.smithi203.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 641 kB/s | 32 kB 00:00 2024-04-28T13:28:11.385 INFO:teuthology.orchestra.run.smithi203.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 8.8 MB/s | 747 kB 00:00 2024-04-28T13:28:11.435 INFO:teuthology.orchestra.run.smithi203.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 3.2 MB/s | 163 kB 00:00 2024-04-28T13:28:11.494 INFO:teuthology.orchestra.run.smithi203.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 2.2 MB/s | 127 kB 00:00 2024-04-28T13:28:11.561 INFO:teuthology.orchestra.run.smithi203.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 7.4 MB/s | 501 kB 00:00 2024-04-28T13:28:11.978 INFO:teuthology.orchestra.run.smithi203.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 13 MB/s | 5.4 MB 00:00 2024-04-28T13:28:12.045 INFO:teuthology.orchestra.run.smithi203.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 686 kB/s | 45 kB 00:00 2024-04-28T13:28:12.104 INFO:teuthology.orchestra.run.smithi203.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 2.3 MB/s | 135 kB 00:00 2024-04-28T13:28:12.154 INFO:teuthology.orchestra.run.smithi203.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 3.2 MB/s | 161 kB 00:00 2024-04-28T13:28:12.221 INFO:teuthology.orchestra.run.smithi203.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 4.8 MB/s | 321 kB 00:00 2024-04-28T13:28:12.288 INFO:teuthology.orchestra.run.smithi203.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 4.4 MB/s | 298 kB 00:00 2024-04-28T13:28:12.338 INFO:teuthology.orchestra.run.smithi203.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 2.0 MB/s | 99 kB 00:00 2024-04-28T13:28:12.389 INFO:teuthology.orchestra.run.smithi203.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-28T13:28:12.714 INFO:teuthology.orchestra.run.smithi203.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 9.5 MB/s | 3.1 MB 00:00 2024-04-28T13:28:12.798 INFO:teuthology.orchestra.run.smithi203.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-28T13:28:12.873 INFO:teuthology.orchestra.run.smithi203.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 461 kB/s | 34 kB 00:00 2024-04-28T13:28:12.948 INFO:teuthology.orchestra.run.smithi203.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.9 MB/s | 142 kB 00:00 2024-04-28T13:28:13.399 INFO:teuthology.orchestra.run.smithi203.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 7.9 MB/s | 3.5 MB 00:00 2024-04-28T13:28:13.557 INFO:teuthology.orchestra.run.smithi203.stdout:(31/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 3.6 MB/s | 11 MB 00:02 2024-04-28T13:28:13.783 INFO:teuthology.orchestra.run.smithi203.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 1.1 MB/s | 249 kB 00:00 2024-04-28T13:28:13.883 INFO:teuthology.orchestra.run.smithi203.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 498 kB/s | 49 kB 00:00 2024-04-28T13:28:13.992 INFO:teuthology.orchestra.run.smithi203.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 153 kB/s | 16 kB 00:00 2024-04-28T13:28:14.209 INFO:teuthology.orchestra.run.smithi203.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 9.2 MB/s | 7.4 MB 00:00 2024-04-28T13:28:14.242 INFO:teuthology.orchestra.run.smithi203.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.1 MB/s | 273 kB 00:00 2024-04-28T13:28:14.335 INFO:teuthology.orchestra.run.smithi203.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 6.0 MB/s | 762 kB 00:00 2024-04-28T13:28:14.360 INFO:teuthology.orchestra.run.smithi203.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 157 kB/s | 18 kB 00:00 2024-04-28T13:28:14.494 INFO:teuthology.orchestra.run.smithi203.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 511 kB/s | 80 kB 00:00 2024-04-28T13:28:14.545 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-28T13:28:14.569 INFO:teuthology.orchestra.run.smithi203.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-04-28T13:28:14.619 INFO:teuthology.orchestra.run.smithi203.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-04-28T13:28:14.678 INFO:teuthology.orchestra.run.smithi203.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-04-28T13:28:14.728 INFO:teuthology.orchestra.run.smithi203.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-28T13:28:14.787 INFO:teuthology.orchestra.run.smithi203.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 7.4 MB/s | 1.2 MB 00:00 2024-04-28T13:28:14.820 INFO:teuthology.orchestra.run.smithi203.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-04-28T13:28:14.888 INFO:teuthology.orchestra.run.smithi203.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-28T13:28:14.921 INFO:teuthology.orchestra.run.smithi203.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-28T13:28:15.472 INFO:teuthology.orchestra.run.smithi203.stdout:(48/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 10 MB/s | 49 MB 00:04 2024-04-28T13:28:15.513 INFO:teuthology.orchestra.run.smithi203.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 689 kB 00:00 2024-04-28T13:28:15.580 INFO:teuthology.orchestra.run.smithi203.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 1.4 MB/s | 943 kB 00:00 2024-04-28T13:28:15.606 INFO:teuthology.orchestra.run.smithi203.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 234 kB/s | 21 kB 00:00 2024-04-28T13:28:15.639 INFO:teuthology.orchestra.run.smithi203.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-04-28T13:28:15.665 INFO:teuthology.orchestra.run.smithi203.stdout:(53/152): boost-program-options-1.75.0-8.el9.x8 556 kB/s | 107 kB 00:00 2024-04-28T13:28:15.690 INFO:teuthology.orchestra.run.smithi203.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 394 kB/s | 33 kB 00:00 2024-04-28T13:28:15.740 INFO:teuthology.orchestra.run.smithi203.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 229 kB/s | 11 kB 00:00 2024-04-28T13:28:15.766 INFO:teuthology.orchestra.run.smithi203.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 245 kB/s | 18 kB 00:00 2024-04-28T13:28:15.808 INFO:teuthology.orchestra.run.smithi203.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-28T13:28:15.833 INFO:teuthology.orchestra.run.smithi203.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 681 kB/s | 45 kB 00:00 2024-04-28T13:28:15.900 INFO:teuthology.orchestra.run.smithi203.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-28T13:28:15.984 INFO:teuthology.orchestra.run.smithi203.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.7 MB/s | 3.0 MB 00:00 2024-04-28T13:28:16.009 INFO:teuthology.orchestra.run.smithi203.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 236 kB/s | 25 kB 00:00 2024-04-28T13:28:16.034 INFO:teuthology.orchestra.run.smithi203.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 960 kB/s | 47 kB 00:00 2024-04-28T13:28:16.068 INFO:teuthology.orchestra.run.smithi203.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-28T13:28:16.102 INFO:teuthology.orchestra.run.smithi203.stdout:(64/152): libtool-2.4.6-45.el9.x86_64.rpm 2.1 MB/s | 582 kB 00:00 2024-04-28T13:28:16.135 INFO:teuthology.orchestra.run.smithi203.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 532 kB/s | 35 kB 00:00 2024-04-28T13:28:16.160 INFO:teuthology.orchestra.run.smithi203.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-28T13:28:16.312 INFO:teuthology.orchestra.run.smithi203.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-04-28T13:28:16.370 INFO:teuthology.orchestra.run.smithi203.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-28T13:28:16.456 INFO:teuthology.orchestra.run.smithi203.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-04-28T13:28:16.487 INFO:teuthology.orchestra.run.smithi203.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-28T13:28:16.579 INFO:teuthology.orchestra.run.smithi203.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-28T13:28:16.680 INFO:teuthology.orchestra.run.smithi203.stdout:(72/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-04-28T13:28:16.705 INFO:teuthology.orchestra.run.smithi203.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 703 kB/s | 87 kB 00:00 2024-04-28T13:28:16.763 INFO:teuthology.orchestra.run.smithi203.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-28T13:28:16.898 INFO:teuthology.orchestra.run.smithi203.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-28T13:28:16.900 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-28T13:28:16.902 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-28T13:28:16.921 INFO:teuthology.orchestra.run.smithi196.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-28T13:28:16.922 INFO:teuthology.orchestra.run.smithi196.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-28T13:28:16.923 INFO:teuthology.orchestra.run.smithi196.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-28T13:28:16.925 INFO:teuthology.orchestra.run.smithi196.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-28T13:28:16.927 INFO:teuthology.orchestra.run.smithi196.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-28T13:28:16.930 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-28T13:28:16.931 INFO:teuthology.orchestra.run.smithi203.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-04-28T13:28:16.931 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-28T13:28:16.947 INFO:teuthology.orchestra.run.smithi203.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 545 kB/s | 26 kB 00:00 2024-04-28T13:28:16.973 INFO:teuthology.orchestra.run.smithi203.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 466 kB/s | 19 kB 00:00 2024-04-28T13:28:17.000 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-04-28T13:28:17.020 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-04-28T13:28:17.020 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-04-28T13:28:17.020 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-04-28T13:28:17.020 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-04-28T13:28:17.021 INFO:teuthology.orchestra.run.smithi196.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-28T13:28:17.021 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-28T13:28:17.021 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-28T13:28:17.021 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-28T13:28:17.021 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-28T13:28:17.021 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-28T13:28:17.021 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-28T13:28:17.021 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-28T13:28:17.021 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-28T13:28:17.021 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-28T13:28:17.022 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-28T13:28:17.022 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-28T13:28:17.022 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-28T13:28:17.022 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-28T13:28:17.022 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-28T13:28:17.022 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-28T13:28:17.022 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-28T13:28:17.022 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-28T13:28:17.022 INFO:teuthology.orchestra.run.smithi196.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-28T13:28:17.022 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-28T13:28:17.022 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-28T13:28:17.022 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-28T13:28:17.022 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-28T13:28:17.022 INFO:teuthology.orchestra.run.smithi196.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-28T13:28:17.023 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-28T13:28:17.023 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-28T13:28:17.023 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-28T13:28:17.023 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-28T13:28:17.023 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-28T13:28:17.023 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-28T13:28:17.023 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-28T13:28:17.023 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-28T13:28:17.023 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-28T13:28:17.023 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-28T13:28:17.023 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-28T13:28:17.023 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-28T13:28:17.023 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-28T13:28:17.023 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-04-28T13:28:17.023 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-28T13:28:17.024 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-28T13:28:17.024 INFO:teuthology.orchestra.run.smithi196.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-28T13:28:17.024 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-28T13:28:17.024 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-04-28T13:28:17.024 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-28T13:28:17.024 INFO:teuthology.orchestra.run.smithi196.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-28T13:28:17.024 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-28T13:28:17.024 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-28T13:28:17.024 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-28T13:28:17.024 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-28T13:28:17.024 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-28T13:28:17.024 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-28T13:28:17.024 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-28T13:28:17.024 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-28T13:28:17.024 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-28T13:28:17.025 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-28T13:28:17.025 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-28T13:28:17.025 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-28T13:28:17.025 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-28T13:28:17.025 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-28T13:28:17.025 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-28T13:28:17.025 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-28T13:28:17.025 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-28T13:28:17.025 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-28T13:28:17.025 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-28T13:28:17.025 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-28T13:28:17.025 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-28T13:28:17.026 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-28T13:28:17.026 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-28T13:28:17.026 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-28T13:28:17.026 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-28T13:28:17.026 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-28T13:28:17.026 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-28T13:28:17.026 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-28T13:28:17.026 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-28T13:28:17.026 INFO:teuthology.orchestra.run.smithi196.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-28T13:28:17.026 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-28T13:28:17.026 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-28T13:28:17.027 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-28T13:28:17.028 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-28T13:28:17.028 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-28T13:28:17.028 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-28T13:28:17.028 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-28T13:28:17.028 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-28T13:28:17.028 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-28T13:28:17.028 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-28T13:28:17.028 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-28T13:28:17.028 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-28T13:28:17.028 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-28T13:28:17.028 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-28T13:28:17.028 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-28T13:28:17.028 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-28T13:28:17.028 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-28T13:28:17.029 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-28T13:28:17.029 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-28T13:28:17.029 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-28T13:28:17.029 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-28T13:28:17.029 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-28T13:28:17.029 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-28T13:28:17.029 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-28T13:28:17.029 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-28T13:28:17.029 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-28T13:28:17.029 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-28T13:28:17.029 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-28T13:28:17.029 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-28T13:28:17.029 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-28T13:28:17.029 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-28T13:28:17.029 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-28T13:28:17.030 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-28T13:28:17.030 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-28T13:28:17.030 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-28T13:28:17.030 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-28T13:28:17.030 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-28T13:28:17.030 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-28T13:28:17.030 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-28T13:28:17.030 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-28T13:28:17.030 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-28T13:28:17.030 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-28T13:28:17.030 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-28T13:28:17.030 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-28T13:28:17.031 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-28T13:28:17.031 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-28T13:28:17.031 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-28T13:28:17.031 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-28T13:28:17.031 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-28T13:28:17.031 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-28T13:28:17.031 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-28T13:28:17.031 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-28T13:28:17.031 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-28T13:28:17.032 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-28T13:28:17.032 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-28T13:28:17.032 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-28T13:28:17.032 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-28T13:28:17.032 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-28T13:28:17.032 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-28T13:28:17.032 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-28T13:28:17.032 INFO:teuthology.orchestra.run.smithi196.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-28T13:28:17.032 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-28T13:28:17.032 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-28T13:28:17.032 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-04-28T13:28:17.032 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-28T13:28:17.032 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:28:17.032 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-04-28T13:28:17.033 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-04-28T13:28:17.033 INFO:teuthology.orchestra.run.smithi196.stdout:Install 148 Packages 2024-04-28T13:28:17.033 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 4 Packages 2024-04-28T13:28:17.033 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:28:17.033 INFO:teuthology.orchestra.run.smithi203.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 172 kB 00:00 2024-04-28T13:28:17.033 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 222 M 2024-04-28T13:28:17.034 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-04-28T13:28:17.057 INFO:teuthology.orchestra.run.smithi203.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 419 kB/s | 35 kB 00:00 2024-04-28T13:28:17.115 INFO:teuthology.orchestra.run.smithi203.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-28T13:28:17.175 INFO:teuthology.orchestra.run.smithi203.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.8 MB/s | 222 kB 00:00 2024-04-28T13:28:17.225 INFO:teuthology.orchestra.run.smithi203.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-28T13:28:17.375 INFO:teuthology.orchestra.run.smithi203.stdout:(84/152): python3-babel-2.9.1-2.el9.noarch.rpm 8.6 MB/s | 6.0 MB 00:00 2024-04-28T13:28:17.401 INFO:teuthology.orchestra.run.smithi203.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 241 kB/s | 42 kB 00:00 2024-04-28T13:28:17.434 INFO:teuthology.orchestra.run.smithi203.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-04-28T13:28:17.468 INFO:teuthology.orchestra.run.smithi203.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-28T13:28:17.502 INFO:teuthology.orchestra.run.smithi203.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-28T13:28:17.527 INFO:teuthology.orchestra.run.smithi203.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 879 kB/s | 51 kB 00:00 2024-04-28T13:28:17.569 INFO:teuthology.orchestra.run.smithi203.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 809 kB/s | 54 kB 00:00 2024-04-28T13:28:17.636 INFO:teuthology.orchestra.run.smithi203.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 631 kB/s | 42 kB 00:00 2024-04-28T13:28:17.703 INFO:teuthology.orchestra.run.smithi203.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 670 kB/s | 44 kB 00:00 2024-04-28T13:28:17.796 INFO:teuthology.orchestra.run.smithi196.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-28T13:28:17.812 INFO:teuthology.orchestra.run.smithi203.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.5 MB/s | 5.1 MB 00:00 2024-04-28T13:28:17.845 INFO:teuthology.orchestra.run.smithi203.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-04-28T13:28:17.896 INFO:teuthology.orchestra.run.smithi203.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 963 kB/s | 47 kB 00:00 2024-04-28T13:28:17.921 INFO:teuthology.orchestra.run.smithi203.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 332 kB/s | 36 kB 00:00 2024-04-28T13:28:17.963 INFO:teuthology.orchestra.run.smithi203.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-28T13:28:17.988 INFO:teuthology.orchestra.run.smithi203.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 879 kB/s | 58 kB 00:00 2024-04-28T13:28:18.038 INFO:teuthology.orchestra.run.smithi203.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-28T13:28:18.084 INFO:teuthology.orchestra.run.smithi203.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 805 kB/s | 94 kB 00:00 2024-04-28T13:28:18.106 INFO:teuthology.orchestra.run.smithi203.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 676 kB/s | 45 kB 00:00 2024-04-28T13:28:18.248 INFO:teuthology.orchestra.run.smithi203.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-04-28T13:28:18.473 INFO:teuthology.orchestra.run.smithi203.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-04-28T13:28:18.699 INFO:teuthology.orchestra.run.smithi203.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-04-28T13:28:18.882 INFO:teuthology.orchestra.run.smithi203.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 758 kB/s | 309 kB 00:00 2024-04-28T13:28:18.930 INFO:teuthology.orchestra.run.smithi196.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 786 kB/s | 889 kB 00:01 2024-04-28T13:28:18.991 INFO:teuthology.orchestra.run.smithi203.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 67 kB/s | 19 kB 00:00 2024-04-28T13:28:19.058 INFO:teuthology.orchestra.run.smithi203.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 677 kB/s | 548 kB 00:00 2024-04-28T13:28:19.083 INFO:teuthology.orchestra.run.smithi203.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 287 kB/s | 26 kB 00:00 2024-04-28T13:28:19.088 INFO:teuthology.orchestra.run.smithi196.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 924 kB/s | 145 kB 00:00 2024-04-28T13:28:19.142 INFO:teuthology.orchestra.run.smithi203.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 585 kB/s | 49 kB 00:00 2024-04-28T13:28:19.201 INFO:teuthology.orchestra.run.smithi203.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 579 kB/s | 67 kB 00:00 2024-04-28T13:28:19.259 INFO:teuthology.orchestra.run.smithi203.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-28T13:28:19.435 INFO:teuthology.orchestra.run.smithi203.stdout:(112/152): libarrow-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-04-28T13:28:19.477 INFO:teuthology.orchestra.run.smithi203.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-04-28T13:28:19.502 INFO:teuthology.orchestra.run.smithi203.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 658 kB/s | 43 kB 00:00 2024-04-28T13:28:19.552 INFO:teuthology.orchestra.run.smithi203.stdout:(115/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.3 MB/s | 839 kB 00:00 2024-04-28T13:28:19.577 INFO:teuthology.orchestra.run.smithi203.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 201 kB/s | 15 kB 00:00 2024-04-28T13:28:19.603 INFO:teuthology.orchestra.run.smithi203.stdout:(117/152): python3-cachetools-4.2.4-1.el9.noarc 257 kB/s | 32 kB 00:00 2024-04-28T13:28:19.637 INFO:teuthology.orchestra.run.smithi203.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-28T13:28:19.678 INFO:teuthology.orchestra.run.smithi203.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-28T13:28:19.714 INFO:teuthology.orchestra.run.smithi203.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-28T13:28:19.737 INFO:teuthology.orchestra.run.smithi203.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 357 kB/s | 35 kB 00:00 2024-04-28T13:28:19.771 INFO:teuthology.orchestra.run.smithi203.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-04-28T13:28:19.804 INFO:teuthology.orchestra.run.smithi203.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-28T13:28:19.830 INFO:teuthology.orchestra.run.smithi203.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 186 kB/s | 11 kB 00:00 2024-04-28T13:28:19.863 INFO:teuthology.orchestra.run.smithi203.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 310 kB/s | 18 kB 00:00 2024-04-28T13:28:19.888 INFO:teuthology.orchestra.run.smithi203.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 400 kB/s | 23 kB 00:00 2024-04-28T13:28:19.923 INFO:teuthology.orchestra.run.smithi203.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 337 kB/s | 19 kB 00:00 2024-04-28T13:28:19.948 INFO:teuthology.orchestra.run.smithi203.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 340 kB/s | 20 kB 00:00 2024-04-28T13:28:20.054 INFO:teuthology.orchestra.run.smithi203.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.0 MB/s | 2.0 MB 00:00 2024-04-28T13:28:20.082 INFO:teuthology.orchestra.run.smithi203.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 348 kB/s | 46 kB 00:00 2024-04-28T13:28:20.107 INFO:teuthology.orchestra.run.smithi203.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-28T13:28:20.132 INFO:teuthology.orchestra.run.smithi203.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-28T13:28:20.174 INFO:teuthology.orchestra.run.smithi203.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 4.1 MB/s | 1.0 MB 00:00 2024-04-28T13:28:20.199 INFO:teuthology.orchestra.run.smithi203.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-04-28T13:28:20.233 INFO:teuthology.orchestra.run.smithi203.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-28T13:28:20.259 INFO:teuthology.orchestra.run.smithi203.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 487 kB/s | 28 kB 00:00 2024-04-28T13:28:20.284 INFO:teuthology.orchestra.run.smithi203.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 831 kB/s | 90 kB 00:00 2024-04-28T13:28:20.309 INFO:teuthology.orchestra.run.smithi203.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 412 kB/s | 31 kB 00:00 2024-04-28T13:28:20.343 INFO:teuthology.orchestra.run.smithi203.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-28T13:28:20.368 INFO:teuthology.orchestra.run.smithi203.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 618 kB/s | 36 kB 00:00 2024-04-28T13:28:20.394 INFO:teuthology.orchestra.run.smithi203.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-04-28T13:28:20.419 INFO:teuthology.orchestra.run.smithi203.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 683 kB/s | 51 kB 00:00 2024-04-28T13:28:20.452 INFO:teuthology.orchestra.run.smithi203.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-28T13:28:20.486 INFO:teuthology.orchestra.run.smithi203.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 980 kB/s | 90 kB 00:00 2024-04-28T13:28:20.511 INFO:teuthology.orchestra.run.smithi203.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 348 kB/s | 20 kB 00:00 2024-04-28T13:28:20.545 INFO:teuthology.orchestra.run.smithi203.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-28T13:28:20.579 INFO:teuthology.orchestra.run.smithi203.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-28T13:28:20.729 INFO:teuthology.orchestra.run.smithi203.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-04-28T13:28:20.954 INFO:teuthology.orchestra.run.smithi203.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.0 MB/s | 1.3 MB 00:00 2024-04-28T13:28:21.063 INFO:teuthology.orchestra.run.smithi196.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 1.4 MB/s | 5.4 MB 00:03 2024-04-28T13:28:21.130 INFO:teuthology.orchestra.run.smithi196.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 1.2 MB/s | 2.4 MB 00:02 2024-04-28T13:28:21.188 INFO:teuthology.orchestra.run.smithi203.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 10 MB/s | 2.4 MB 00:00 2024-04-28T13:28:21.579 INFO:teuthology.orchestra.run.smithi203.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 3.1 MB/s | 3.1 MB 00:00 2024-04-28T13:28:21.980 INFO:teuthology.orchestra.run.smithi203.stdout:(152/152): librados2-19.0.0-3306.gbc612ffb.el9. 2.4 MB/s | 3.4 MB 00:01 2024-04-28T13:28:21.989 INFO:teuthology.orchestra.run.smithi196.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 1.6 MB/s | 1.5 MB 00:00 2024-04-28T13:28:21.992 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-04-28T13:28:21.992 INFO:teuthology.orchestra.run.smithi203.stdout:Total 15 MB/s | 222 MB 00:14 2024-04-28T13:28:23.533 INFO:teuthology.orchestra.run.smithi196.stdout:(7/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 3.4 MB/s | 21 MB 00:06 2024-04-28T13:28:24.762 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-04-28T13:28:24.922 INFO:teuthology.orchestra.run.smithi196.stdout:(8/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 1.2 MB/s | 4.7 MB 00:03 2024-04-28T13:28:24.997 INFO:teuthology.orchestra.run.smithi196.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 341 kB/s | 25 kB 00:00 2024-04-28T13:28:25.002 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-04-28T13:28:25.002 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-04-28T13:28:27.870 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-04-28T13:28:27.871 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-04-28T13:28:28.531 INFO:teuthology.orchestra.run.smithi196.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 2.1 MB/s | 11 MB 00:04 2024-04-28T13:28:28.647 INFO:teuthology.orchestra.run.smithi196.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.9 MB/s | 225 kB 00:00 2024-04-28T13:28:28.714 INFO:teuthology.orchestra.run.smithi196.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 410 kB/s | 27 kB 00:00 2024-04-28T13:28:28.806 INFO:teuthology.orchestra.run.smithi196.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 349 kB/s | 32 kB 00:00 2024-04-28T13:28:29.190 INFO:teuthology.orchestra.run.smithi196.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 1.9 MB/s | 747 kB 00:00 2024-04-28T13:28:29.357 INFO:teuthology.orchestra.run.smithi196.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 986 kB/s | 163 kB 00:00 2024-04-28T13:28:29.457 INFO:teuthology.orchestra.run.smithi196.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.3 MB/s | 127 kB 00:00 2024-04-28T13:28:29.774 INFO:teuthology.orchestra.run.smithi196.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 1.5 MB/s | 501 kB 00:00 2024-04-28T13:28:32.240 INFO:teuthology.orchestra.run.smithi196.stdout:(18/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 1.6 MB/s | 17 MB 00:10 2024-04-28T13:28:32.332 INFO:teuthology.orchestra.run.smithi196.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 497 kB/s | 45 kB 00:00 2024-04-28T13:28:32.466 INFO:teuthology.orchestra.run.smithi196.stdout:(20/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 2.0 MB/s | 5.4 MB 00:02 2024-04-28T13:28:32.525 INFO:teuthology.orchestra.run.smithi196.stdout:(21/152): python3-ceph-common-19.0.0-3306.gbc61 706 kB/s | 135 kB 00:00 2024-04-28T13:28:32.625 INFO:teuthology.orchestra.run.smithi196.stdout:(22/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.0 MB/s | 161 kB 00:00 2024-04-28T13:28:32.758 INFO:teuthology.orchestra.run.smithi196.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 1.3 MB/s | 321 kB 00:00 2024-04-28T13:28:32.859 INFO:teuthology.orchestra.run.smithi196.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 996 kB/s | 99 kB 00:00 2024-04-28T13:28:32.892 INFO:teuthology.orchestra.run.smithi196.stdout:(25/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 1.1 MB/s | 298 kB 00:00 2024-04-28T13:28:32.951 INFO:teuthology.orchestra.run.smithi196.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 937 kB/s | 85 kB 00:00 2024-04-28T13:28:33.135 INFO:teuthology.orchestra.run.smithi196.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 935 kB/s | 171 kB 00:00 2024-04-28T13:28:33.235 INFO:teuthology.orchestra.run.smithi196.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 343 kB/s | 34 kB 00:00 2024-04-28T13:28:33.410 INFO:teuthology.orchestra.run.smithi196.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 816 kB/s | 142 kB 00:00 2024-04-28T13:28:34.200 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-04-28T13:28:34.312 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-28T13:28:34.420 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-28T13:28:34.604 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-28T13:28:34.725 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-28T13:28:34.766 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-28T13:28:34.902 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-28T13:28:34.977 INFO:teuthology.orchestra.run.smithi196.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 1.5 MB/s | 3.1 MB 00:02 2024-04-28T13:28:35.309 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-28T13:28:35.421 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T13:28:35.586 INFO:teuthology.orchestra.run.smithi196.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 1.6 MB/s | 3.5 MB 00:02 2024-04-28T13:28:35.636 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T13:28:35.672 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T13:28:35.819 INFO:teuthology.orchestra.run.smithi196.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 1.0 MB/s | 249 kB 00:00 2024-04-28T13:28:35.835 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T13:28:35.874 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-28T13:28:35.919 INFO:teuthology.orchestra.run.smithi196.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 493 kB/s | 49 kB 00:00 2024-04-28T13:28:36.020 INFO:teuthology.orchestra.run.smithi196.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 166 kB/s | 16 kB 00:00 2024-04-28T13:28:36.240 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-28T13:28:36.270 INFO:teuthology.orchestra.run.smithi196.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.1 MB/s | 273 kB 00:00 2024-04-28T13:28:36.302 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-28T13:28:36.327 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T13:28:36.476 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T13:28:36.503 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-28T13:28:36.589 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-28T13:28:36.620 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-28T13:28:36.697 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-28T13:28:36.862 INFO:teuthology.orchestra.run.smithi196.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 1.3 MB/s | 762 kB 00:00 2024-04-28T13:28:36.979 INFO:teuthology.orchestra.run.smithi196.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 157 kB/s | 18 kB 00:00 2024-04-28T13:28:37.016 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-28T13:28:37.087 INFO:teuthology.orchestra.run.smithi196.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 753 kB/s | 80 kB 00:00 2024-04-28T13:28:37.089 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-28T13:28:37.229 INFO:teuthology.orchestra.run.smithi196.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.6 MB/s | 803 kB 00:00 2024-04-28T13:28:37.280 INFO:teuthology.orchestra.run.smithi196.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-28T13:28:37.289 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-28T13:28:37.330 INFO:teuthology.orchestra.run.smithi196.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-28T13:28:37.408 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-28T13:28:37.422 INFO:teuthology.orchestra.run.smithi196.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-04-28T13:28:37.464 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-28T13:28:37.472 INFO:teuthology.orchestra.run.smithi196.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-28T13:28:37.512 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-28T13:28:37.522 INFO:teuthology.orchestra.run.smithi196.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-28T13:28:37.573 INFO:teuthology.orchestra.run.smithi196.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-28T13:28:37.617 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-28T13:28:37.623 INFO:teuthology.orchestra.run.smithi196.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.3 MB/s | 215 kB 00:00 2024-04-28T13:28:37.668 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-28T13:28:37.730 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-28T13:28:37.782 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-28T13:28:37.821 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-28T13:28:37.823 INFO:teuthology.orchestra.run.smithi196.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-04-28T13:28:37.915 INFO:teuthology.orchestra.run.smithi196.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-04-28T13:28:37.938 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-28T13:28:37.965 INFO:teuthology.orchestra.run.smithi196.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-28T13:28:38.011 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-28T13:28:38.015 INFO:teuthology.orchestra.run.smithi196.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 435 kB/s | 21 kB 00:00 2024-04-28T13:28:38.048 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-28T13:28:38.082 INFO:teuthology.orchestra.run.smithi196.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-28T13:28:38.107 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-28T13:28:38.133 INFO:teuthology.orchestra.run.smithi196.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 657 kB/s | 33 kB 00:00 2024-04-28T13:28:38.214 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-28T13:28:38.259 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-28T13:28:38.314 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-28T13:28:38.325 INFO:teuthology.orchestra.run.smithi196.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-04-28T13:28:38.350 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-28T13:28:38.375 INFO:teuthology.orchestra.run.smithi196.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 370 kB/s | 18 kB 00:00 2024-04-28T13:28:38.426 INFO:teuthology.orchestra.run.smithi196.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 227 kB/s | 11 kB 00:00 2024-04-28T13:28:38.458 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-28T13:28:38.484 INFO:teuthology.orchestra.run.smithi196.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-28T13:28:38.519 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-28T13:28:38.534 INFO:teuthology.orchestra.run.smithi196.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 914 kB/s | 45 kB 00:00 2024-04-28T13:28:38.601 INFO:teuthology.orchestra.run.smithi196.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.9 MB/s | 662 kB 00:00 2024-04-28T13:28:38.618 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-28T13:28:38.668 INFO:teuthology.orchestra.run.smithi196.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 582 kB 00:00 2024-04-28T13:28:38.719 INFO:teuthology.orchestra.run.smithi196.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 516 kB/s | 25 kB 00:00 2024-04-28T13:28:38.794 INFO:teuthology.orchestra.run.smithi196.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 638 kB/s | 47 kB 00:00 2024-04-28T13:28:38.795 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-28T13:28:38.853 INFO:teuthology.orchestra.run.smithi196.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.0 MB/s | 292 kB 00:00 2024-04-28T13:28:38.857 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-28T13:28:38.903 INFO:teuthology.orchestra.run.smithi196.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 3.8 MB/s | 188 kB 00:00 2024-04-28T13:28:38.953 INFO:teuthology.orchestra.run.smithi196.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 713 kB/s | 35 kB 00:00 2024-04-28T13:28:39.128 INFO:teuthology.orchestra.run.smithi196.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-28T13:28:39.296 INFO:teuthology.orchestra.run.smithi196.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-04-28T13:28:39.387 INFO:teuthology.orchestra.run.smithi196.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-28T13:28:39.471 INFO:teuthology.orchestra.run.smithi196.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-28T13:28:39.521 INFO:teuthology.orchestra.run.smithi196.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.7 MB/s | 133 kB 00:00 2024-04-28T13:28:39.566 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-28T13:28:39.738 INFO:teuthology.orchestra.run.smithi196.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-04-28T13:28:39.741 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-28T13:28:39.783 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-28T13:28:39.805 INFO:teuthology.orchestra.run.smithi196.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-28T13:28:39.830 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-28T13:28:39.855 INFO:teuthology.orchestra.run.smithi196.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-28T13:28:39.879 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-28T13:28:39.954 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-28T13:28:40.047 INFO:teuthology.orchestra.run.smithi196.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 31 MB/s | 6.0 MB 00:00 2024-04-28T13:28:40.098 INFO:teuthology.orchestra.run.smithi196.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-04-28T13:28:40.156 INFO:teuthology.orchestra.run.smithi196.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-28T13:28:40.198 INFO:teuthology.orchestra.run.smithi196.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 616 kB/s | 26 kB 00:00 2024-04-28T13:28:40.237 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-28T13:28:40.240 INFO:teuthology.orchestra.run.smithi196.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 466 kB/s | 19 kB 00:00 2024-04-28T13:28:40.321 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-28T13:28:40.332 INFO:teuthology.orchestra.run.smithi196.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-28T13:28:40.399 INFO:teuthology.orchestra.run.smithi196.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 527 kB/s | 35 kB 00:00 2024-04-28T13:28:40.407 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-28T13:28:40.511 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-28T13:28:40.553 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-28T13:28:40.592 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-28T13:28:40.616 INFO:teuthology.orchestra.run.smithi196.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-04-28T13:28:40.678 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-28T13:28:40.683 INFO:teuthology.orchestra.run.smithi196.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-28T13:28:40.726 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-28T13:28:40.733 INFO:teuthology.orchestra.run.smithi196.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-28T13:28:40.784 INFO:teuthology.orchestra.run.smithi196.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-28T13:28:40.790 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-28T13:28:40.834 INFO:teuthology.orchestra.run.smithi196.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 859 kB/s | 42 kB 00:00 2024-04-28T13:28:40.854 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-28T13:28:40.901 INFO:teuthology.orchestra.run.smithi196.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-04-28T13:28:40.951 INFO:teuthology.orchestra.run.smithi196.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-28T13:28:41.010 INFO:teuthology.orchestra.run.smithi196.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-28T13:28:41.053 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-28T13:28:41.060 INFO:teuthology.orchestra.run.smithi196.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-28T13:28:41.108 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-28T13:28:41.111 INFO:teuthology.orchestra.run.smithi196.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-28T13:28:41.145 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-28T13:28:41.201 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-28T13:28:41.331 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-28T13:28:41.378 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-28T13:28:41.426 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-28T13:28:41.460 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-28T13:28:41.493 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-28T13:28:41.653 INFO:teuthology.orchestra.run.smithi196.stdout:(90/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-04-28T13:28:41.703 INFO:teuthology.orchestra.run.smithi196.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 853 kB/s | 42 kB 00:00 2024-04-28T13:28:41.705 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-28T13:28:41.753 INFO:teuthology.orchestra.run.smithi196.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 897 kB/s | 44 kB 00:00 2024-04-28T13:28:41.837 INFO:teuthology.orchestra.run.smithi196.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-28T13:28:41.887 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-28T13:28:41.920 INFO:teuthology.orchestra.run.smithi196.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 435 kB/s | 36 kB 00:00 2024-04-28T13:28:41.934 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-28T13:28:41.976 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-28T13:28:41.987 INFO:teuthology.orchestra.run.smithi196.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 719 kB/s | 47 kB 00:00 2024-04-28T13:28:42.017 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-28T13:28:42.054 INFO:teuthology.orchestra.run.smithi196.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-28T13:28:42.076 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-28T13:28:42.113 INFO:teuthology.orchestra.run.smithi196.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-28T13:28:42.138 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-28T13:28:42.177 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-28T13:28:42.197 INFO:teuthology.orchestra.run.smithi196.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-28T13:28:42.225 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-28T13:28:42.275 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-28T13:28:42.280 INFO:teuthology.orchestra.run.smithi196.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 773 kB/s | 64 kB 00:00 2024-04-28T13:28:42.349 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-28T13:28:42.372 INFO:teuthology.orchestra.run.smithi196.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 492 kB/s | 45 kB 00:00 2024-04-28T13:28:42.399 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-28T13:28:42.572 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-28T13:28:42.641 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-28T13:28:42.681 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-28T13:28:42.750 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-28T13:28:42.831 INFO:teuthology.orchestra.run.smithi196.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 49 kB/s | 22 kB 00:00 2024-04-28T13:28:42.898 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-28T13:28:42.962 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-28T13:28:43.622 INFO:teuthology.orchestra.run.smithi196.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-04-28T13:28:43.848 INFO:teuthology.orchestra.run.smithi196.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.4 MB/s | 548 kB 00:00 2024-04-28T13:28:43.914 INFO:teuthology.orchestra.run.smithi196.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 4.6 MB/s | 309 kB 00:00 2024-04-28T13:28:43.931 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-28T13:28:43.965 INFO:teuthology.orchestra.run.smithi196.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 391 kB/s | 19 kB 00:00 2024-04-28T13:28:43.995 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-28T13:28:44.028 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-28T13:28:44.154 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-28T13:28:44.174 INFO:teuthology.orchestra.run.smithi196.stdout:(106/152): libarrow-9.0.0-11.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-28T13:28:44.224 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-28T13:28:44.226 INFO:teuthology.orchestra.run.smithi196.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 509 kB/s | 26 kB 00:00 2024-04-28T13:28:44.268 INFO:teuthology.orchestra.run.smithi196.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 1.2 MB/s | 49 kB 00:00 2024-04-28T13:28:44.318 INFO:teuthology.orchestra.run.smithi196.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-28T13:28:44.332 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-28T13:28:44.369 INFO:teuthology.orchestra.run.smithi196.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-28T13:28:44.380 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-28T13:28:44.502 INFO:teuthology.orchestra.run.smithi196.stdout:(111/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.2 MB/s | 839 kB 00:00 2024-04-28T13:28:44.569 INFO:teuthology.orchestra.run.smithi196.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-04-28T13:28:44.616 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-28T13:28:44.619 INFO:teuthology.orchestra.run.smithi196.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 885 kB/s | 43 kB 00:00 2024-04-28T13:28:44.661 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-28T13:28:44.670 INFO:teuthology.orchestra.run.smithi196.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 653 kB/s | 32 kB 00:00 2024-04-28T13:28:44.720 INFO:teuthology.orchestra.run.smithi196.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 305 kB/s | 15 kB 00:00 2024-04-28T13:28:44.770 INFO:teuthology.orchestra.run.smithi196.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-28T13:28:44.770 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-28T13:28:44.829 INFO:teuthology.orchestra.run.smithi196.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 6.1 MB/s | 358 kB 00:00 2024-04-28T13:28:44.896 INFO:teuthology.orchestra.run.smithi196.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-04-28T13:28:44.946 INFO:teuthology.orchestra.run.smithi196.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 717 kB/s | 35 kB 00:00 2024-04-28T13:28:44.996 INFO:teuthology.orchestra.run.smithi196.stdout:(120/152): python3-google-auth-2.29.0-1.el9.noa 4.1 MB/s | 206 kB 00:00 2024-04-28T13:28:45.114 INFO:teuthology.orchestra.run.smithi196.stdout:(121/152): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-04-28T13:28:45.164 INFO:teuthology.orchestra.run.smithi196.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.9 MB/s | 144 kB 00:00 2024-04-28T13:28:45.214 INFO:teuthology.orchestra.run.smithi196.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 216 kB/s | 11 kB 00:00 2024-04-28T13:28:45.265 INFO:teuthology.orchestra.run.smithi196.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 359 kB/s | 18 kB 00:00 2024-04-28T13:28:45.315 INFO:teuthology.orchestra.run.smithi196.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 468 kB/s | 23 kB 00:00 2024-04-28T13:28:45.365 INFO:teuthology.orchestra.run.smithi196.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 395 kB/s | 19 kB 00:00 2024-04-28T13:28:45.619 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-28T13:28:45.640 INFO:teuthology.orchestra.run.smithi196.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 72 kB/s | 20 kB 00:00 2024-04-28T13:28:45.724 INFO:teuthology.orchestra.run.smithi196.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-28T13:28:45.761 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-28T13:28:45.774 INFO:teuthology.orchestra.run.smithi196.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 938 kB/s | 46 kB 00:00 2024-04-28T13:28:45.825 INFO:teuthology.orchestra.run.smithi196.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2024-04-28T13:28:45.875 INFO:teuthology.orchestra.run.smithi196.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-28T13:28:45.959 INFO:teuthology.orchestra.run.smithi196.stdout:(132/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-28T13:28:46.009 INFO:teuthology.orchestra.run.smithi196.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 333 kB/s | 16 kB 00:00 2024-04-28T13:28:46.060 INFO:teuthology.orchestra.run.smithi196.stdout:(134/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-28T13:28:46.110 INFO:teuthology.orchestra.run.smithi196.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 573 kB/s | 28 kB 00:00 2024-04-28T13:28:46.160 INFO:teuthology.orchestra.run.smithi196.stdout:(136/152): python3-repoze-lru-0.7-16.el9.noarch 626 kB/s | 31 kB 00:00 2024-04-28T13:28:46.210 INFO:teuthology.orchestra.run.smithi196.stdout:(137/152): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-28T13:28:46.252 INFO:teuthology.orchestra.run.smithi196.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-04-28T13:28:46.294 INFO:teuthology.orchestra.run.smithi196.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 873 kB/s | 36 kB 00:00 2024-04-28T13:28:46.336 INFO:teuthology.orchestra.run.smithi196.stdout:(140/152): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-04-28T13:28:46.395 INFO:teuthology.orchestra.run.smithi196.stdout:(141/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2024-04-28T13:28:46.445 INFO:teuthology.orchestra.run.smithi196.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-28T13:28:46.587 INFO:teuthology.orchestra.run.smithi196.stdout:(143/152): ceph-mgr-diskprediction-local-19.0.0 653 kB/s | 7.4 MB 00:11 2024-04-28T13:28:46.629 INFO:teuthology.orchestra.run.smithi196.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.3 MB/s | 427 kB 00:00 2024-04-28T13:28:46.687 INFO:teuthology.orchestra.run.smithi196.stdout:(145/152): re2-20211101-3.el9.x86_64.rpm 3.3 MB/s | 192 kB 00:00 2024-04-28T13:28:46.713 INFO:teuthology.orchestra.run.smithi196.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2024-04-28T13:28:46.788 INFO:teuthology.orchestra.run.smithi196.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-28T13:28:47.401 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-28T13:28:47.511 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-28T13:28:47.552 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-28T13:28:47.595 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-28T13:28:47.986 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-28T13:28:48.032 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-28T13:28:48.186 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-28T13:28:48.222 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-28T13:28:48.284 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-28T13:28:48.834 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-28T13:28:48.869 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-28T13:28:48.970 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-28T13:28:48.996 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-28T13:28:49.013 INFO:teuthology.orchestra.run.smithi196.stdout:(148/152): librados2-19.0.0-3306.gbc612ffb.el9. 1.5 MB/s | 3.4 MB 00:02 2024-04-28T13:28:49.213 INFO:teuthology.orchestra.run.smithi196.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.8 MB/s | 1.3 MB 00:00 2024-04-28T13:28:49.330 INFO:teuthology.orchestra.run.smithi196.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-04-28T13:28:51.272 INFO:teuthology.orchestra.run.smithi196.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 718 kB/s | 3.1 MB 00:04 2024-04-28T13:28:51.843 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T13:28:51.893 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T13:28:52.567 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T13:28:52.622 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-28T13:28:52.790 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-28T13:28:52.790 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-28T13:28:52.790 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-28T13:28:52.790 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:28:52.831 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-28T13:28:56.171 INFO:teuthology.orchestra.run.smithi196.stdout:(152/152): ceph-test-19.0.0-3306.gbc612ffb.el9. 1.6 MB/s | 49 MB 00:31 2024-04-28T13:28:56.174 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-04-28T13:28:56.174 INFO:teuthology.orchestra.run.smithi196.stdout:Total 5.7 MB/s | 222 MB 00:39 2024-04-28T13:28:58.827 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-04-28T13:28:59.068 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-04-28T13:28:59.068 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-04-28T13:29:01.770 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-04-28T13:29:01.770 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-04-28T13:29:05.732 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-28T13:29:05.733 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /sys 2024-04-28T13:29:05.733 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /proc 2024-04-28T13:29:05.733 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /mnt 2024-04-28T13:29:05.733 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /var/tmp 2024-04-28T13:29:05.733 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /home 2024-04-28T13:29:05.733 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /root 2024-04-28T13:29:05.733 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /tmp 2024-04-28T13:29:05.733 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:29:05.966 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-28T13:29:06.014 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-28T13:29:06.014 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:06.014 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-28T13:29:06.014 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:29:07.669 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-28T13:29:07.758 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-28T13:29:07.758 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:07.759 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-28T13:29:07.759 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-28T13:29:07.759 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-28T13:29:07.759 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:29:08.011 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-28T13:29:08.058 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-04-28T13:29:08.100 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-28T13:29:08.100 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:08.100 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-28T13:29:08.100 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-28T13:29:08.100 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-28T13:29:08.101 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:29:08.194 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-28T13:29:08.292 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-28T13:29:08.404 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-28T13:29:08.480 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-28T13:29:08.520 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-28T13:29:08.595 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-28T13:29:08.639 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-28T13:29:08.684 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-28T13:29:08.684 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:08.684 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-28T13:29:08.684 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-28T13:29:08.684 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-28T13:29:08.684 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:29:08.812 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-28T13:29:08.873 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-28T13:29:08.950 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-28T13:29:09.089 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-28T13:29:09.184 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T13:29:09.190 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-28T13:29:09.339 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-28T13:29:09.355 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T13:29:09.385 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T13:29:09.510 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T13:29:09.542 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-28T13:29:09.581 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-28T13:29:09.696 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-28T13:29:09.860 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-28T13:29:09.864 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-28T13:29:09.922 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-28T13:29:09.931 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-28T13:29:09.956 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T13:29:09.967 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-28T13:29:09.999 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-28T13:29:10.112 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T13:29:10.148 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-28T13:29:10.237 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-28T13:29:10.274 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-28T13:29:10.355 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-28T13:29:10.634 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-28T13:29:10.669 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-28T13:29:10.698 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-28T13:29:10.749 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-28T13:29:10.750 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-28T13:29:10.861 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-28T13:29:10.896 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-28T13:29:10.955 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-28T13:29:11.075 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-28T13:29:11.116 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-28T13:29:11.183 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-28T13:29:11.286 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-28T13:29:11.337 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-28T13:29:11.402 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-28T13:29:11.450 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-28T13:29:11.489 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-28T13:29:11.592 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-28T13:29:11.659 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-28T13:29:11.700 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-28T13:29:11.752 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-28T13:29:11.859 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-28T13:29:11.894 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-28T13:29:11.944 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-28T13:29:11.975 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-28T13:29:12.083 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-28T13:29:12.147 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-28T13:29:12.226 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-28T13:29:12.254 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-28T13:29:12.274 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-28T13:29:12.430 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-28T13:29:12.491 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-28T13:29:13.163 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-28T13:29:13.272 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-28T13:29:13.308 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-28T13:29:13.353 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-28T13:29:13.374 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-28T13:29:13.393 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-28T13:29:13.407 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-28T13:29:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-28T13:29:13.554 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-28T13:29:13.664 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-28T13:29:13.730 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-28T13:29:13.735 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-28T13:29:13.768 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-28T13:29:13.807 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-28T13:29:13.848 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-28T13:29:13.848 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:13.848 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-28T13:29:13.848 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-28T13:29:13.848 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-28T13:29:13.849 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:29:13.910 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-28T13:29:13.913 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-28T13:29:13.948 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-28T13:29:13.952 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-28T13:29:13.991 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-28T13:29:14.059 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-28T13:29:14.090 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-28T13:29:14.107 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-28T13:29:14.154 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-28T13:29:14.173 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-28T13:29:14.236 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-28T13:29:14.432 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-28T13:29:14.489 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-28T13:29:14.526 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-28T13:29:14.582 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-28T13:29:14.710 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-28T13:29:14.750 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-28T13:29:14.798 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-28T13:29:14.832 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-28T13:29:14.865 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-28T13:29:15.071 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-28T13:29:15.152 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-28T13:29:15.188 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-28T13:29:15.264 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-28T13:29:15.265 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:15.265 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-28T13:29:15.265 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-28T13:29:15.265 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-28T13:29:15.265 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:29:15.298 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-28T13:29:15.314 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-28T13:29:15.399 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-28T13:29:15.399 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:15.399 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-28T13:29:15.399 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:29:15.430 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-28T13:29:15.472 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-28T13:29:15.482 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-28T13:29:15.530 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-28T13:29:15.581 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-28T13:29:15.581 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:15.581 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-28T13:29:15.581 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-28T13:29:15.581 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-28T13:29:15.582 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:29:15.596 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-28T13:29:15.679 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-28T13:29:15.728 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-28T13:29:15.783 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-28T13:29:15.831 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-28T13:29:15.905 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-28T13:29:15.953 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-28T13:29:15.957 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-28T13:29:16.030 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-28T13:29:16.031 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:16.031 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-28T13:29:16.031 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-28T13:29:16.031 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-28T13:29:16.031 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:29:16.092 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-28T13:29:16.159 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-28T13:29:16.210 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-28T13:29:16.281 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-28T13:29:16.429 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-28T13:29:16.489 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-28T13:29:17.477 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-28T13:29:17.530 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-28T13:29:17.563 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-28T13:29:17.683 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-28T13:29:17.749 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-28T13:29:17.848 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-28T13:29:17.890 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-28T13:29:18.120 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-28T13:29:18.145 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-28T13:29:18.245 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-28T13:29:19.056 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-28T13:29:19.199 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-28T13:29:20.848 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-28T13:29:20.959 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-28T13:29:21.021 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-28T13:29:21.073 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-28T13:29:21.108 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-28T13:29:21.166 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-28T13:29:21.258 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-28T13:29:21.304 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-28T13:29:21.352 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-28T13:29:21.462 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-28T13:29:21.496 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-28T13:29:21.599 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-28T13:29:21.636 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-28T13:29:21.692 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-28T13:29:21.692 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-28T13:29:21.745 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-28T13:29:21.773 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-28T13:29:22.268 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-28T13:29:22.308 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-28T13:29:22.399 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-28T13:29:22.426 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-28T13:29:22.995 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T13:29:23.050 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T13:29:23.104 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T13:29:24.469 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-28T13:29:24.589 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-28T13:29:24.704 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-28T13:29:24.881 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-28T13:29:24.929 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-28T13:29:24.929 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-28T13:29:25.053 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-28T13:29:25.103 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-28T13:29:25.103 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-28T13:29:25.121 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T13:29:25.164 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T13:29:25.243 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-28T13:29:25.383 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T13:29:25.651 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T13:29:25.651 INFO:teuthology.orchestra.run.smithi203.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-28T13:29:25.652 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:29:25.821 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T13:29:25.868 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-28T13:29:26.028 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-28T13:29:26.028 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-28T13:29:26.029 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-28T13:29:26.029 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:29:26.068 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-28T13:29:31.622 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-28T13:29:31.622 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-28T13:29:31.622 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-28T13:29:31.622 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-28T13:29:31.623 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-28T13:29:31.623 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-28T13:29:31.623 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-28T13:29:31.623 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T13:29:31.623 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T13:29:31.623 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-28T13:29:31.623 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-28T13:29:31.623 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-28T13:29:31.623 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T13:29:31.623 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-28T13:29:31.623 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-28T13:29:31.623 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-28T13:29:31.623 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-28T13:29:31.623 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-28T13:29:31.624 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-28T13:29:31.624 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-28T13:29:31.624 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-28T13:29:31.624 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-28T13:29:31.624 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-28T13:29:31.624 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-28T13:29:31.624 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-28T13:29:31.624 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-28T13:29:31.624 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-28T13:29:31.624 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-28T13:29:31.624 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-28T13:29:31.624 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-28T13:29:31.636 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-28T13:29:31.636 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-28T13:29:31.637 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-28T13:29:31.637 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-28T13:29:31.637 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-28T13:29:31.637 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-28T13:29:31.637 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-28T13:29:31.637 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-28T13:29:31.637 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-28T13:29:31.637 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-28T13:29:31.637 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-28T13:29:31.637 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-28T13:29:31.637 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-28T13:29:31.637 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-28T13:29:31.637 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-28T13:29:31.638 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-28T13:29:31.638 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-28T13:29:31.638 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-28T13:29:31.638 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-28T13:29:31.638 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-28T13:29:31.638 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-28T13:29:31.638 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-28T13:29:31.638 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-28T13:29:31.638 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-28T13:29:31.638 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-28T13:29:31.638 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-28T13:29:31.638 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-28T13:29:31.639 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-28T13:29:31.639 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-28T13:29:31.639 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-28T13:29:31.639 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-28T13:29:31.639 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-28T13:29:31.639 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-28T13:29:31.639 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-28T13:29:31.639 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-28T13:29:31.639 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-28T13:29:31.639 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-28T13:29:31.639 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-28T13:29:31.639 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-28T13:29:31.639 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-28T13:29:31.639 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-28T13:29:31.639 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-28T13:29:31.640 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-28T13:29:31.640 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-28T13:29:31.640 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-28T13:29:31.640 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-28T13:29:31.640 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-28T13:29:31.640 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-28T13:29:31.640 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-28T13:29:31.640 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-28T13:29:31.640 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-28T13:29:31.640 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-28T13:29:31.640 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-28T13:29:31.640 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-28T13:29:31.640 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-28T13:29:31.640 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-28T13:29:31.641 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-28T13:29:31.641 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-28T13:29:31.641 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-28T13:29:31.641 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-28T13:29:31.641 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-28T13:29:31.641 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-28T13:29:31.641 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-28T13:29:31.641 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-28T13:29:31.641 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-28T13:29:31.641 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-28T13:29:31.641 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-28T13:29:31.641 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-28T13:29:31.641 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-28T13:29:31.641 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-28T13:29:31.642 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-28T13:29:31.642 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-28T13:29:31.642 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-28T13:29:31.642 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-28T13:29:31.642 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-28T13:29:31.642 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-28T13:29:31.642 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-28T13:29:31.642 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-28T13:29:31.642 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-28T13:29:31.642 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-28T13:29:31.642 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-28T13:29:31.642 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-28T13:29:31.642 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-28T13:29:31.642 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-28T13:29:31.643 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-28T13:29:31.643 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-28T13:29:31.643 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-28T13:29:31.643 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-28T13:29:31.643 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-28T13:29:31.643 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-28T13:29:31.643 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-28T13:29:31.643 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-28T13:29:31.643 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-28T13:29:31.643 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-28T13:29:31.643 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-28T13:29:31.643 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-28T13:29:31.643 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-28T13:29:31.643 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-28T13:29:31.643 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-28T13:29:31.644 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-28T13:29:31.644 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-28T13:29:31.644 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-28T13:29:31.644 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-28T13:29:31.644 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-28T13:29:31.644 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-28T13:29:31.644 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-28T13:29:31.644 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-28T13:29:31.644 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-28T13:29:31.644 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-28T13:29:31.644 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-28T13:29:31.644 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-28T13:29:31.644 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-28T13:29:31.644 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-28T13:29:31.644 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-28T13:29:31.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-28T13:29:31.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-28T13:29:31.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-28T13:29:31.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-28T13:29:31.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-28T13:29:31.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-28T13:29:31.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-28T13:29:31.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-28T13:29:31.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-28T13:29:31.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-28T13:29:31.645 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-28T13:29:34.043 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T13:29:34.043 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:29:34.043 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2024-04-28T13:29:34.043 INFO:teuthology.orchestra.run.smithi203.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.043 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.043 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-28T13:29:34.043 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-28T13:29:34.043 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-04-28T13:29:34.043 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-28T13:29:34.043 INFO:teuthology.orchestra.run.smithi203.stdout: automake-1.16.2-8.el9.noarch 2024-04-28T13:29:34.043 INFO:teuthology.orchestra.run.smithi203.stdout: bison-3.7.4-5.el9.x86_64 2024-04-28T13:29:34.043 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-28T13:29:34.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:29:34.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:29:34.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:29:34.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:29:34.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:29:34.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:29:34.045 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.045 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.045 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:29:34.045 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.045 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.045 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.045 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:29:34.045 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:29:34.045 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.045 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.045 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:29:34.045 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-28T13:29:34.045 INFO:teuthology.orchestra.run.smithi203.stdout: flex-2.6.4-9.el9.x86_64 2024-04-28T13:29:34.045 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-28T13:29:34.046 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-28T13:29:34.046 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-28T13:29:34.046 INFO:teuthology.orchestra.run.smithi203.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-28T13:29:34.046 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-28T13:29:34.046 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-28T13:29:34.046 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-28T13:29:34.046 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-28T13:29:34.046 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-28T13:29:34.046 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.046 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.046 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.046 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-28T13:29:34.046 INFO:teuthology.orchestra.run.smithi203.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-28T13:29:34.047 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-28T13:29:34.047 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-28T13:29:34.047 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-28T13:29:34.047 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.047 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.047 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-28T13:29:34.047 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.047 INFO:teuthology.orchestra.run.smithi203.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-28T13:29:34.047 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-28T13:29:34.047 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-28T13:29:34.047 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-28T13:29:34.047 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-28T13:29:34.047 INFO:teuthology.orchestra.run.smithi203.stdout: lua-5.4.4-4.el9.x86_64 2024-04-28T13:29:34.047 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-28T13:29:34.048 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-28T13:29:34.048 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-28T13:29:34.048 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-28T13:29:34.048 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-28T13:29:34.048 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-28T13:29:34.048 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-13.14-1.el9.x86_64 2024-04-28T13:29:34.048 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-28T13:29:34.048 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-28T13:29:34.048 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-28T13:29:34.048 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-28T13:29:34.048 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-28T13:29:34.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-28T13:29:34.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-28T13:29:34.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-28T13:29:34.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-28T13:29:34.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-28T13:29:34.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-28T13:29:34.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-28T13:29:34.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-28T13:29:34.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-28T13:29:34.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-28T13:29:34.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-28T13:29:34.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-28T13:29:34.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-28T13:29:34.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-28T13:29:34.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-28T13:29:34.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-28T13:29:34.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-28T13:29:34.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-28T13:29:34.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-28T13:29:34.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-28T13:29:34.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-28T13:29:34.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-28T13:29:34.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-28T13:29:34.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-28T13:29:34.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-28T13:29:34.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-28T13:29:34.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-28T13:29:34.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-28T13:29:34.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-28T13:29:34.051 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-28T13:29:34.051 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-28T13:29:34.051 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-28T13:29:34.051 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-28T13:29:34.051 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-28T13:29:34.051 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-28T13:29:34.051 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply-3.11-14.el9.noarch 2024-04-28T13:29:34.051 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-28T13:29:34.051 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-28T13:29:34.051 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-28T13:29:34.051 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-28T13:29:34.051 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-28T13:29:34.051 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-28T13:29:34.051 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-28T13:29:34.051 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-28T13:29:34.052 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-28T13:29:34.052 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.052 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.052 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-28T13:29:34.052 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-28T13:29:34.052 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-28T13:29:34.052 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.052 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-28T13:29:34.052 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-28T13:29:34.052 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-28T13:29:34.052 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-28T13:29:34.052 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-28T13:29:34.052 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-28T13:29:34.052 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-28T13:29:34.053 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-28T13:29:34.053 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-28T13:29:34.053 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-28T13:29:34.053 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-28T13:29:34.053 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-28T13:29:34.053 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-28T13:29:34.053 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-28T13:29:34.053 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-28T13:29:34.053 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.053 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.053 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:29:34.053 INFO:teuthology.orchestra.run.smithi203.stdout: re2-1:20211101-3.el9.x86_64 2024-04-28T13:29:34.053 INFO:teuthology.orchestra.run.smithi203.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-28T13:29:34.053 INFO:teuthology.orchestra.run.smithi203.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-28T13:29:34.054 INFO:teuthology.orchestra.run.smithi203.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-28T13:29:34.054 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-28T13:29:34.054 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-28T13:29:34.054 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-28T13:29:34.054 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:29:34.054 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-04-28T13:29:34.627 DEBUG:teuthology.parallel:result is None 2024-04-28T13:29:38.860 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-28T13:29:38.860 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-04-28T13:29:38.860 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-04-28T13:29:38.860 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-04-28T13:29:38.860 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-04-28T13:29:38.861 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-04-28T13:29:38.861 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-04-28T13:29:38.861 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-04-28T13:29:38.861 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:29:39.072 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-28T13:29:39.116 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-28T13:29:39.116 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:39.116 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-28T13:29:39.116 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:29:40.707 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-28T13:29:40.811 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-28T13:29:40.811 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:40.811 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-28T13:29:40.811 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-28T13:29:40.811 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-28T13:29:40.811 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:29:41.057 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-28T13:29:41.144 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-28T13:29:41.144 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:41.144 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-28T13:29:41.144 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-28T13:29:41.144 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-28T13:29:41.144 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:29:41.633 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-28T13:29:41.721 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-28T13:29:41.721 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:41.721 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-28T13:29:41.721 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-28T13:29:41.721 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-28T13:29:41.721 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:29:41.860 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-28T13:29:41.921 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-28T13:29:41.994 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-28T13:29:42.225 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-28T13:29:42.287 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-28T13:29:42.373 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-28T13:29:42.480 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-28T13:29:42.639 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-28T13:29:42.695 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-28T13:29:42.735 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-28T13:29:42.768 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-28T13:29:43.354 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-28T13:29:43.388 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-28T13:29:43.443 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-28T13:29:43.552 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-28T13:29:43.582 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-28T13:29:44.885 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-28T13:29:44.925 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-28T13:29:46.022 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-28T13:29:46.058 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-28T13:29:46.203 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-28T13:29:46.382 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-28T13:29:46.418 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-28T13:29:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-28T13:29:46.492 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:46.492 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-28T13:29:46.492 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-28T13:29:46.492 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-28T13:29:46.492 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:29:46.570 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-28T13:29:46.605 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-28T13:29:46.763 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-28T13:29:46.829 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-28T13:29:47.836 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-28T13:29:47.880 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-28T13:29:47.948 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-28T13:29:47.949 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:47.949 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-28T13:29:47.949 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-28T13:29:47.949 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-28T13:29:47.949 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:29:47.997 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-28T13:29:48.065 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-28T13:29:48.066 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:48.066 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-28T13:29:48.066 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:29:48.140 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-28T13:29:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-28T13:29:48.215 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:48.215 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-28T13:29:48.215 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-28T13:29:48.215 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-28T13:29:48.215 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:29:48.574 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-28T13:29:48.649 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-28T13:29:48.649 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T13:29:48.649 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-28T13:29:48.649 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-28T13:29:48.649 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-28T13:29:48.649 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:29:53.717 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-28T13:29:53.772 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-28T13:29:53.863 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-28T13:29:53.910 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-28T13:29:53.960 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-28T13:29:54.292 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-28T13:29:54.352 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-28T13:29:54.380 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-28T13:29:55.275 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T13:29:55.327 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T13:29:55.383 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T13:29:56.699 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-28T13:29:56.817 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-28T13:29:56.915 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-28T13:29:57.089 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-28T13:29:57.131 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-28T13:29:57.131 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-28T13:29:57.250 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-28T13:29:57.321 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-28T13:29:57.322 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-28T13:29:57.453 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-28T13:29:57.585 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T13:29:57.864 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T13:29:57.865 INFO:teuthology.orchestra.run.smithi196.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-28T13:29:57.867 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:03.537 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-28T13:30:03.537 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-28T13:30:03.537 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-28T13:30:03.537 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-28T13:30:03.537 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-28T13:30:03.537 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-28T13:30:03.537 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-28T13:30:03.537 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T13:30:03.537 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T13:30:03.537 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-28T13:30:03.538 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-28T13:30:03.538 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-28T13:30:03.538 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T13:30:03.538 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-28T13:30:03.538 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-28T13:30:03.538 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-28T13:30:03.538 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-28T13:30:03.538 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-28T13:30:03.538 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-28T13:30:03.538 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-28T13:30:03.538 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-28T13:30:03.538 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-28T13:30:03.538 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-28T13:30:03.538 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-28T13:30:03.539 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-28T13:30:03.539 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-28T13:30:03.539 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-28T13:30:03.539 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-28T13:30:03.539 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-28T13:30:03.539 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-28T13:30:03.550 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-28T13:30:03.550 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-28T13:30:03.550 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-28T13:30:03.551 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-28T13:30:03.551 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-28T13:30:03.551 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-28T13:30:03.551 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-28T13:30:03.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-28T13:30:03.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-28T13:30:03.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-28T13:30:03.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-28T13:30:03.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-28T13:30:03.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-28T13:30:03.553 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-28T13:30:03.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-28T13:30:03.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-28T13:30:03.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-28T13:30:03.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-28T13:30:03.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-28T13:30:03.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-28T13:30:03.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-28T13:30:03.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-28T13:30:03.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-28T13:30:03.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-28T13:30:03.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-28T13:30:03.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-28T13:30:03.554 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-28T13:30:03.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-28T13:30:03.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-28T13:30:03.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-28T13:30:03.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-28T13:30:03.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-28T13:30:03.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-28T13:30:03.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-28T13:30:03.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-28T13:30:03.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-28T13:30:03.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-28T13:30:03.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-28T13:30:03.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-28T13:30:03.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-28T13:30:03.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-28T13:30:03.555 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-28T13:30:03.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-28T13:30:03.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-28T13:30:03.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-28T13:30:03.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-28T13:30:03.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-28T13:30:03.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-28T13:30:03.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-28T13:30:03.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-28T13:30:03.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-28T13:30:03.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-28T13:30:03.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-28T13:30:03.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-28T13:30:03.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-28T13:30:03.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-28T13:30:03.556 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-28T13:30:03.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-28T13:30:03.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-28T13:30:03.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-28T13:30:03.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-28T13:30:03.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-28T13:30:03.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-28T13:30:03.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-28T13:30:03.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-28T13:30:03.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-28T13:30:03.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-28T13:30:03.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-28T13:30:03.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-28T13:30:03.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-28T13:30:03.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-28T13:30:03.557 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-28T13:30:03.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-28T13:30:03.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-28T13:30:03.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-28T13:30:03.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-28T13:30:03.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-28T13:30:03.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-28T13:30:03.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-28T13:30:03.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-28T13:30:03.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-28T13:30:03.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-28T13:30:03.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-28T13:30:03.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-28T13:30:03.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-28T13:30:03.558 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-28T13:30:03.559 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-28T13:30:03.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-28T13:30:03.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-28T13:30:03.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-28T13:30:03.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-28T13:30:03.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-28T13:30:03.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-28T13:30:03.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-28T13:30:03.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-28T13:30:03.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-28T13:30:03.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-28T13:30:03.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-28T13:30:03.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-28T13:30:03.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-28T13:30:03.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-28T13:30:03.560 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-28T13:30:03.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-28T13:30:03.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-28T13:30:03.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-28T13:30:03.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-28T13:30:03.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-28T13:30:03.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-28T13:30:03.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-28T13:30:03.561 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-28T13:30:05.964 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T13:30:05.964 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:05.964 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-04-28T13:30:05.964 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.965 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.965 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-28T13:30:05.965 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-28T13:30:05.965 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-04-28T13:30:05.965 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-28T13:30:05.965 INFO:teuthology.orchestra.run.smithi196.stdout: automake-1.16.2-8.el9.noarch 2024-04-28T13:30:05.965 INFO:teuthology.orchestra.run.smithi196.stdout: bison-3.7.4-5.el9.x86_64 2024-04-28T13:30:05.965 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-28T13:30:05.965 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.965 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.965 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.965 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.965 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:30:05.965 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.967 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:30:05.967 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:30:05.967 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.967 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.967 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T13:30:05.967 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-28T13:30:05.967 INFO:teuthology.orchestra.run.smithi196.stdout: flex-2.6.4-9.el9.x86_64 2024-04-28T13:30:05.967 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-28T13:30:05.967 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-28T13:30:05.967 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-28T13:30:05.967 INFO:teuthology.orchestra.run.smithi196.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-28T13:30:05.967 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-28T13:30:05.967 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-28T13:30:05.967 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-28T13:30:05.968 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-28T13:30:05.968 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-28T13:30:05.968 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.968 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.968 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.968 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-28T13:30:05.968 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-28T13:30:05.968 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-28T13:30:05.968 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-28T13:30:05.968 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-28T13:30:05.968 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.968 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.968 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-28T13:30:05.969 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.969 INFO:teuthology.orchestra.run.smithi196.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-28T13:30:05.969 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-28T13:30:05.969 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-28T13:30:05.969 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-28T13:30:05.969 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-28T13:30:05.969 INFO:teuthology.orchestra.run.smithi196.stdout: lua-5.4.4-4.el9.x86_64 2024-04-28T13:30:05.969 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-28T13:30:05.969 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-28T13:30:05.969 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-28T13:30:05.969 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-28T13:30:05.969 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-28T13:30:05.969 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-28T13:30:05.969 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-13.14-1.el9.x86_64 2024-04-28T13:30:05.970 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-28T13:30:05.970 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-28T13:30:05.970 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-28T13:30:05.970 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-28T13:30:05.970 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-28T13:30:05.970 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-28T13:30:05.970 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-28T13:30:05.970 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-28T13:30:05.970 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-28T13:30:05.970 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-28T13:30:05.970 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.970 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.970 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.971 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-28T13:30:05.971 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-28T13:30:05.971 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-28T13:30:05.971 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-28T13:30:05.971 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-28T13:30:05.971 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-28T13:30:05.971 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-28T13:30:05.971 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-28T13:30:05.971 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-28T13:30:05.971 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-28T13:30:05.971 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-28T13:30:05.971 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-28T13:30:05.971 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-28T13:30:05.971 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-28T13:30:05.971 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-28T13:30:05.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-28T13:30:05.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-28T13:30:05.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-28T13:30:05.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-28T13:30:05.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-28T13:30:05.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-28T13:30:05.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-28T13:30:05.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-28T13:30:05.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-28T13:30:05.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-28T13:30:05.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-28T13:30:05.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-28T13:30:05.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-28T13:30:05.972 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-28T13:30:05.973 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-28T13:30:05.973 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-04-28T13:30:05.973 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-28T13:30:05.973 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-28T13:30:05.973 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-28T13:30:05.973 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-28T13:30:05.973 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-28T13:30:05.973 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-28T13:30:05.973 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-28T13:30:05.973 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-28T13:30:05.973 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-28T13:30:05.973 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.973 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.973 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-28T13:30:05.974 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-28T13:30:05.974 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-28T13:30:05.974 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.974 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-28T13:30:05.974 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-28T13:30:05.974 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-28T13:30:05.974 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-28T13:30:05.974 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-28T13:30:05.974 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-28T13:30:05.974 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-28T13:30:05.974 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-28T13:30:05.974 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-28T13:30:05.974 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-28T13:30:05.974 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-28T13:30:05.975 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-28T13:30:05.975 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-28T13:30:05.975 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-28T13:30:05.975 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-28T13:30:05.975 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.975 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.975 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T13:30:05.975 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-3.el9.x86_64 2024-04-28T13:30:05.975 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-28T13:30:05.975 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-28T13:30:05.975 INFO:teuthology.orchestra.run.smithi196.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-28T13:30:05.975 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-28T13:30:05.975 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-28T13:30:05.976 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-28T13:30:05.976 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:05.976 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-04-28T13:30:06.518 DEBUG:teuthology.parallel:result is None 2024-04-28T13:30:06.518 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T13:30:06.664 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-28T13:30:06.720 INFO:teuthology.orchestra.run.smithi196.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-28T13:30:06.721 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-28T13:30:06.721 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-28T13:30:06.723 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T13:30:06.869 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-28T13:30:06.943 INFO:teuthology.orchestra.run.smithi203.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-28T13:30:06.943 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-28T13:30:06.943 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-28T13:30:06.945 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-28T13:30:06.945 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:06.945 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-28T13:30:07.015 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:07.015 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-28T13:30:07.105 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-28T13:30:07.105 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:07.105 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-04-28T13:30:07.171 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-28T13:30:07.280 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:07.280 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2024-04-28T13:30:07.370 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-28T13:30:07.453 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-28T13:30:07.454 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:07.454 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-28T13:30:07.518 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-28T13:30:07.622 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:07.622 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-28T13:30:07.711 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-28T13:30:07.840 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-28T13:30:07.841 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:07.841 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-04-28T13:30:07.903 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-28T13:30:08.008 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:08.008 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2024-04-28T13:30:08.096 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-28T13:30:08.182 INFO:teuthology.run_tasks:Running task ceph... 2024-04-28T13:30:08.267 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-28T13:30:08.267 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /var/log/ceph 2024-04-28T13:30:08.273 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /var/log/ceph 2024-04-28T13:30:08.357 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-28T13:30:08.358 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-28T13:30:08.362 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-28T13:30:08.445 INFO:tasks.ceph:Creating extra log directories... 2024-04-28T13:30:08.445 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-28T13:30:08.450 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-28T13:30:08.546 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-28T13:30:08.547 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-28T13:30:08.547 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-28T13:30:08.547 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:08.547 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-28T13:30:08.589 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-28T13:30:08.789 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-28T13:30:08.854 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:08.854 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-28T13:30:08.912 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-28T13:30:09.142 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-28T13:30:09.234 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-28T13:30:09.235 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-28T13:30:09.235 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/quiesce}', 'email': None, 'first_in_suite': False, 'job_id': '7676911', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow request'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi196.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI7L8TE5So2DQv96t6FnSiAvpZaOROivbIf2fyRlxXKTgmq3DTpY0WPzfNamw+B3wfidZAzU0uOrvF6H1I2X2g0=', 'smithi203.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHYTvuYQNVm4jN95gDka/WmqvB4XshGdGOq29jZP/IqzYmL0Ee4E/59D4XGQeznrW/74dP6qqCG/Y7eMwSIaH5w='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow request'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_quiesce']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-28_11:41:23', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-28T13:30:09.235 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-28T13:30:09.240 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-28T13:30:09.295 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/run/ceph 2024-04-28T13:30:09.325 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/run/ceph 2024-04-28T13:30:09.413 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:09.413 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-04-28T13:30:09.455 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-28T13:30:09.455 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-04-28T13:30:09.536 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-28T13:30:09.536 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T13:30:09.536 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-28T13:30:09.536 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T13:30:09.536 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T13:30:09.536 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-28 13:29:58.387957534 +0000 2024-04-28T13:30:09.537 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-28 13:27:15.319892867 +0000 2024-04-28T13:30:09.537 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-28 13:27:15.319892867 +0000 2024-04-28T13:30:09.537 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-28 13:27:15.319892867 +0000 2024-04-28T13:30:09.537 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-28T13:30:09.635 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-28T13:30:09.636 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-28T13:30:09.636 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000389537 s, 1.3 MB/s 2024-04-28T13:30:09.638 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-28T13:30:09.679 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-04-28T13:30:09.759 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-28T13:30:09.759 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T13:30:09.759 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-28T13:30:09.759 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T13:30:09.759 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T13:30:09.759 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-28 13:29:58.387957534 +0000 2024-04-28T13:30:09.760 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-28 13:27:15.950879438 +0000 2024-04-28T13:30:09.760 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-28 13:27:15.950879438 +0000 2024-04-28T13:30:09.760 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-28 13:27:15.950879438 +0000 2024-04-28T13:30:09.760 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-28T13:30:09.857 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-28T13:30:09.857 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-28T13:30:09.858 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000286055 s, 1.8 MB/s 2024-04-28T13:30:09.860 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-28T13:30:09.899 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-04-28T13:30:09.980 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-28T13:30:09.980 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T13:30:09.980 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-28T13:30:09.980 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T13:30:09.980 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T13:30:09.980 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-28 13:29:58.388957511 +0000 2024-04-28T13:30:09.980 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-28 13:27:16.526867180 +0000 2024-04-28T13:30:09.980 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-28 13:27:16.526867180 +0000 2024-04-28T13:30:09.980 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-28 13:27:16.526867180 +0000 2024-04-28T13:30:09.981 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-28T13:30:10.081 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-28T13:30:10.082 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-28T13:30:10.082 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000272994 s, 1.9 MB/s 2024-04-28T13:30:10.084 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-28T13:30:10.125 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-04-28T13:30:10.206 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-28T13:30:10.206 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T13:30:10.206 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-28T13:30:10.206 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T13:30:10.207 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T13:30:10.207 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-28 13:29:58.389957490 +0000 2024-04-28T13:30:10.207 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-28 13:27:17.139854135 +0000 2024-04-28T13:30:10.207 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-28 13:27:17.139854135 +0000 2024-04-28T13:30:10.207 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-28 13:27:17.139854135 +0000 2024-04-28T13:30:10.207 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-28T13:30:10.312 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-28T13:30:10.313 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-28T13:30:10.313 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000297532 s, 1.7 MB/s 2024-04-28T13:30:10.315 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-28T13:30:10.357 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-28T13:30:10.357 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:10.357 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2024-04-28T13:30:10.415 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-28T13:30:10.415 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2024-04-28T13:30:10.465 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-28T13:30:10.465 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T13:30:10.466 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-28T13:30:10.466 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T13:30:10.466 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T13:30:10.466 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-28 13:29:26.131298042 +0000 2024-04-28T13:30:10.466 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-28 13:27:15.334546196 +0000 2024-04-28T13:30:10.466 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-28 13:27:15.334546196 +0000 2024-04-28T13:30:10.466 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-28 13:27:15.334546196 +0000 2024-04-28T13:30:10.466 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-28T13:30:10.545 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-28T13:30:10.545 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-28T13:30:10.545 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000345902 s, 1.5 MB/s 2024-04-28T13:30:10.549 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-28T13:30:10.604 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2024-04-28T13:30:10.655 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-28T13:30:10.655 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T13:30:10.655 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-28T13:30:10.655 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T13:30:10.655 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T13:30:10.655 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-28 13:29:26.133297995 +0000 2024-04-28T13:30:10.655 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-28 13:27:15.932532166 +0000 2024-04-28T13:30:10.655 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-28 13:27:15.932532166 +0000 2024-04-28T13:30:10.655 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-28 13:27:15.932532166 +0000 2024-04-28T13:30:10.656 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-28T13:30:10.731 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-28T13:30:10.731 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-28T13:30:10.731 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000300694 s, 1.7 MB/s 2024-04-28T13:30:10.735 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-28T13:30:10.787 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2024-04-28T13:30:10.835 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-28T13:30:10.835 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T13:30:10.835 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-28T13:30:10.835 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T13:30:10.836 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T13:30:10.836 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-28 13:29:26.133297995 +0000 2024-04-28T13:30:10.836 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-28 13:27:16.512518558 +0000 2024-04-28T13:30:10.836 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-28 13:27:16.512518558 +0000 2024-04-28T13:30:10.836 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-28 13:27:16.512518558 +0000 2024-04-28T13:30:10.836 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-28T13:30:10.908 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-28T13:30:10.908 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-28T13:30:10.908 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.00029494 s, 1.7 MB/s 2024-04-28T13:30:10.911 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-28T13:30:10.964 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2024-04-28T13:30:11.011 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-28T13:30:11.012 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T13:30:11.012 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-28T13:30:11.012 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T13:30:11.012 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T13:30:11.012 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-28 13:29:26.135297947 +0000 2024-04-28T13:30:11.012 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-28 13:27:17.126504153 +0000 2024-04-28T13:30:11.012 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-28 13:27:17.126504153 +0000 2024-04-28T13:30:11.012 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-28 13:27:17.126504153 +0000 2024-04-28T13:30:11.012 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-28T13:30:11.086 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-28T13:30:11.086 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-28T13:30:11.086 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000373647 s, 1.4 MB/s 2024-04-28T13:30:11.090 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-28T13:30:11.142 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-28T13:30:11.142 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi196.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-28T13:30:11.142 INFO:tasks.ceph:Generating config... 2024-04-28T13:30:11.143 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-28T13:30:11.143 INFO:tasks.ceph:[client] debug client = 20 2024-04-28T13:30:11.143 INFO:tasks.ceph:[client] debug ms = 1 2024-04-28T13:30:11.144 INFO:tasks.ceph:[client] pid file = 2024-04-28T13:30:11.144 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-28T13:30:11.144 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-28T13:30:11.144 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-28T13:30:11.144 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-28T13:30:11.144 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-28T13:30:11.144 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-28T13:30:11.144 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-28T13:30:11.144 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-28T13:30:11.144 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-28T13:30:11.144 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-28T13:30:11.144 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-28T13:30:11.144 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-28T13:30:11.144 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-28T13:30:11.144 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-28T13:30:11.145 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-28T13:30:11.145 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-28T13:30:11.145 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-28T13:30:11.145 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-28T13:30:11.145 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-28T13:30:11.145 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-28T13:30:11.145 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-28T13:30:11.145 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-28T13:30:11.145 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-28T13:30:11.145 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-28T13:30:11.145 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-28T13:30:11.145 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-28T13:30:11.145 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-28T13:30:11.145 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-28T13:30:11.145 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-28T13:30:11.145 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-28T13:30:11.146 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-28T13:30:11.146 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-28T13:30:11.146 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-28T13:30:11.146 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-28T13:30:11.146 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-28T13:30:11.146 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-28T13:30:11.146 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-28T13:30:11.146 INFO:tasks.ceph:Setting up mon.a... 2024-04-28T13:30:11.146 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-28T13:30:11.249 INFO:teuthology.orchestra.run.smithi196.stdout:creating /etc/ceph/ceph.keyring 2024-04-28T13:30:11.256 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-28T13:30:11.367 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-28T13:30:11.431 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.196'), ('mon.b', '172.21.15.203'), ('mon.c', '[v2:172.21.15.203:3301,v1:172.21.15.203:6790]')] 2024-04-28T13:30:11.431 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.196,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-28T13:30:11.432 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:11.432 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-28T13:30:11.472 DEBUG:teuthology.orchestra.run.smithi196:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.196 --add b 172.21.15.203 --addv c '[v2:172.21.15.203:3301,v1:172.21.15.203:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-28T13:30:11.594 INFO:teuthology.orchestra.run.smithi196.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-28T13:30:11.594 INFO:teuthology.orchestra.run.smithi196.stdout:monmaptool: generated fsid 270aea32-2e16-4a97-85f7-aeae39bc8c76 2024-04-28T13:30:11.594 INFO:teuthology.orchestra.run.smithi196.stdout:setting min_mon_release = quincy 2024-04-28T13:30:11.594 INFO:teuthology.orchestra.run.smithi196.stdout:epoch 0 2024-04-28T13:30:11.595 INFO:teuthology.orchestra.run.smithi196.stdout:fsid 270aea32-2e16-4a97-85f7-aeae39bc8c76 2024-04-28T13:30:11.595 INFO:teuthology.orchestra.run.smithi196.stdout:last_changed 2024-04-28T13:30:11.594991+0000 2024-04-28T13:30:11.595 INFO:teuthology.orchestra.run.smithi196.stdout:created 2024-04-28T13:30:11.594991+0000 2024-04-28T13:30:11.595 INFO:teuthology.orchestra.run.smithi196.stdout:min_mon_release 17 (quincy) 2024-04-28T13:30:11.595 INFO:teuthology.orchestra.run.smithi196.stdout:election_strategy: 1 2024-04-28T13:30:11.595 INFO:teuthology.orchestra.run.smithi196.stdout:0: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.a 2024-04-28T13:30:11.595 INFO:teuthology.orchestra.run.smithi196.stdout:1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.b 2024-04-28T13:30:11.595 INFO:teuthology.orchestra.run.smithi196.stdout:2: [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] mon.c 2024-04-28T13:30:11.595 INFO:teuthology.orchestra.run.smithi196.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-28T13:30:11.597 DEBUG:teuthology.orchestra.run.smithi196:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-28T13:30:11.635 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 270aea32-2e16-4a97-85f7-aeae39bc8c76... 2024-04-28T13:30:11.636 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-28T13:30:11.682 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-28T13:30:11.796 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2024-04-28T13:30:11.796 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2024-04-28T13:30:11.796 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-28T13:30:11.796 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2024-04-28T13:30:11.797 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.797 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2024-04-28T13:30:11.797 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.797 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2024-04-28T13:30:11.797 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.797 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2024-04-28T13:30:11.797 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2024-04-28T13:30:11.797 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.797 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2024-04-28T13:30:11.797 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2024-04-28T13:30:11.797 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.797 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-28T13:30:11.797 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2024-04-28T13:30:11.798 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.798 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2024-04-28T13:30:11.798 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-28T13:30:11.798 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.798 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2024-04-28T13:30:11.798 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.798 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2024-04-28T13:30:11.798 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2024-04-28T13:30:11.798 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2024-04-28T13:30:11.798 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2024-04-28T13:30:11.798 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2024-04-28T13:30:11.798 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2024-04-28T13:30:11.798 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2024-04-28T13:30:11.798 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-28T13:30:11.798 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-28T13:30:11.799 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2024-04-28T13:30:11.799 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.799 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-28T13:30:11.799 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.799 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2024-04-28T13:30:11.799 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.799 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2024-04-28T13:30:11.799 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.799 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2024-04-28T13:30:11.799 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2024-04-28T13:30:11.799 INFO:teuthology.orchestra.run.smithi196.stdout: mon health detail to clog = false 2024-04-28T13:30:11.799 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.196,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-28T13:30:11.799 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on skipped message = False 2024-04-28T13:30:11.799 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = 270aea32-2e16-4a97-85f7-aeae39bc8c76 2024-04-28T13:30:11.799 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.800 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2024-04-28T13:30:11.800 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2024-04-28T13:30:11.800 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.800 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2024-04-28T13:30:11.800 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2024-04-28T13:30:11.800 INFO:teuthology.orchestra.run.smithi196.stdout: osd mclock profile = high_recovery_ops 2024-04-28T13:30:11.800 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.800 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2024-04-28T13:30:11.800 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2024-04-28T13:30:11.800 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.800 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2024-04-28T13:30:11.800 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2024-04-28T13:30:11.800 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2024-04-28T13:30:11.800 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug trim objects = true 2024-04-28T13:30:11.801 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.801 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2024-04-28T13:30:11.801 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2024-04-28T13:30:11.801 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.801 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2024-04-28T13:30:11.801 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.801 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2024-04-28T13:30:11.801 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.801 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2024-04-28T13:30:11.801 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.801 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2024-04-28T13:30:11.801 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2024-04-28T13:30:11.801 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.801 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2024-04-28T13:30:11.801 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2024-04-28T13:30:11.802 INFO:teuthology.orchestra.run.smithi196.stdout: bdev async discard = True 2024-04-28T13:30:11.802 INFO:teuthology.orchestra.run.smithi196.stdout: bdev enable discard = True 2024-04-28T13:30:11.802 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore allocator = bitmap 2024-04-28T13:30:11.802 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2024-04-28T13:30:11.802 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2024-04-28T13:30:11.802 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 1/20 2024-04-28T13:30:11.802 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 1/20 2024-04-28T13:30:11.802 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-28T13:30:11.802 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 20 2024-04-28T13:30:11.802 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 4/10 2024-04-28T13:30:11.802 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2024-04-28T13:30:11.802 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2024-04-28T13:30:11.802 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2024-04-28T13:30:11.802 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 0.95 2024-04-28T13:30:11.802 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = bluestore 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-28T13:30:11.803 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2024-04-28T13:30:11.804 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-28T13:30:11.804 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2024-04-28T13:30:11.804 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2024-04-28T13:30:11.804 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-28T13:30:11.804 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2024-04-28T13:30:11.804 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2024-04-28T13:30:11.804 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-28T13:30:11.804 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.804 INFO:teuthology.orchestra.run.smithi196.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-28T13:30:11.804 INFO:teuthology.orchestra.run.smithi196.stdout: auth mon ticket ttl = 660 # 11m 2024-04-28T13:30:11.804 INFO:teuthology.orchestra.run.smithi196.stdout: auth service ticket ttl = 240 # 4m 2024-04-28T13:30:11.804 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.804 INFO:teuthology.orchestra.run.smithi196.stdout: # don't complain about insecure global_id in the test suite 2024-04-28T13:30:11.804 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-28T13:30:11.805 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-28T13:30:11.805 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.805 INFO:teuthology.orchestra.run.smithi196.stdout: # 1m isn't quite enough 2024-04-28T13:30:11.805 INFO:teuthology.orchestra.run.smithi196.stdout: mon_down_mkfs_grace = 2m 2024-04-28T13:30:11.805 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.805 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_filestore_osds = false 2024-04-28T13:30:11.805 INFO:teuthology.orchestra.run.smithi196.stdout: mon down mkfs grace = 300 2024-04-28T13:30:11.805 INFO:teuthology.orchestra.run.smithi196.stdout: mon op complaint time = 120 2024-04-28T13:30:11.805 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:30:11.805 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2024-04-28T13:30:11.805 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2024-04-28T13:30:11.805 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2024-04-28T13:30:11.805 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2024-04-28T13:30:11.805 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-28T13:30:11.806 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-28T13:30:11.806 INFO:teuthology.orchestra.run.smithi196.stdout: client mount timeout = 600 2024-04-28T13:30:11.806 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-04-28T13:30:11.806 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-28T13:30:11.806 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = "" 2024-04-28T13:30:11.806 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-04-28T13:30:11.806 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-04-28T13:30:11.806 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2024-04-28T13:30:11.806 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2024-04-28T13:30:11.806 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2024-04-28T13:30:11.806 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-04-28T13:30:11.806 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-04-28T13:30:11.806 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-04-28T13:30:11.806 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-04-28T13:30:11.806 INFO:teuthology.orchestra.run.smithi196.stdout:[mds] 2024-04-28T13:30:11.807 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds = 20 2024-04-28T13:30:11.807 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds balancer = 20 2024-04-28T13:30:11.807 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-28T13:30:11.807 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug frag = True 2024-04-28T13:30:11.807 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug scatterstat = True 2024-04-28T13:30:11.807 INFO:teuthology.orchestra.run.smithi196.stdout: mds op complaint time = 180 2024-04-28T13:30:11.807 INFO:teuthology.orchestra.run.smithi196.stdout: mds verify scatter = True 2024-04-28T13:30:11.807 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-04-28T13:30:11.807 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-04-28T13:30:11.807 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-04-28T13:30:11.826 INFO:teuthology.orchestra.run.smithi203.stdout:[global] 2024-04-28T13:30:11.826 INFO:teuthology.orchestra.run.smithi203.stdout: chdir = "" 2024-04-28T13:30:11.826 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-28T13:30:11.826 INFO:teuthology.orchestra.run.smithi203.stdout: auth supported = cephx 2024-04-28T13:30:11.826 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.826 INFO:teuthology.orchestra.run.smithi203.stdout: filestore xattr use omap = true 2024-04-28T13:30:11.826 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.826 INFO:teuthology.orchestra.run.smithi203.stdout: mon clock drift allowed = 1.000 2024-04-28T13:30:11.826 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: osd crush chooseleaf type = 0 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: auth debug = true 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on old message = true 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on bug = true 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: mon pg warn max object skew = 0 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: # disable pg_autoscaler by default for new pools 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default size = 2 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow primary affinity = true 2024-04-28T13:30:11.827 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow pg remap = true 2024-04-28T13:30:11.828 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on legacy crush tunables = false 2024-04-28T13:30:11.828 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on crush straw calc version zero = false 2024-04-28T13:30:11.828 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on no sortbitwise = false 2024-04-28T13:30:11.828 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on osd down out interval zero = false 2024-04-28T13:30:11.828 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on too few osds = false 2024-04-28T13:30:11.828 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-28T13:30:11.828 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-28T13:30:11.828 INFO:teuthology.orchestra.run.smithi203.stdout: mon_allow_pool_size_one = true 2024-04-28T13:30:11.828 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.828 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-28T13:30:11.828 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.828 INFO:teuthology.orchestra.run.smithi203.stdout: osd default data pool replay window = 5 2024-04-28T13:30:11.828 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.828 INFO:teuthology.orchestra.run.smithi203.stdout: mon allow pool delete = true 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout: mon cluster log file level = debug 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout: debug asserts on shutdown = true 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout: mon health detail to clog = false 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout: mon host = "172.21.15.196,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on skipped message = False 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout: fsid = 270aea32-2e16-4a97-85f7-aeae39bc8c76 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout:[osd] 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout: osd journal size = 100 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub load threshold = 5.0 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub max interval = 600 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout: osd mclock profile = high_recovery_ops 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.829 INFO:teuthology.orchestra.run.smithi203.stdout: osd recover clone overlap = true 2024-04-28T13:30:11.830 INFO:teuthology.orchestra.run.smithi203.stdout: osd recovery max chunk = 1048576 2024-04-28T13:30:11.830 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.830 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug shutdown = true 2024-04-28T13:30:11.830 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug op order = true 2024-04-28T13:30:11.830 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug verify stray on activate = true 2024-04-28T13:30:11.830 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug trim objects = true 2024-04-28T13:30:11.830 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.830 INFO:teuthology.orchestra.run.smithi203.stdout: osd open classes on start = true 2024-04-28T13:30:11.830 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug pg log writeout = true 2024-04-28T13:30:11.830 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.830 INFO:teuthology.orchestra.run.smithi203.stdout: osd deep scrub update digest min age = 30 2024-04-28T13:30:11.830 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.830 INFO:teuthology.orchestra.run.smithi203.stdout: osd map max advance = 10 2024-04-28T13:30:11.830 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.831 INFO:teuthology.orchestra.run.smithi203.stdout: journal zero on create = true 2024-04-28T13:30:11.831 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.831 INFO:teuthology.orchestra.run.smithi203.stdout: filestore ondisk finisher threads = 3 2024-04-28T13:30:11.831 INFO:teuthology.orchestra.run.smithi203.stdout: filestore apply finisher threads = 3 2024-04-28T13:30:11.831 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.831 INFO:teuthology.orchestra.run.smithi203.stdout: bdev debug aio = true 2024-04-28T13:30:11.831 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug misdirected ops = true 2024-04-28T13:30:11.831 INFO:teuthology.orchestra.run.smithi203.stdout: bdev async discard = True 2024-04-28T13:30:11.831 INFO:teuthology.orchestra.run.smithi203.stdout: bdev enable discard = True 2024-04-28T13:30:11.831 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore allocator = bitmap 2024-04-28T13:30:11.831 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore block size = 96636764160 2024-04-28T13:30:11.831 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore fsck on mount = True 2024-04-28T13:30:11.831 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluefs = 1/20 2024-04-28T13:30:11.831 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluestore = 1/20 2024-04-28T13:30:11.832 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-28T13:30:11.832 INFO:teuthology.orchestra.run.smithi203.stdout: debug osd = 20 2024-04-28T13:30:11.832 INFO:teuthology.orchestra.run.smithi203.stdout: debug rocksdb = 4/10 2024-04-28T13:30:11.832 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd backfillfull_ratio = 0.85 2024-04-28T13:30:11.832 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd full ratio = 0.9 2024-04-28T13:30:11.832 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd nearfull ratio = 0.8 2024-04-28T13:30:11.832 INFO:teuthology.orchestra.run.smithi203.stdout: osd failsafe full ratio = 0.95 2024-04-28T13:30:11.832 INFO:teuthology.orchestra.run.smithi203.stdout: osd objectstore = bluestore 2024-04-28T13:30:11.832 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-28T13:30:11.832 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.832 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr] 2024-04-28T13:30:11.832 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-28T13:30:11.832 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgr = 20 2024-04-28T13:30:11.832 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-28T13:30:11.832 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-28T13:30:11.833 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-28T13:30:11.833 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-28T13:30:11.833 INFO:teuthology.orchestra.run.smithi203.stdout: mgr/telemetry/nag = false 2024-04-28T13:30:11.833 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-04-28T13:30:11.833 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.833 INFO:teuthology.orchestra.run.smithi203.stdout:[mon] 2024-04-28T13:30:11.833 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-28T13:30:11.833 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-28T13:30:11.833 INFO:teuthology.orchestra.run.smithi203.stdout: debug paxos = 20 2024-04-28T13:30:11.833 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-28T13:30:11.833 INFO:teuthology.orchestra.run.smithi203.stdout: mon data avail warn = 5 2024-04-28T13:30:11.833 INFO:teuthology.orchestra.run.smithi203.stdout: mon mgr mkfs grace = 240 2024-04-28T13:30:11.833 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-28T13:30:11.833 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd reporter subtree level = osd 2024-04-28T13:30:11.833 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd prime pg temp = true 2024-04-28T13:30:11.834 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-28T13:30:11.834 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.834 INFO:teuthology.orchestra.run.smithi203.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-28T13:30:11.834 INFO:teuthology.orchestra.run.smithi203.stdout: auth mon ticket ttl = 660 # 11m 2024-04-28T13:30:11.834 INFO:teuthology.orchestra.run.smithi203.stdout: auth service ticket ttl = 240 # 4m 2024-04-28T13:30:11.834 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.834 INFO:teuthology.orchestra.run.smithi203.stdout: # don't complain about insecure global_id in the test suite 2024-04-28T13:30:11.834 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-28T13:30:11.834 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-28T13:30:11.834 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.834 INFO:teuthology.orchestra.run.smithi203.stdout: # 1m isn't quite enough 2024-04-28T13:30:11.834 INFO:teuthology.orchestra.run.smithi203.stdout: mon_down_mkfs_grace = 2m 2024-04-28T13:30:11.834 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.834 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_filestore_osds = false 2024-04-28T13:30:11.834 INFO:teuthology.orchestra.run.smithi203.stdout: mon down mkfs grace = 300 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout: mon op complaint time = 120 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout:[client] 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout: rgw cache enabled = true 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable ops log = true 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable usage log = true 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout: client mount timeout = 600 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = "" 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.a] 2024-04-28T13:30:11.835 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.b] 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.c] 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout:[mds] 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds = 20 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds balancer = 20 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug frag = True 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug scatterstat = True 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout: mds op complaint time = 180 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout: mds verify scatter = True 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-28T13:30:11.836 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-28T13:30:11.858 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-28T13:30:11.858 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-28T13:30:11.974 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-28T13:30:11.974 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:11.974 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-28T13:30:12.011 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:12.011 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-28T13:30:12.093 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:30:12.093 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:12.093 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-28T13:30:12.093 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-28T13:30:12.234 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:12.234 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-28T13:30:12.314 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:30:12.315 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:12.315 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-28T13:30:12.315 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-28T13:30:12.438 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:12.439 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-28T13:30:12.536 INFO:tasks.ceph:Setting up mon nodes... 2024-04-28T13:30:12.536 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-28T13:30:12.536 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-28T13:30:12.655 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-28T13:30:12.662 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-28T13:30:12.841 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-28T13:30:12.853 INFO:tasks.ceph:Setting up mds nodes... 2024-04-28T13:30:12.853 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-28T13:30:12.975 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-28T13:30:12.981 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-28T13:30:13.046 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-28T13:30:13.157 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-28T13:30:13.163 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-28T13:30:13.222 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-28T13:30:13.373 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-28T13:30:13.380 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-28T13:30:13.464 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-28T13:30:13.607 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-28T13:30:13.615 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-28T13:30:13.695 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-28T13:30:13.696 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-28T13:30:13.807 INFO:teuthology.orchestra.run.smithi196.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-28T13:30:13.852 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-28T13:30:13.965 INFO:teuthology.orchestra.run.smithi196.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-28T13:30:13.998 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-28T13:30:14.109 INFO:teuthology.orchestra.run.smithi203.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-28T13:30:14.153 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-28T13:30:14.263 INFO:teuthology.orchestra.run.smithi203.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-28T13:30:14.313 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-28T13:30:14.313 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi196.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-28T13:30:14.313 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-28T13:30:14.390 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-28T13:30:14.390 INFO:tasks.ceph:role: osd.0 2024-04-28T13:30:14.390 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:30:14.390 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-28T13:30:14.464 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T13:30:14.465 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T13:30:14.465 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T13:30:14.465 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T13:30:14.465 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T13:30:14.465 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-04-28T13:30:14.465 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T13:30:14.465 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T13:30:14.465 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T13:30:14.466 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T13:30:14.872 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-04-28T13:30:14.879 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-04-28T13:30:14.879 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-28T13:30:15.050 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-28T13:30:15.138 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-28T13:30:15.208 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-28T13:30:15.209 INFO:tasks.ceph:role: osd.1 2024-04-28T13:30:15.209 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:30:15.209 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-28T13:30:15.284 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T13:30:15.284 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T13:30:15.284 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T13:30:15.284 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T13:30:15.284 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T13:30:15.284 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-04-28T13:30:15.284 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T13:30:15.285 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T13:30:15.285 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T13:30:15.285 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T13:30:15.640 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-04-28T13:30:15.645 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-04-28T13:30:15.645 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-28T13:30:15.753 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-28T13:30:15.847 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-28T13:30:15.920 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-28T13:30:15.920 INFO:tasks.ceph:role: osd.2 2024-04-28T13:30:15.920 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:30:15.920 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-28T13:30:16.006 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T13:30:16.007 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T13:30:16.007 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T13:30:16.007 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T13:30:16.007 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T13:30:16.007 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-04-28T13:30:16.007 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T13:30:16.007 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T13:30:16.007 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T13:30:16.007 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T13:30:16.396 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-04-28T13:30:16.402 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-04-28T13:30:16.402 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-28T13:30:16.572 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-28T13:30:16.666 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-28T13:30:16.751 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-28T13:30:16.751 INFO:tasks.ceph:role: osd.3 2024-04-28T13:30:16.751 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:30:16.751 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-28T13:30:16.838 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T13:30:16.838 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T13:30:16.838 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T13:30:16.839 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T13:30:16.839 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T13:30:16.839 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-04-28T13:30:16.839 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T13:30:16.839 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T13:30:16.839 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T13:30:16.839 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T13:30:17.249 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-04-28T13:30:17.255 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-04-28T13:30:17.255 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-28T13:30:17.409 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-28T13:30:17.499 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T13:30:17.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:17.640+0000 7fbcbe533640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-28T13:30:17.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:17.641+0000 7fbcbe533640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-28T13:30:17.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:17.641+0000 7fbcbe533640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-28T13:30:17.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:17.641+0000 7fbcbe533640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-28T13:30:17.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:17.641+0000 7fbcbe533640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-28T13:30:17.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:17.645+0000 7fbcbe533640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-28T13:30:20.888 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-28T13:30:20.970 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T13:30:21.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:21.101+0000 7fc0a5418640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-28T13:30:21.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:21.102+0000 7fc0a5418640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-28T13:30:21.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:21.102+0000 7fc0a5418640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-28T13:30:21.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:21.102+0000 7fc0a5418640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-28T13:30:21.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:21.102+0000 7fc0a5418640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-28T13:30:21.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:21.104+0000 7fc0a5418640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-28T13:30:24.056 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-28T13:30:24.142 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T13:30:24.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:24.268+0000 7f33f2f21640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-28T13:30:24.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:24.269+0000 7f33f2f21640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-28T13:30:24.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:24.269+0000 7f33f2f21640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-28T13:30:24.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:24.269+0000 7f33f2f21640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-28T13:30:24.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:24.269+0000 7f33f2f21640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-28T13:30:24.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:24.271+0000 7f33f2f21640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-28T13:30:27.481 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-28T13:30:27.567 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T13:30:27.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:27.692+0000 7ffbeb61e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-28T13:30:27.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:27.692+0000 7ffbeb61e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-28T13:30:27.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:27.692+0000 7ffbeb61e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-28T13:30:27.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:27.692+0000 7ffbeb61e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-28T13:30:27.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:27.692+0000 7ffbeb61e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-28T13:30:27.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:27.695+0000 7ffbeb61e640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-28T13:30:30.658 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-28T13:30:30.746 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-28T13:30:30.837 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-28T13:30:30.837 INFO:tasks.ceph:role: osd.4 2024-04-28T13:30:30.837 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:30:30.838 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-28T13:30:30.930 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T13:30:30.931 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T13:30:30.931 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T13:30:30.931 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T13:30:30.931 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T13:30:30.931 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-28T13:30:30.931 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T13:30:30.931 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T13:30:30.931 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T13:30:30.931 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T13:30:31.345 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-28T13:30:31.351 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-28T13:30:31.351 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-28T13:30:31.495 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-28T13:30:31.575 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-28T13:30:31.649 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-28T13:30:31.649 INFO:tasks.ceph:role: osd.5 2024-04-28T13:30:31.649 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:30:31.649 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-28T13:30:31.734 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T13:30:31.734 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T13:30:31.734 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T13:30:31.734 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T13:30:31.735 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T13:30:31.735 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-28T13:30:31.735 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T13:30:31.735 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T13:30:31.735 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T13:30:31.735 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T13:30:32.092 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-28T13:30:32.098 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-28T13:30:32.098 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-28T13:30:32.206 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-28T13:30:32.292 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-28T13:30:32.369 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-28T13:30:32.369 INFO:tasks.ceph:role: osd.6 2024-04-28T13:30:32.369 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:30:32.370 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-28T13:30:32.456 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T13:30:32.456 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T13:30:32.456 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T13:30:32.456 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T13:30:32.457 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T13:30:32.457 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-28T13:30:32.457 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T13:30:32.457 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T13:30:32.457 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T13:30:32.457 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T13:30:32.842 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-28T13:30:32.848 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-28T13:30:32.848 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-28T13:30:32.968 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-28T13:30:33.058 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-28T13:30:33.139 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-28T13:30:33.139 INFO:tasks.ceph:role: osd.7 2024-04-28T13:30:33.140 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:30:33.140 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-28T13:30:33.228 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T13:30:33.229 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T13:30:33.229 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T13:30:33.229 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T13:30:33.229 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T13:30:33.229 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-28T13:30:33.229 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T13:30:33.229 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T13:30:33.229 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T13:30:33.229 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T13:30:33.636 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-28T13:30:33.644 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-28T13:30:33.645 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-28T13:30:33.806 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-28T13:30:33.903 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T13:30:34.037 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:34.036+0000 7fdda83c0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-28T13:30:34.038 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:34.037+0000 7fdda83c0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-28T13:30:34.038 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:34.037+0000 7fdda83c0640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-28T13:30:34.039 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:34.037+0000 7fdda83c0640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-28T13:30:34.039 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:34.037+0000 7fdda83c0640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-28T13:30:34.041 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:34.041+0000 7fdda83c0640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-28T13:30:36.997 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-28T13:30:37.073 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T13:30:37.185 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:37.184+0000 7f7fdb42c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-28T13:30:37.185 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:37.185+0000 7f7fdb42c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-28T13:30:37.185 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:37.185+0000 7f7fdb42c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-28T13:30:37.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:37.185+0000 7f7fdb42c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-28T13:30:37.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:37.185+0000 7f7fdb42c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-28T13:30:37.187 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:37.187+0000 7f7fdb42c640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-28T13:30:39.236 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:30:39.244 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:30:40.163 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-28T13:30:40.245 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T13:30:40.372 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:40.371+0000 7f15fdcb5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-28T13:30:40.373 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:40.372+0000 7f15fdcb5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-28T13:30:40.373 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:40.372+0000 7f15fdcb5640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-28T13:30:40.373 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:40.372+0000 7f15fdcb5640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-28T13:30:40.373 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:40.372+0000 7f15fdcb5640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-28T13:30:40.375 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:40.374+0000 7f15fdcb5640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-28T13:30:43.352 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-28T13:30:43.441 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T13:30:43.566 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:43.566+0000 7f261962e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-28T13:30:43.567 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:43.566+0000 7f261962e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-28T13:30:43.567 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:43.566+0000 7f261962e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-28T13:30:43.567 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:43.567+0000 7f261962e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-28T13:30:43.567 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:43.567+0000 7f261962e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-28T13:30:43.569 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:43.569+0000 7f261962e640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-28T13:30:46.762 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-28T13:30:46.850 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-28T13:30:46.850 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:46.850 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-28T13:30:46.942 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:46.942 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-28T13:30:47.019 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:47.019 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-28T13:30:47.096 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:47.096 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-28T13:30:47.223 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:47.223 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-28T13:30:47.309 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:47.309 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-28T13:30:47.381 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:47.381 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-28T13:30:47.494 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:47.494 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-28T13:30:47.581 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:47.581 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-28T13:30:47.665 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:47.665 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-28T13:30:47.740 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:47.740 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-28T13:30:47.821 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:47.822 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-28T13:30:47.907 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:47.907 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-28T13:30:48.027 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:48.027 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-28T13:30:48.109 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:48.109 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-28T13:30:48.161 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:48.161 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-28T13:30:48.207 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:48.207 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-28T13:30:48.264 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:48.264 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-28T13:30:48.315 INFO:tasks.ceph:Adding keys to all mons... 2024-04-28T13:30:48.315 DEBUG:teuthology.orchestra.run.smithi196:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-28T13:30:48.322 DEBUG:teuthology.orchestra.run.smithi203:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-28T13:30:48.391 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr.y] 2024-04-28T13:30:48.391 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBkTy5mh2AXJxAA+BPc+Xi43waIQJ2YjGmw9Q== 2024-04-28T13:30:48.391 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-04-28T13:30:48.391 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBkTy5mGDY2OhAAO3Cl3Q176Jk7wky/cM1g7A== 2024-04-28T13:30:48.391 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-04-28T13:30:48.391 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBlTy5mJqNwCRAA9EAW936ooX1fIhI8D5Y71w== 2024-04-28T13:30:48.391 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.0] 2024-04-28T13:30:48.391 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBpTy5mOG48JhAAMO70X2PHMWLcrTFLBuDe1Q== 2024-04-28T13:30:48.391 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.1] 2024-04-28T13:30:48.391 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBtTy5mADYZBhAARTLL+BqFjPyzcEKny+CBYg== 2024-04-28T13:30:48.391 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.2] 2024-04-28T13:30:48.391 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBwTy5m26gJEBAAuBn3BMPn/EZzOLzMeaaX2w== 2024-04-28T13:30:48.392 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.3] 2024-04-28T13:30:48.392 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBzTy5mrx1KKRAAUG56DD6uIyHP7oD0wOpRkQ== 2024-04-28T13:30:48.392 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr.x] 2024-04-28T13:30:48.392 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBkTy5mV7E7MhAAtess+f/dkv11zc9Ujl2jqg== 2024-04-28T13:30:48.392 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-04-28T13:30:48.392 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBlTy5m5wRGFhAAzFebm/R1yHYctJDIlWndyQ== 2024-04-28T13:30:48.392 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-04-28T13:30:48.392 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBlTy5m0Bo4JBAASWXwIPgangccm1ErTZtFxg== 2024-04-28T13:30:48.392 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.4] 2024-04-28T13:30:48.392 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQB6Ty5mPoFDAhAA/sprBELVpeGanXkX7jCEHw== 2024-04-28T13:30:48.392 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.5] 2024-04-28T13:30:48.392 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQB9Ty5m77ATCxAA0Fxy75nfDOZjBxd5an+1Gg== 2024-04-28T13:30:48.392 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.6] 2024-04-28T13:30:48.392 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCATy5mrnQ4FhAAmKIZXHzagKbIkyof69CBTA== 2024-04-28T13:30:48.393 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.7] 2024-04-28T13:30:48.393 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCDTy5mSQbQIRAAUHxlY004z6danZMCClG+BQ== 2024-04-28T13:30:48.393 INFO:teuthology.orchestra.run.smithi196.stdout:[client.0] 2024-04-28T13:30:48.393 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBlTy5m8movMBAAnKfzHF0pAkX1F/dZiB0qBg== 2024-04-28T13:30:48.393 INFO:teuthology.orchestra.run.smithi196.stdout:[client.2] 2024-04-28T13:30:48.393 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBlTy5mpPCYORAAPt0CzLT0pHKQTOTtE3L1xw== 2024-04-28T13:30:48.393 INFO:teuthology.orchestra.run.smithi196.stdout:[client.1] 2024-04-28T13:30:48.393 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBmTy5m9BmQBhAAQh8TiIkSt7WqdfovcCbPBg== 2024-04-28T13:30:48.393 INFO:teuthology.orchestra.run.smithi196.stdout:[client.3] 2024-04-28T13:30:48.393 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBmTy5m0oq7DxAAm765AI1ow5e0KlQoBRCr/Q== 2024-04-28T13:30:48.394 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.y] 2024-04-28T13:30:48.394 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBkTy5mh2AXJxAA+BPc+Xi43waIQJ2YjGmw9Q== 2024-04-28T13:30:48.394 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-04-28T13:30:48.394 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBkTy5mGDY2OhAAO3Cl3Q176Jk7wky/cM1g7A== 2024-04-28T13:30:48.394 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-04-28T13:30:48.394 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBlTy5mJqNwCRAA9EAW936ooX1fIhI8D5Y71w== 2024-04-28T13:30:48.394 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.0] 2024-04-28T13:30:48.394 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBpTy5mOG48JhAAMO70X2PHMWLcrTFLBuDe1Q== 2024-04-28T13:30:48.394 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.1] 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBtTy5mADYZBhAARTLL+BqFjPyzcEKny+CBYg== 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.2] 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBwTy5m26gJEBAAuBn3BMPn/EZzOLzMeaaX2w== 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.3] 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBzTy5mrx1KKRAAUG56DD6uIyHP7oD0wOpRkQ== 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.x] 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBkTy5mV7E7MhAAtess+f/dkv11zc9Ujl2jqg== 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBlTy5m5wRGFhAAzFebm/R1yHYctJDIlWndyQ== 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBlTy5m0Bo4JBAASWXwIPgangccm1ErTZtFxg== 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.4] 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQB6Ty5mPoFDAhAA/sprBELVpeGanXkX7jCEHw== 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.5] 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQB9Ty5m77ATCxAA0Fxy75nfDOZjBxd5an+1Gg== 2024-04-28T13:30:48.395 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.6] 2024-04-28T13:30:48.396 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCATy5mrnQ4FhAAmKIZXHzagKbIkyof69CBTA== 2024-04-28T13:30:48.396 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.7] 2024-04-28T13:30:48.396 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCDTy5mSQbQIRAAUHxlY004z6danZMCClG+BQ== 2024-04-28T13:30:48.396 INFO:teuthology.orchestra.run.smithi203.stdout:[client.0] 2024-04-28T13:30:48.396 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBlTy5m8movMBAAnKfzHF0pAkX1F/dZiB0qBg== 2024-04-28T13:30:48.396 INFO:teuthology.orchestra.run.smithi203.stdout:[client.2] 2024-04-28T13:30:48.396 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBlTy5mpPCYORAAPt0CzLT0pHKQTOTtE3L1xw== 2024-04-28T13:30:48.396 INFO:teuthology.orchestra.run.smithi203.stdout:[client.1] 2024-04-28T13:30:48.396 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBmTy5m9BmQBhAAQh8TiIkSt7WqdfovcCbPBg== 2024-04-28T13:30:48.396 INFO:teuthology.orchestra.run.smithi203.stdout:[client.3] 2024-04-28T13:30:48.396 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBmTy5m0oq7DxAAm765AI1ow5e0KlQoBRCr/Q== 2024-04-28T13:30:48.398 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-28T13:30:48.445 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-28T13:30:48.574 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T13:30:48.578 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T13:30:48.715 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T13:30:48.720 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T13:30:48.856 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:48.860 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:48.996 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:49.003 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:49.137 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:49.145 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:49.280 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:49.293 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:49.427 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-28T13:30:49.432 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-28T13:30:49.567 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T13:30:49.575 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T13:30:49.713 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T13:30:49.721 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T13:30:49.857 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:49.865 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:49.992 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:49.998 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:50.129 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:50.134 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:50.259 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:50.264 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T13:30:50.392 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T13:30:50.396 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T13:30:50.523 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T13:30:50.528 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T13:30:50.655 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T13:30:50.659 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T13:30:50.795 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T13:30:50.802 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T13:30:50.935 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-28T13:30:50.935 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-28T13:30:51.008 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-28T13:30:51.145 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-28T13:30:51.212 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-28T13:30:51.300 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-28T13:30:51.447 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-28T13:30:51.521 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-28T13:30:51.606 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-28T13:30:51.756 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-28T13:30:51.829 DEBUG:teuthology.orchestra.run.smithi196:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-28T13:30:51.836 DEBUG:teuthology.orchestra.run.smithi203:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-28T13:30:51.885 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-28T13:30:51.885 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-28T13:30:51.885 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-28T13:30:51.892 INFO:tasks.ceph.mon.a:Started 2024-04-28T13:30:51.892 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-28T13:30:51.893 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-28T13:30:51.897 INFO:tasks.ceph.mon.b:Started 2024-04-28T13:30:51.897 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-28T13:30:51.898 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-28T13:30:51.902 INFO:tasks.ceph.mon.c:Started 2024-04-28T13:30:51.902 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-28T13:30:51.902 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-28T13:30:51.902 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-28T13:30:51.909 INFO:tasks.ceph.mgr.y:Started 2024-04-28T13:30:51.909 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-28T13:30:51.909 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-28T13:30:51.913 INFO:tasks.ceph.mgr.x:Started 2024-04-28T13:30:51.914 INFO:tasks.ceph:Setting crush tunables to default 2024-04-28T13:30:51.914 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd crush tunables default 2024-04-28T13:30:52.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.088+0000 7f6732400640 1 Processor -- start 2024-04-28T13:30:52.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.088+0000 7f6732400640 1 -- start start 2024-04-28T13:30:52.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.089+0000 7f6732400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 0x7f672c070000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:52.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.089+0000 7f6732400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 0x7f672c0d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:52.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.089+0000 7f6732400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f672c0ce8f0 con 0x7f672c070540 2024-04-28T13:30:52.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.089+0000 7f6732400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f672c0d0a10 con 0x7f672c15d150 2024-04-28T13:30:52.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.089+0000 7f6732400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f672c0cebd0 con 0x7f672c0ce460 2024-04-28T13:30:52.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.089+0000 7f6731000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 0x7f672c070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:52.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.089+0000 7f672be00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 0x7f672c0d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:52.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.090+0000 7f6731a00640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f672c070540 0x7f672c0cf580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.196:34480/0 (socket says 172.21.15.196:34480) 2024-04-28T13:30:52.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.090+0000 7f6731a00640 1 -- 172.21.15.196:0/2263704816 learned_addr learned my addr 172.21.15.196:0/2263704816 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:30:52.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.090+0000 7f6731000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 0x7f672c070000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:51496/0 (socket says 172.21.15.196:51496) 2024-04-28T13:30:52.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.090+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 msgr2=0x7f672c070000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-28T13:30:52.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.090+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 msgr2=0x7f672c070000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T13:30:52.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.090+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 0x7f672c070000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T13:30:52.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.090+0000 7f672be00640 1 -- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 msgr2=0x7f672c0d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-28T13:30:52.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.090+0000 7f672be00640 1 -- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 msgr2=0x7f672c0d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T13:30:52.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.090+0000 7f672be00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 0x7f672c0d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T13:30:52.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.090+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 0x7f672c070000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T13:30:52.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.090+0000 7f672be00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 0x7f672c0d1460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T13:30:52.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.291+0000 7f672be00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 0x7f672c0d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:52.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.291+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 0x7f672c070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:52.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.291+0000 7f672be00640 1 -- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 msgr2=0x7f672c0d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-28T13:30:52.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.291+0000 7f672be00640 1 -- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 msgr2=0x7f672c0d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T13:30:52.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.291+0000 7f672be00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 0x7f672c0d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T13:30:52.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.291+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 msgr2=0x7f672c070000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-28T13:30:52.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.291+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 msgr2=0x7f672c070000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T13:30:52.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.291+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 0x7f672c070000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T13:30:52.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.291+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 0x7f672c070000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-28T13:30:52.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.291+0000 7f672be00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 0x7f672c0d1460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-28T13:30:52.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.693+0000 7f672be00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 0x7f672c0d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:52.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.693+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 0x7f672c070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:52.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.693+0000 7f672be00640 1 -- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 msgr2=0x7f672c0d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-28T13:30:52.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.693+0000 7f672be00640 1 -- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 msgr2=0x7f672c0d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T13:30:52.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.693+0000 7f672be00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 0x7f672c0d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T13:30:52.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.693+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 msgr2=0x7f672c070000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-28T13:30:52.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.693+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 msgr2=0x7f672c070000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T13:30:52.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.693+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 0x7f672c070000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T13:30:52.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.693+0000 7f672be00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 0x7f672c0d1460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-28T13:30:52.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:52.693+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 0x7f672c070000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-28T13:30:53.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:53.495+0000 7f672be00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 0x7f672c0d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:53.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:53.495+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 0x7f672c070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:53.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:53.495+0000 7f672be00640 1 -- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 msgr2=0x7f672c0d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-28T13:30:53.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:53.495+0000 7f672be00640 1 -- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 msgr2=0x7f672c0d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T13:30:53.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:53.495+0000 7f672be00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 0x7f672c0d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T13:30:53.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:53.496+0000 7f672be00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 0x7f672c0d1460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-28T13:30:53.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:53.496+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 msgr2=0x7f672c070000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-28T13:30:53.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:53.496+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 msgr2=0x7f672c070000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T13:30:53.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:53.496+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 0x7f672c070000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T13:30:53.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:53.496+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 0x7f672c070000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-28T13:30:55.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.090+0000 7f672a000640 1 -- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 msgr2=0x7f672c0d1460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T13:30:55.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.090+0000 7f672a000640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0ce460 0x7f672c0d1460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:55.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.090+0000 7f672a000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 msgr2=0x7f672c070000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T13:30:55.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.090+0000 7f672a000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.203:3300/0 conn(0x7f672c15d150 0x7f672c070000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:55.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.090+0000 7f672a000640 1 -- 172.21.15.196:0/2263704816 >> v1:172.21.15.203:6789/0 conn(0x7f672c070540 legacy=0x7f672c0cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:55.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.090+0000 7f672a000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 0x7f670c005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:55.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.090+0000 7f672a000640 1 -- 172.21.15.196:0/2263704816 --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f672c0d0c60 con 0x7f670c006170 2024-04-28T13:30:55.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.090+0000 7f672a000640 1 -- 172.21.15.196:0/2263704816 --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f672c0d0a10 con 0x7f670c0036c0 2024-04-28T13:30:55.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.090+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 0x7f670c005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:55.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.090+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 msgr2=0x7f670c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-28T13:30:55.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.090+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 msgr2=0x7f670c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T13:30:55.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.090+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 0x7f670c005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T13:30:55.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.091+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 0x7f670c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T13:30:55.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.292+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 0x7f670c005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:55.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.292+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 msgr2=0x7f670c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-28T13:30:55.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.292+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 msgr2=0x7f670c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T13:30:55.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.292+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 0x7f670c005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T13:30:55.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.292+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 0x7f670c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-28T13:30:55.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.694+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 0x7f670c005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:55.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.694+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 msgr2=0x7f670c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-28T13:30:55.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.694+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 msgr2=0x7f670c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T13:30:55.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.694+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 0x7f670c005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T13:30:55.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:55.694+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 0x7f670c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-28T13:30:56.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:56.496+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 0x7f670c005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:56.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:56.496+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 msgr2=0x7f670c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-28T13:30:56.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:56.496+0000 7f6731000640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 msgr2=0x7f670c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T13:30:56.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:56.496+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 0x7f670c005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T13:30:56.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:56.496+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 0x7f670c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-28T13:30:57.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.143+0000 7f672b400640 1 -- 172.21.15.196:0/2263704816 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 520693625 0 0) 0x7f672c0d0c60 con 0x7f670c006170 2024-04-28T13:30:57.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.143+0000 7f672b400640 1 -- 172.21.15.196:0/2263704816 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6714003520 con 0x7f670c006170 2024-04-28T13:30:57.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.145+0000 7f672b400640 1 -- 172.21.15.196:0/2263704816 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2865861923 0 0) 0x7f6714003520 con 0x7f670c006170 2024-04-28T13:30:57.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.145+0000 7f672b400640 1 -- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 msgr2=0x7f670c005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T13:30:57.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.145+0000 7f672b400640 1 --2- 172.21.15.196:0/2263704816 >> v2:172.21.15.196:3300/0 conn(0x7f670c0036c0 0x7f670c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:57.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.145+0000 7f672b400640 1 -- 172.21.15.196:0/2263704816 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c00af20 con 0x7f670c006170 2024-04-28T13:30:57.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.145+0000 7f672b400640 1 -- 172.21.15.196:0/2263704816 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f671800d580 con 0x7f670c006170 2024-04-28T13:30:57.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.145+0000 7f672b400640 1 -- 172.21.15.196:0/2263704816 >> v1:172.21.15.196:6789/0 conn(0x7f670c006170 legacy=0x7f670c008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:57.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.145+0000 7f6731a00640 1 -- 172.21.15.196:0/2263704816 reap_dead start 2024-04-28T13:30:57.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.145+0000 7f672b400640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6714003a00 0x7f6714003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:57.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.145+0000 7f672b400640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6714004800 0x7f6714004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:57.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.145+0000 7f672b400640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67140053c0 0x7f671400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:57.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.145+0000 7f672b400640 1 -- 172.21.15.196:0/2263704816 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f672c0d0a10 con 0x7f6714004800 2024-04-28T13:30:57.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.145+0000 7f672b400640 1 -- 172.21.15.196:0/2263704816 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f671400c090 con 0x7f6714003a00 2024-04-28T13:30:57.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.145+0000 7f672b400640 1 -- 172.21.15.196:0/2263704816 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f671400c290 con 0x7f67140053c0 2024-04-28T13:30:57.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.146+0000 7f672be00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6714004800 0x7f6714004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:57.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.146+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6714003a00 0x7f6714003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:57.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.147+0000 7f6731a00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67140053c0 0x7f671400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:57.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.147+0000 7f672be00640 1 -- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67140053c0 msgr2=0x7f671400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:30:57.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.147+0000 7f672be00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67140053c0 0x7f671400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:57.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.147+0000 7f672be00640 1 -- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6714003a00 msgr2=0x7f6714003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:30:57.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.147+0000 7f672be00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6714003a00 0x7f6714003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:57.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.147+0000 7f672be00640 1 -- 172.21.15.196:0/2263704816 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6714003520 con 0x7f6714004800 2024-04-28T13:30:57.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.147+0000 7f6731000640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6714003a00 0x7f6714003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:30:57.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.149+0000 7f672be00640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6714004800 0x7f6714004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6718006fd0 tx=0x7f6714005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de994052c43fa6d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:30:57.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.151+0000 7f672b400640 1 -- 172.21.15.196:0/2263704816 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718039070 con 0x7f6714004800 2024-04-28T13:30:57.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.151+0000 7f672b400640 1 -- 172.21.15.196:0/2263704816 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6718034410 con 0x7f6714004800 2024-04-28T13:30:57.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.152+0000 7f672b400640 1 -- 172.21.15.196:0/2263704816 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671800ddd0 con 0x7f6714004800 2024-04-28T13:30:57.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.153+0000 7f6732400640 1 -- 172.21.15.196:0/2263704816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6714004800 msgr2=0x7f6714004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:57.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.153+0000 7f6732400640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6714004800 0x7f6714004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6718006fd0 tx=0x7f6714005830 comp rx=0 tx=0).stop 2024-04-28T13:30:57.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.153+0000 7f6732400640 1 -- 172.21.15.196:0/2263704816 shutdown_connections 2024-04-28T13:30:57.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.153+0000 7f6732400640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6714004800 0x7f6714004c80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:57.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.153+0000 7f6732400640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6714003a00 0x7f6714003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:57.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.153+0000 7f6732400640 1 --2- 172.21.15.196:0/2263704816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67140053c0 0x7f671400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:57.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.153+0000 7f6732400640 1 -- 172.21.15.196:0/2263704816 >> 172.21.15.196:0/2263704816 conn(0x7f672c0d2e40 msgr2=0x7f672c0d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:30:57.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.153+0000 7f6732400640 1 -- 172.21.15.196:0/2263704816 shutdown_connections 2024-04-28T13:30:57.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.153+0000 7f6732400640 1 -- 172.21.15.196:0/2263704816 wait complete. 2024-04-28T13:30:57.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.156+0000 7f6732400640 1 Processor -- start 2024-04-28T13:30:57.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.156+0000 7f6732400640 1 -- start start 2024-04-28T13:30:57.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.158+0000 7f6732400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6714003a00 0x7f672c1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:57.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.158+0000 7f6732400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6714004800 0x7f672c09d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:57.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.158+0000 7f6732400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67140053c0 0x7f672c0c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:57.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.158+0000 7f6732400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f672c0d3ca0 con 0x7f6714003a00 2024-04-28T13:30:57.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.158+0000 7f6732400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f672c0d3b20 con 0x7f67140053c0 2024-04-28T13:30:57.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.158+0000 7f6732400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f672c070000 con 0x7f6714004800 2024-04-28T13:30:57.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.158+0000 7f6731000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6714003a00 0x7f672c1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:57.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.158+0000 7f6731000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6714003a00 0x7f672c1557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:53312/0 (socket says 172.21.15.196:53312) 2024-04-28T13:30:57.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.158+0000 7f6731000640 1 -- 172.21.15.196:0/1676893847 learned_addr learned my addr 172.21.15.196:0/1676893847 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:30:57.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.158+0000 7f6731a00640 1 --2- 172.21.15.196:0/1676893847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67140053c0 0x7f672c0c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:57.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.158+0000 7f672be00640 1 --2- 172.21.15.196:0/1676893847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6714004800 0x7f672c09d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:57.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.159+0000 7f6731000640 1 -- 172.21.15.196:0/1676893847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6714004800 msgr2=0x7f672c09d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:57.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.159+0000 7f6731000640 1 --2- 172.21.15.196:0/1676893847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6714004800 0x7f672c09d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:57.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.159+0000 7f6731000640 1 -- 172.21.15.196:0/1676893847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67140053c0 msgr2=0x7f672c0c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:57.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.159+0000 7f6731000640 1 --2- 172.21.15.196:0/1676893847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67140053c0 0x7f672c0c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:57.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.159+0000 7f6731000640 1 -- 172.21.15.196:0/1676893847 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672c0c7870 con 0x7f6714003a00 2024-04-28T13:30:57.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.159+0000 7f6731000640 1 --2- 172.21.15.196:0/1676893847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6714003a00 0x7f672c1557e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6720017ab0 tx=0x7f6720017f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:30:57.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.159+0000 7f6728c00640 1 -- 172.21.15.196:0/1676893847 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720016c50 con 0x7f6714003a00 2024-04-28T13:30:57.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.159+0000 7f6728c00640 1 -- 172.21.15.196:0/1676893847 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6720028070 con 0x7f6714003a00 2024-04-28T13:30:57.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.159+0000 7f6732400640 1 -- 172.21.15.196:0/1676893847 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672c156640 con 0x7f6714003a00 2024-04-28T13:30:57.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.159+0000 7f6728c00640 1 -- 172.21.15.196:0/1676893847 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67200233f0 con 0x7f6714003a00 2024-04-28T13:30:57.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.160+0000 7f6732400640 1 -- 172.21.15.196:0/1676893847 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f672c156c00 con 0x7f6714003a00 2024-04-28T13:30:57.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.160+0000 7f6728c00640 1 -- 172.21.15.196:0/1676893847 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6720023590 con 0x7f6714003a00 2024-04-28T13:30:57.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.160+0000 7f6728c00640 1 -- 172.21.15.196:0/1676893847 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f672002da30 con 0x7f6714003a00 2024-04-28T13:30:57.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.161+0000 7f6732400640 1 -- 172.21.15.196:0/1676893847 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f672c050730 con 0x7f6714003a00 2024-04-28T13:30:57.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.165+0000 7f6728c00640 1 -- 172.21.15.196:0/1676893847 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6720020070 con 0x7f6714003a00 2024-04-28T13:30:57.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.239+0000 7f6732400640 1 -- 172.21.15.196:0/1676893847 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f672c0c6140 con 0x7f6714003a00 2024-04-28T13:30:57.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.244+0000 7f6728c00640 1 -- 172.21.15.196:0/1676893847 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f672002d460 con 0x7f6714003a00 2024-04-28T13:30:57.245 INFO:teuthology.orchestra.run.smithi196.stderr:adjusted tunables profile to default 2024-04-28T13:30:57.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.245+0000 7f6732400640 1 -- 172.21.15.196:0/1676893847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6714003a00 msgr2=0x7f672c1557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:57.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.245+0000 7f6732400640 1 --2- 172.21.15.196:0/1676893847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6714003a00 0x7f672c1557e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6720017ab0 tx=0x7f6720017f70 comp rx=0 tx=0).stop 2024-04-28T13:30:57.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.245+0000 7f6732400640 1 -- 172.21.15.196:0/1676893847 shutdown_connections 2024-04-28T13:30:57.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.245+0000 7f6732400640 1 --2- 172.21.15.196:0/1676893847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67140053c0 0x7f672c0c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:57.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.245+0000 7f6732400640 1 --2- 172.21.15.196:0/1676893847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6714004800 0x7f672c09d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:57.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.245+0000 7f6732400640 1 --2- 172.21.15.196:0/1676893847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6714003a00 0x7f672c1557e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:57.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.245+0000 7f6732400640 1 -- 172.21.15.196:0/1676893847 >> 172.21.15.196:0/1676893847 conn(0x7f672c0d2e40 msgr2=0x7f672c070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:30:57.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.245+0000 7f6732400640 1 -- 172.21.15.196:0/1676893847 shutdown_connections 2024-04-28T13:30:57.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:30:57.245+0000 7f6732400640 1 -- 172.21.15.196:0/1676893847 wait complete. 2024-04-28T13:30:57.264 INFO:tasks.ceph:check_enable_crimson: False 2024-04-28T13:30:57.265 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-28T13:30:57.265 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:57.265 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-28T13:30:57.307 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:30:57.306+0000 7f3c2a23b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-28T13:30:57.344 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:57.345 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-28T13:30:57.468 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:57.468 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-28T13:30:57.515 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:30:57.514+0000 7f3c2a23b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-28T13:30:57.551 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:30:57.551 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-28T13:30:57.581 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:30:57.580+0000 7f3c2a23b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-28T13:30:57.639 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:57.639 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-28T13:30:57.719 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:57.719 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-28T13:30:57.796 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:57.796 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-28T13:30:57.891 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:30:57.891+0000 7f3c2a23b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-28T13:30:57.910 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:30:57.911 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-28T13:30:57.950 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:30:57.950+0000 7f3c2a23b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-28T13:30:57.977 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 8953d3d4-6285-4b7b-9a65-e32dc56d87ab 0 2024-04-28T13:30:58.208 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.208+0000 7f7652600640 1 Processor -- start 2024-04-28T13:30:58.209 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.208+0000 7f7652600640 1 -- start start 2024-04-28T13:30:58.211 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.210+0000 7f7652600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764c0d3c10 0x7f764c0b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:58.211 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.210+0000 7f7652600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f764c0b8370 0x7f764c0d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:58.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.210+0000 7f7652600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f764c0cfb80 con 0x7f764c0d3830 2024-04-28T13:30:58.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.210+0000 7f7652600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f764c0d2ee0 con 0x7f764c0b8370 2024-04-28T13:30:58.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.210+0000 7f7652600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f764c0d31c0 con 0x7f764c0d3c10 2024-04-28T13:30:58.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.211+0000 7f7651c00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f764c0b8370 0x7f764c0d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:58.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.211+0000 7f764be00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764c0d3c10 0x7f764c0b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:58.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.211+0000 7f7651c00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f764c0b8370 0x7f764c0d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50268/0 (socket says 172.21.15.203:50268) 2024-04-28T13:30:58.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.211+0000 7f7651c00640 1 -- 172.21.15.203:0/4102039269 learned_addr learned my addr 172.21.15.203:0/4102039269 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-28T13:30:58.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.211+0000 7f764be00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764c0d3c10 0x7f764c0b7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:50640/0 (socket says 172.21.15.203:50640) 2024-04-28T13:30:58.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.211+0000 7f764be00640 1 -- 172.21.15.203:0/4102039269 >> v2:172.21.15.203:3300/0 conn(0x7f764c0b8370 msgr2=0x7f764c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:30:58.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.211+0000 7f764be00640 1 --2- 172.21.15.203:0/4102039269 >> v2:172.21.15.203:3300/0 conn(0x7f764c0b8370 0x7f764c0d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.212 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.211+0000 7f764be00640 1 -- 172.21.15.203:0/4102039269 >> v1:172.21.15.196:6789/0 conn(0x7f764c0d3830 legacy=0x7f764c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:30:58.213 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.211+0000 7f764be00640 1 -- 172.21.15.203:0/4102039269 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c0cfe00 con 0x7f764c0d3c10 2024-04-28T13:30:58.214 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.212+0000 7f764be00640 1 --2- 172.21.15.203:0/4102039269 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764c0d3c10 0x7f764c0b7e30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f76380097a0 tx=0x7f763802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdc41466b1227136 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:30:58.214 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.212+0000 7f764b400640 1 -- 172.21.15.203:0/4102039269 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638037070 con 0x7f764c0d3c10 2024-04-28T13:30:58.214 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.212+0000 7f764b400640 1 -- 172.21.15.203:0/4102039269 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f763802ae40 con 0x7f764c0d3c10 2024-04-28T13:30:58.214 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.212+0000 7f764b400640 1 -- 172.21.15.203:0/4102039269 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638032660 con 0x7f764c0d3c10 2024-04-28T13:30:58.214 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.213+0000 7f7652600640 1 -- 172.21.15.203:0/4102039269 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764c0d3c10 msgr2=0x7f764c0b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:58.214 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.213+0000 7f7652600640 1 --2- 172.21.15.203:0/4102039269 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764c0d3c10 0x7f764c0b7e30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f76380097a0 tx=0x7f763802a130 comp rx=0 tx=0).stop 2024-04-28T13:30:58.214 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.213+0000 7f7652600640 1 -- 172.21.15.203:0/4102039269 shutdown_connections 2024-04-28T13:30:58.214 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.213+0000 7f7652600640 1 --2- 172.21.15.203:0/4102039269 >> v2:172.21.15.203:3300/0 conn(0x7f764c0b8370 0x7f764c0d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.215 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.213+0000 7f7652600640 1 --2- 172.21.15.203:0/4102039269 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764c0d3c10 0x7f764c0b7e30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.215 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.213+0000 7f7652600640 1 -- 172.21.15.203:0/4102039269 >> 172.21.15.203:0/4102039269 conn(0x7f764c0d4fe0 msgr2=0x7f764c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:30:58.215 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.213+0000 7f7652600640 1 -- 172.21.15.203:0/4102039269 shutdown_connections 2024-04-28T13:30:58.215 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.213+0000 7f7652600640 1 -- 172.21.15.203:0/4102039269 wait complete. 2024-04-28T13:30:58.216 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.216+0000 7f7652600640 1 Processor -- start 2024-04-28T13:30:58.216 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.216+0000 7f7652600640 1 -- start start 2024-04-28T13:30:58.219 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.217+0000 7f7652600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f764c0b8370 0x7f764c0cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:58.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.217+0000 7f7652600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764c0d3830 0x7f764c1f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:58.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.218+0000 7f7652600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f764c0d3c10 0x7f764c162360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:58.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.218+0000 7f7652600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f764c0d2bb0 con 0x7f764c0b8370 2024-04-28T13:30:58.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.218+0000 7f7652600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f764c0d5d90 con 0x7f764c0d3c10 2024-04-28T13:30:58.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.218+0000 7f7652600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f764c0d0b40 con 0x7f764c0d3830 2024-04-28T13:30:58.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.218+0000 7f7651c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f764c0d3c10 0x7f764c162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:58.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.218+0000 7f764be00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764c0d3830 0x7f764c1f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:58.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.218+0000 7f7651c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f764c0d3c10 0x7f764c162360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50274/0 (socket says 172.21.15.203:50274) 2024-04-28T13:30:58.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.218+0000 7f7651c00640 1 -- 172.21.15.203:0/2572180830 learned_addr learned my addr 172.21.15.203:0/2572180830 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-28T13:30:58.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.218+0000 7f7651200640 1 --2- 172.21.15.203:0/2572180830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f764c0b8370 0x7f764c0cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:58.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.218+0000 7f7651c00640 1 -- 172.21.15.203:0/2572180830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764c0d3830 msgr2=0x7f764c1f85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:58.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.218+0000 7f7651c00640 1 --2- 172.21.15.203:0/2572180830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764c0d3830 0x7f764c1f85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.221 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.218+0000 7f7651c00640 1 -- 172.21.15.203:0/2572180830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f764c0b8370 msgr2=0x7f764c0cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:58.221 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.218+0000 7f7651c00640 1 --2- 172.21.15.203:0/2572180830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f764c0b8370 0x7f764c0cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.221 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.218+0000 7f7651c00640 1 -- 172.21.15.203:0/2572180830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c1f9500 con 0x7f764c0d3c10 2024-04-28T13:30:58.221 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.218+0000 7f7651200640 1 --2- 172.21.15.203:0/2572180830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f764c0b8370 0x7f764c0cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:30:58.221 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.219+0000 7f7651c00640 1 --2- 172.21.15.203:0/2572180830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f764c0d3c10 0x7f764c162360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f764400efc0 tx=0x7f764400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:30:58.221 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.219+0000 7f7648c00640 1 -- 172.21.15.203:0/2572180830 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764400cd20 con 0x7f764c0d3c10 2024-04-28T13:30:58.221 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.219+0000 7f7648c00640 1 -- 172.21.15.203:0/2572180830 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7644003990 con 0x7f764c0d3c10 2024-04-28T13:30:58.221 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.219+0000 7f7648c00640 1 -- 172.21.15.203:0/2572180830 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644003d80 con 0x7f764c0d3c10 2024-04-28T13:30:58.221 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.219+0000 7f7652600640 1 -- 172.21.15.203:0/2572180830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764c1f8f40 con 0x7f764c0d3c10 2024-04-28T13:30:58.221 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.219+0000 7f7652600640 1 -- 172.21.15.203:0/2572180830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764c15fe90 con 0x7f764c0d3c10 2024-04-28T13:30:58.222 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.220+0000 7f7648c00640 1 -- 172.21.15.203:0/2572180830 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7644012050 con 0x7f764c0d3c10 2024-04-28T13:30:58.222 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.220+0000 7f7648c00640 1 -- 172.21.15.203:0/2572180830 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f7644014940 con 0x7f764c0d3c10 2024-04-28T13:30:58.222 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.220+0000 7f7652600640 1 -- 172.21.15.203:0/2572180830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f764c050730 con 0x7f764c0d3c10 2024-04-28T13:30:58.223 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.223+0000 7f7648c00640 1 -- 172.21.15.203:0/2572180830 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7644010040 con 0x7f764c0d3c10 2024-04-28T13:30:58.225 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:30:58.224+0000 7f192bfec040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-28T13:30:58.303 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.303+0000 7f7652600640 1 -- 172.21.15.203:0/2572180830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8953d3d4-6285-4b7b-9a65-e32dc56d87ab", "id": 0} v 0) -- 0x7f764c0ba760 con 0x7f764c0d3c10 2024-04-28T13:30:58.310 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.310+0000 7f7648c00640 1 -- 172.21.15.203:0/2572180830 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8953d3d4-6285-4b7b-9a65-e32dc56d87ab", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f7644014b90 con 0x7f764c0d3c10 2024-04-28T13:30:58.311 INFO:teuthology.orchestra.run.smithi203.stdout:0 2024-04-28T13:30:58.312 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.312+0000 7f7652600640 1 -- 172.21.15.203:0/2572180830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f764c0d3c10 msgr2=0x7f764c162360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:58.312 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.312+0000 7f7652600640 1 --2- 172.21.15.203:0/2572180830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f764c0d3c10 0x7f764c162360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f764400efc0 tx=0x7f764400c490 comp rx=0 tx=0).stop 2024-04-28T13:30:58.313 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.312+0000 7f7652600640 1 -- 172.21.15.203:0/2572180830 shutdown_connections 2024-04-28T13:30:58.313 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.312+0000 7f7652600640 1 --2- 172.21.15.203:0/2572180830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f764c0d3c10 0x7f764c162360 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.313 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.312+0000 7f7652600640 1 --2- 172.21.15.203:0/2572180830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764c0d3830 0x7f764c1f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.313 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.312+0000 7f7652600640 1 --2- 172.21.15.203:0/2572180830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f764c0b8370 0x7f764c0cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.313 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.312+0000 7f7652600640 1 -- 172.21.15.203:0/2572180830 >> 172.21.15.203:0/2572180830 conn(0x7f764c0d4fe0 msgr2=0x7f764c0d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:30:58.313 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.312+0000 7f7652600640 1 -- 172.21.15.203:0/2572180830 shutdown_connections 2024-04-28T13:30:58.313 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.312+0000 7f7652600640 1 -- 172.21.15.203:0/2572180830 wait complete. 2024-04-28T13:30:58.328 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 1f041afc-6cd2-4e7f-a1a4-eb0ffce15749 1 2024-04-28T13:30:58.434 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:30:58.433+0000 7f192bfec040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-28T13:30:58.501 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:30:58.500+0000 7f192bfec040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-28T13:30:58.507 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.507+0000 7f7506a00640 1 Processor -- start 2024-04-28T13:30:58.507 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.507+0000 7f7506a00640 1 -- start start 2024-04-28T13:30:58.509 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.508+0000 7f7506a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f750015f680 0x7f75000d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:58.509 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.508+0000 7f7506a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75000d4f20 0x7f75000d53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:58.510 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.508+0000 7f7506a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75000d58f0 con 0x7f7500163aa0 2024-04-28T13:30:58.510 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.508+0000 7f7506a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f750015fa60 con 0x7f750015f680 2024-04-28T13:30:58.510 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.508+0000 7f7506a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f75000d3f50 con 0x7f75000d4f20 2024-04-28T13:30:58.510 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.508+0000 7f7504a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75000d4f20 0x7f75000d53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:58.510 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.509+0000 7f7504a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75000d4f20 0x7f75000d53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:50656/0 (socket says 172.21.15.203:50656) 2024-04-28T13:30:58.510 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.509+0000 7f7504a00640 1 -- 172.21.15.203:0/3929351791 learned_addr learned my addr 172.21.15.203:0/3929351791 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-28T13:30:58.510 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.509+0000 7f74ffe00640 1 --2- 172.21.15.203:0/3929351791 >> v2:172.21.15.196:3300/0 conn(0x7f750015f680 0x7f75000d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:58.510 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.509+0000 7f7504a00640 1 -- 172.21.15.203:0/3929351791 >> v2:172.21.15.196:3300/0 conn(0x7f750015f680 msgr2=0x7f75000d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:30:58.510 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.509+0000 7f7504a00640 1 --2- 172.21.15.203:0/3929351791 >> v2:172.21.15.196:3300/0 conn(0x7f750015f680 0x7f75000d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.510 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.509+0000 7f7504a00640 1 -- 172.21.15.203:0/3929351791 >> v1:172.21.15.196:6789/0 conn(0x7f7500163aa0 legacy=0x7f75000b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:30:58.510 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.509+0000 7f7504a00640 1 -- 172.21.15.203:0/3929351791 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75000b5090 con 0x7f75000d4f20 2024-04-28T13:30:58.511 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.510+0000 7f7504a00640 1 --2- 172.21.15.203:0/3929351791 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75000d4f20 0x7f75000d53b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f74f8009e50 tx=0x7f74f802a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd7e17e78cb214c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:30:58.512 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.510+0000 7f74fea00640 1 -- 172.21.15.203:0/3929351791 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8036070 con 0x7f75000d4f20 2024-04-28T13:30:58.512 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.510+0000 7f74fea00640 1 -- 172.21.15.203:0/3929351791 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74f8031600 con 0x7f75000d4f20 2024-04-28T13:30:58.512 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.510+0000 7f74fea00640 1 -- 172.21.15.203:0/3929351791 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f80319d0 con 0x7f75000d4f20 2024-04-28T13:30:58.512 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.511+0000 7f7506a00640 1 -- 172.21.15.203:0/3929351791 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75000d4f20 msgr2=0x7f75000d53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:58.512 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.511+0000 7f7506a00640 1 --2- 172.21.15.203:0/3929351791 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75000d4f20 0x7f75000d53b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f74f8009e50 tx=0x7f74f802a6c0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.512 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.511+0000 7f7506a00640 1 -- 172.21.15.203:0/3929351791 shutdown_connections 2024-04-28T13:30:58.512 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.511+0000 7f7506a00640 1 --2- 172.21.15.203:0/3929351791 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75000d4f20 0x7f75000d53b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.512 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.511+0000 7f7506a00640 1 --2- 172.21.15.203:0/3929351791 >> v2:172.21.15.196:3300/0 conn(0x7f750015f680 0x7f75000d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.512 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.511+0000 7f7506a00640 1 -- 172.21.15.203:0/3929351791 >> 172.21.15.203:0/3929351791 conn(0x7f750008eca0 msgr2=0x7f750009e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:30:58.512 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.511+0000 7f7506a00640 1 -- 172.21.15.203:0/3929351791 shutdown_connections 2024-04-28T13:30:58.512 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.511+0000 7f7506a00640 1 -- 172.21.15.203:0/3929351791 wait complete. 2024-04-28T13:30:58.513 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.513+0000 7f7506a00640 1 Processor -- start 2024-04-28T13:30:58.513 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.513+0000 7f7506a00640 1 -- start start 2024-04-28T13:30:58.515 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f7506a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75000d4f20 0x7f750015a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:58.515 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f7506a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f750015f680 0x7f750015fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:58.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f7506a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7500163aa0 0x7f75001f8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:58.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f7506a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7500163f30 con 0x7f7500163aa0 2024-04-28T13:30:58.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f7506a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75000b6280 con 0x7f750015f680 2024-04-28T13:30:58.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f7506a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f75000cfd80 con 0x7f75000d4f20 2024-04-28T13:30:58.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f74ffe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75000d4f20 0x7f750015a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:58.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f74ffe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75000d4f20 0x7f750015a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:50658/0 (socket says 172.21.15.203:50658) 2024-04-28T13:30:58.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f74ffe00640 1 -- 172.21.15.203:0/3794493243 learned_addr learned my addr 172.21.15.203:0/3794493243 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-28T13:30:58.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f74ff400640 1 --2- 172.21.15.203:0/3794493243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f750015f680 0x7f750015fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:58.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f7504a00640 1 --2- 172.21.15.203:0/3794493243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7500163aa0 0x7f75001f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:58.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f74ffe00640 1 -- 172.21.15.203:0/3794493243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f750015f680 msgr2=0x7f750015fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:58.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f74ffe00640 1 --2- 172.21.15.203:0/3794493243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f750015f680 0x7f750015fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f74ffe00640 1 -- 172.21.15.203:0/3794493243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7500163aa0 msgr2=0x7f75001f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:58.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f74ffe00640 1 --2- 172.21.15.203:0/3794493243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7500163aa0 0x7f75001f8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.514+0000 7f74ffe00640 1 -- 172.21.15.203:0/3794493243 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7500160350 con 0x7f75000d4f20 2024-04-28T13:30:58.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.515+0000 7f74ffe00640 1 --2- 172.21.15.203:0/3794493243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75000d4f20 0x7f750015a580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f74e800d800 tx=0x7f74e800dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:30:58.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.515+0000 7f74f7e00640 1 -- 172.21.15.203:0/3794493243 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e800b6c0 con 0x7f75000d4f20 2024-04-28T13:30:58.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.515+0000 7f74f7e00640 1 -- 172.21.15.203:0/3794493243 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74e8019070 con 0x7f75000d4f20 2024-04-28T13:30:58.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.515+0000 7f74f7e00640 1 -- 172.21.15.203:0/3794493243 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e800ba80 con 0x7f75000d4f20 2024-04-28T13:30:58.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.515+0000 7f7506a00640 1 -- 172.21.15.203:0/3794493243 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7500160630 con 0x7f75000d4f20 2024-04-28T13:30:58.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.515+0000 7f7506a00640 1 -- 172.21.15.203:0/3794493243 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7500160bf0 con 0x7f75000d4f20 2024-04-28T13:30:58.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.515+0000 7f74f7e00640 1 -- 172.21.15.203:0/3794493243 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f74e800bc20 con 0x7f75000d4f20 2024-04-28T13:30:58.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.515+0000 7f74f7e00640 1 -- 172.21.15.203:0/3794493243 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f74e8014910 con 0x7f75000d4f20 2024-04-28T13:30:58.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.516+0000 7f7506a00640 1 -- 172.21.15.203:0/3794493243 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f750006a290 con 0x7f75000d4f20 2024-04-28T13:30:58.519 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.519+0000 7f74f7e00640 1 -- 172.21.15.203:0/3794493243 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f74e8014b10 con 0x7f75000d4f20 2024-04-28T13:30:58.597 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.597+0000 7f7506a00640 1 -- 172.21.15.203:0/3794493243 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1f041afc-6cd2-4e7f-a1a4-eb0ffce15749", "id": 1} v 0) -- 0x7f75000b45a0 con 0x7f75000d4f20 2024-04-28T13:30:58.604 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.604+0000 7f74f7e00640 1 -- 172.21.15.203:0/3794493243 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1f041afc-6cd2-4e7f-a1a4-eb0ffce15749", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f74e8014dc0 con 0x7f75000d4f20 2024-04-28T13:30:58.604 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-28T13:30:58.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.605+0000 7f7506a00640 1 -- 172.21.15.203:0/3794493243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75000d4f20 msgr2=0x7f750015a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:58.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.605+0000 7f7506a00640 1 --2- 172.21.15.203:0/3794493243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75000d4f20 0x7f750015a580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f74e800d800 tx=0x7f74e800dcc0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.605+0000 7f7506a00640 1 -- 172.21.15.203:0/3794493243 shutdown_connections 2024-04-28T13:30:58.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.605+0000 7f7506a00640 1 --2- 172.21.15.203:0/3794493243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7500163aa0 0x7f75001f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.605+0000 7f7506a00640 1 --2- 172.21.15.203:0/3794493243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f750015f680 0x7f750015fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.605+0000 7f7506a00640 1 --2- 172.21.15.203:0/3794493243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75000d4f20 0x7f750015a580 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.605+0000 7f7506a00640 1 -- 172.21.15.203:0/3794493243 >> 172.21.15.203:0/3794493243 conn(0x7f750008eca0 msgr2=0x7f7500163070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:30:58.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.605+0000 7f7506a00640 1 -- 172.21.15.203:0/3794493243 shutdown_connections 2024-04-28T13:30:58.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.605+0000 7f7506a00640 1 -- 172.21.15.203:0/3794493243 wait complete. 2024-04-28T13:30:58.620 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new c52ac708-974d-4c56-834b-c384bb41afac 2 2024-04-28T13:30:58.786 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.785+0000 7fd7dce00640 1 Processor -- start 2024-04-28T13:30:58.786 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.785+0000 7fd7dce00640 1 -- start start 2024-04-28T13:30:58.788 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.787+0000 7fd7dce00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7d80d3830 0x7fd7d80d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:58.788 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.787+0000 7fd7dce00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd7d80d3c10 0x7fd7d80b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:58.788 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.787+0000 7fd7dce00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7d80cfb80 con 0x7fd7d80b8370 2024-04-28T13:30:58.788 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.787+0000 7fd7dce00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd7d80d2ee0 con 0x7fd7d80d3c10 2024-04-28T13:30:58.788 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.787+0000 7fd7dce00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d80d31c0 con 0x7fd7d80d3830 2024-04-28T13:30:58.788 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.787+0000 7fd7d7400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7d80d3830 0x7fd7d80d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:58.788 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.787+0000 7fd7d7400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7d80d3830 0x7fd7d80d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:50670/0 (socket says 172.21.15.203:50670) 2024-04-28T13:30:58.789 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.787+0000 7fd7d7400640 1 -- 172.21.15.203:0/920594111 learned_addr learned my addr 172.21.15.203:0/920594111 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-28T13:30:58.789 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.787+0000 7fd7d6a00640 1 --2- 172.21.15.203:0/920594111 >> v2:172.21.15.203:3300/0 conn(0x7fd7d80d3c10 0x7fd7d80b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:58.789 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.788+0000 7fd7d7400640 1 -- 172.21.15.203:0/920594111 >> v2:172.21.15.203:3300/0 conn(0x7fd7d80d3c10 msgr2=0x7fd7d80b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:30:58.789 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.788+0000 7fd7d7400640 1 --2- 172.21.15.203:0/920594111 >> v2:172.21.15.203:3300/0 conn(0x7fd7d80d3c10 0x7fd7d80b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.789 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.788+0000 7fd7d7400640 1 -- 172.21.15.203:0/920594111 >> v1:172.21.15.196:6789/0 conn(0x7fd7d80b8370 legacy=0x7fd7d80d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:30:58.789 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.788+0000 7fd7d7400640 1 -- 172.21.15.203:0/920594111 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d80cfe00 con 0x7fd7d80d3830 2024-04-28T13:30:58.790 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.789+0000 7fd7d7400640 1 --2- 172.21.15.203:0/920594111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7d80d3830 0x7fd7d80d2400 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009e50 tx=0x7fd7c802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc6c3303f5652d55 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:30:58.790 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.789+0000 7fd7d6000640 1 -- 172.21.15.203:0/920594111 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8036070 con 0x7fd7d80d3830 2024-04-28T13:30:58.790 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.789+0000 7fd7d6000640 1 -- 172.21.15.203:0/920594111 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c8031600 con 0x7fd7d80d3830 2024-04-28T13:30:58.790 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.789+0000 7fd7d6000640 1 -- 172.21.15.203:0/920594111 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80319d0 con 0x7fd7d80d3830 2024-04-28T13:30:58.791 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.790+0000 7fd7dce00640 1 -- 172.21.15.203:0/920594111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7d80d3830 msgr2=0x7fd7d80d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:58.791 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.790+0000 7fd7dce00640 1 --2- 172.21.15.203:0/920594111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7d80d3830 0x7fd7d80d2400 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009e50 tx=0x7fd7c802a6d0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.791 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.790+0000 7fd7dce00640 1 -- 172.21.15.203:0/920594111 shutdown_connections 2024-04-28T13:30:58.791 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.790+0000 7fd7dce00640 1 --2- 172.21.15.203:0/920594111 >> v2:172.21.15.203:3300/0 conn(0x7fd7d80d3c10 0x7fd7d80b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.791 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.790+0000 7fd7dce00640 1 --2- 172.21.15.203:0/920594111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7d80d3830 0x7fd7d80d2400 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.791 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.790+0000 7fd7dce00640 1 -- 172.21.15.203:0/920594111 >> 172.21.15.203:0/920594111 conn(0x7fd7d80d4fe0 msgr2=0x7fd7d80d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:30:58.791 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.790+0000 7fd7dce00640 1 -- 172.21.15.203:0/920594111 shutdown_connections 2024-04-28T13:30:58.791 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.790+0000 7fd7dce00640 1 -- 172.21.15.203:0/920594111 wait complete. 2024-04-28T13:30:58.792 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:30:58.790+0000 7f192bfec040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-28T13:30:58.792 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.792+0000 7fd7dce00640 1 Processor -- start 2024-04-28T13:30:58.792 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.792+0000 7fd7dce00640 1 -- start start 2024-04-28T13:30:58.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.793+0000 7fd7dce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d80b8370 0x7fd7d81606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:58.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.793+0000 7fd7dce00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7d80d3830 0x7fd7d814b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:58.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.793+0000 7fd7dce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7d80d3c10 0x7fd7d815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:58.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.793+0000 7fd7dce00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d80d5d90 con 0x7fd7d80b8370 2024-04-28T13:30:58.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.793+0000 7fd7dce00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d80d2940 con 0x7fd7d80d3c10 2024-04-28T13:30:58.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.793+0000 7fd7dce00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d80d2bb0 con 0x7fd7d80d3830 2024-04-28T13:30:58.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.793+0000 7fd7d6a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7d80d3830 0x7fd7d814b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:58.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.793+0000 7fd7d7e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7d80d3c10 0x7fd7d815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:58.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.793+0000 7fd7d6a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7d80d3830 0x7fd7d814b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:50686/0 (socket says 172.21.15.203:50686) 2024-04-28T13:30:58.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.793+0000 7fd7d6a00640 1 -- 172.21.15.203:0/2356648268 learned_addr learned my addr 172.21.15.203:0/2356648268 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-28T13:30:58.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.793+0000 7fd7d7400640 1 --2- 172.21.15.203:0/2356648268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d80b8370 0x7fd7d81606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:58.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.794+0000 7fd7d7e00640 1 -- 172.21.15.203:0/2356648268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7d80d3830 msgr2=0x7fd7d814b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:58.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.794+0000 7fd7d7e00640 1 --2- 172.21.15.203:0/2356648268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7d80d3830 0x7fd7d814b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.794+0000 7fd7d7e00640 1 -- 172.21.15.203:0/2356648268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d80b8370 msgr2=0x7fd7d81606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:58.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.794+0000 7fd7d7e00640 1 --2- 172.21.15.203:0/2356648268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d80b8370 0x7fd7d81606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.794+0000 7fd7d7e00640 1 -- 172.21.15.203:0/2356648268 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d8160c10 con 0x7fd7d80d3c10 2024-04-28T13:30:58.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.794+0000 7fd7d7e00640 1 --2- 172.21.15.203:0/2356648268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7d80d3c10 0x7fd7d815ebf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000c410 tx=0x7fd7d000c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:30:58.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.794+0000 7fd7cf400640 1 -- 172.21.15.203:0/2356648268 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0011790 con 0x7fd7d80d3c10 2024-04-28T13:30:58.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.794+0000 7fd7cf400640 1 -- 172.21.15.203:0/2356648268 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7d0017070 con 0x7fd7d80d3c10 2024-04-28T13:30:58.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.794+0000 7fd7dce00640 1 -- 172.21.15.203:0/2356648268 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d81f5f40 con 0x7fd7d80d3c10 2024-04-28T13:30:58.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.794+0000 7fd7cf400640 1 -- 172.21.15.203:0/2356648268 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0011b70 con 0x7fd7d80d3c10 2024-04-28T13:30:58.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.794+0000 7fd7dce00640 1 -- 172.21.15.203:0/2356648268 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d81f5000 con 0x7fd7d80d3c10 2024-04-28T13:30:58.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.794+0000 7fd7cf400640 1 -- 172.21.15.203:0/2356648268 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd7d0011d10 con 0x7fd7d80d3c10 2024-04-28T13:30:58.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.795+0000 7fd7cf400640 1 -- 172.21.15.203:0/2356648268 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fd7d0007020 con 0x7fd7d80d3c10 2024-04-28T13:30:58.797 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.795+0000 7fd7dce00640 1 -- 172.21.15.203:0/2356648268 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d81f5320 con 0x7fd7d80d3c10 2024-04-28T13:30:58.798 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.798+0000 7fd7cf400640 1 -- 172.21.15.203:0/2356648268 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd7d81f5320 con 0x7fd7d80d3c10 2024-04-28T13:30:58.845 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:30:58.844+0000 7f192bfec040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-28T13:30:58.871 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:30:58.870+0000 7f3c2a23b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-28T13:30:58.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.874+0000 7fd7dce00640 1 -- 172.21.15.203:0/2356648268 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c52ac708-974d-4c56-834b-c384bb41afac", "id": 2} v 0) -- 0x7fd7d8050730 con 0x7fd7d80d3c10 2024-04-28T13:30:58.881 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.881+0000 7fd7cf400640 1 -- 172.21.15.203:0/2356648268 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c52ac708-974d-4c56-834b-c384bb41afac", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fd7d0021020 con 0x7fd7d80d3c10 2024-04-28T13:30:58.881 INFO:teuthology.orchestra.run.smithi203.stdout:2 2024-04-28T13:30:58.883 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.882+0000 7fd7dce00640 1 -- 172.21.15.203:0/2356648268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7d80d3c10 msgr2=0x7fd7d815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:58.883 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.882+0000 7fd7dce00640 1 --2- 172.21.15.203:0/2356648268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7d80d3c10 0x7fd7d815ebf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000c410 tx=0x7fd7d000c8d0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.883 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.882+0000 7fd7dce00640 1 -- 172.21.15.203:0/2356648268 shutdown_connections 2024-04-28T13:30:58.883 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.882+0000 7fd7dce00640 1 --2- 172.21.15.203:0/2356648268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7d80d3c10 0x7fd7d815ebf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.883 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.882+0000 7fd7dce00640 1 --2- 172.21.15.203:0/2356648268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7d80d3830 0x7fd7d814b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.883 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.882+0000 7fd7dce00640 1 --2- 172.21.15.203:0/2356648268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d80b8370 0x7fd7d81606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:58.883 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.882+0000 7fd7dce00640 1 -- 172.21.15.203:0/2356648268 >> 172.21.15.203:0/2356648268 conn(0x7fd7d80d4fe0 msgr2=0x7fd7d80d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:30:58.883 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.882+0000 7fd7dce00640 1 -- 172.21.15.203:0/2356648268 shutdown_connections 2024-04-28T13:30:58.883 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:58.882+0000 7fd7dce00640 1 -- 172.21.15.203:0/2356648268 wait complete. 2024-04-28T13:30:58.899 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 6cad7e98-14c8-4217-a1ad-c1c1378c0c1c 3 2024-04-28T13:30:58.984 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:30:58.983+0000 7f3c2a23b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-28T13:30:59.038 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:30:59.038+0000 7f3c2a23b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-28T13:30:59.074 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.074+0000 7efed6000640 1 Processor -- start 2024-04-28T13:30:59.074 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.074+0000 7efed6000640 1 -- start start 2024-04-28T13:30:59.076 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.075+0000 7efed6000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efed00d3830 0x7efed00d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:59.077 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.075+0000 7efed6000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efed00d3c10 0x7efed00b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:59.077 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.075+0000 7efed6000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efed00cfb80 con 0x7efed00b8370 2024-04-28T13:30:59.077 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.075+0000 7efed6000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7efed00d2ee0 con 0x7efed00d3c10 2024-04-28T13:30:59.077 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.075+0000 7efed6000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efed00d31c0 con 0x7efed00d3830 2024-04-28T13:30:59.077 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.076+0000 7efed4c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efed00d3830 0x7efed00d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:59.077 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.076+0000 7efecfe00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efed00d3c10 0x7efed00b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:59.077 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.076+0000 7efed4c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efed00d3830 0x7efed00d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:50696/0 (socket says 172.21.15.203:50696) 2024-04-28T13:30:59.077 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.076+0000 7efed4c00640 1 -- 172.21.15.203:0/408446560 learned_addr learned my addr 172.21.15.203:0/408446560 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-28T13:30:59.077 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.076+0000 7efecfe00640 1 -- 172.21.15.203:0/408446560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efed00d3830 msgr2=0x7efed00d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:30:59.077 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.076+0000 7efecfe00640 1 --2- 172.21.15.203:0/408446560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efed00d3830 0x7efed00d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.077 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.076+0000 7efecfe00640 1 -- 172.21.15.203:0/408446560 >> v1:172.21.15.203:6789/0 conn(0x7efed00b8370 legacy=0x7efed00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:59.077 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.076+0000 7efecfe00640 1 -- 172.21.15.203:0/408446560 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed00cfe00 con 0x7efed00d3c10 2024-04-28T13:30:59.078 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.076+0000 7efed4c00640 1 --2- 172.21.15.203:0/408446560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efed00d3830 0x7efed00d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:30:59.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.077+0000 7efecfe00640 1 --2- 172.21.15.203:0/408446560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed00d3c10 0x7efed00b7e30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efebc0097a0 tx=0x7efebc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aef42fb387279ab server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:30:59.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.078+0000 7efecf400640 1 -- 172.21.15.203:0/408446560 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc037070 con 0x7efed00d3c10 2024-04-28T13:30:59.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.078+0000 7efecf400640 1 -- 172.21.15.203:0/408446560 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efebc032440 con 0x7efed00d3c10 2024-04-28T13:30:59.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.078+0000 7efecf400640 1 -- 172.21.15.203:0/408446560 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc032810 con 0x7efed00d3c10 2024-04-28T13:30:59.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.078+0000 7efed6000640 1 -- 172.21.15.203:0/408446560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed00d3c10 msgr2=0x7efed00b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:59.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.078+0000 7efed6000640 1 --2- 172.21.15.203:0/408446560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed00d3c10 0x7efed00b7e30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efebc0097a0 tx=0x7efebc02a180 comp rx=0 tx=0).stop 2024-04-28T13:30:59.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.078+0000 7efed6000640 1 -- 172.21.15.203:0/408446560 shutdown_connections 2024-04-28T13:30:59.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.078+0000 7efed6000640 1 --2- 172.21.15.203:0/408446560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed00d3c10 0x7efed00b7e30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.078+0000 7efed6000640 1 --2- 172.21.15.203:0/408446560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efed00d3830 0x7efed00d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.078+0000 7efed6000640 1 -- 172.21.15.203:0/408446560 >> 172.21.15.203:0/408446560 conn(0x7efed00d4fe0 msgr2=0x7efed00d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:30:59.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.078+0000 7efed6000640 1 -- 172.21.15.203:0/408446560 shutdown_connections 2024-04-28T13:30:59.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.078+0000 7efed6000640 1 -- 172.21.15.203:0/408446560 wait complete. 2024-04-28T13:30:59.080 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.080+0000 7efed6000640 1 Processor -- start 2024-04-28T13:30:59.080 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.080+0000 7efed6000640 1 -- start start 2024-04-28T13:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.081+0000 7efed6000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed00b8370 0x7efed014f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.081+0000 7efed6000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efed00d3830 0x7efed014f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.081+0000 7efed6000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efed00d3c10 0x7efed00b8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.081+0000 7efed6000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efed00d5d90 con 0x7efed00d3c10 2024-04-28T13:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.081+0000 7efed6000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efed00d2940 con 0x7efed00b8370 2024-04-28T13:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.081+0000 7efed6000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efed00d2bb0 con 0x7efed00d3830 2024-04-28T13:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.081+0000 7efed4c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed00b8370 0x7efed014f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:59.083 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.081+0000 7efecfe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efed00d3830 0x7efed014f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:59.083 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.081+0000 7efecfe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efed00d3830 0x7efed014f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:50706/0 (socket says 172.21.15.203:50706) 2024-04-28T13:30:59.083 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.081+0000 7efecfe00640 1 -- 172.21.15.203:0/1745147538 learned_addr learned my addr 172.21.15.203:0/1745147538 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-28T13:30:59.083 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.082+0000 7efed5600640 1 --2- 172.21.15.203:0/1745147538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efed00d3c10 0x7efed00b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:59.083 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.082+0000 7efecfe00640 1 -- 172.21.15.203:0/1745147538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed00b8370 msgr2=0x7efed014f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:59.083 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.082+0000 7efecfe00640 1 --2- 172.21.15.203:0/1745147538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed00b8370 0x7efed014f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.083 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.082+0000 7efecfe00640 1 -- 172.21.15.203:0/1745147538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efed00d3c10 msgr2=0x7efed00b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:59.083 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.082+0000 7efecfe00640 1 --2- 172.21.15.203:0/1745147538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efed00d3c10 0x7efed00b8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.083 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.082+0000 7efecfe00640 1 -- 172.21.15.203:0/1745147538 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed00b9e70 con 0x7efed00d3830 2024-04-28T13:30:59.084 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.082+0000 7efecfe00640 1 --2- 172.21.15.203:0/1745147538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efed00d3830 0x7efed014f550 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efebc035040 tx=0x7efebc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:30:59.084 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.082+0000 7efeccc00640 1 -- 172.21.15.203:0/1745147538 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc037040 con 0x7efed00d3830 2024-04-28T13:30:59.084 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.083+0000 7efeccc00640 1 -- 172.21.15.203:0/1745147538 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efebc030500 con 0x7efed00d3830 2024-04-28T13:30:59.085 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.083+0000 7efeccc00640 1 -- 172.21.15.203:0/1745147538 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc030910 con 0x7efed00d3830 2024-04-28T13:30:59.085 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.083+0000 7efed6000640 1 -- 172.21.15.203:0/1745147538 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed00b98b0 con 0x7efed00d3830 2024-04-28T13:30:59.085 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.083+0000 7efed6000640 1 -- 172.21.15.203:0/1745147538 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efed00ce7c0 con 0x7efed00d3830 2024-04-28T13:30:59.085 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.083+0000 7efeccc00640 1 -- 172.21.15.203:0/1745147538 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efebc044020 con 0x7efed00d3830 2024-04-28T13:30:59.085 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.083+0000 7efeccc00640 1 -- 172.21.15.203:0/1745147538 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7efebc03b430 con 0x7efed00d3830 2024-04-28T13:30:59.085 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.084+0000 7efed6000640 1 -- 172.21.15.203:0/1745147538 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed0050730 con 0x7efed00d3830 2024-04-28T13:30:59.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.087+0000 7efeccc00640 1 -- 172.21.15.203:0/1745147538 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7efebc03b710 con 0x7efed00d3830 2024-04-28T13:30:59.163 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.163+0000 7efed6000640 1 -- 172.21.15.203:0/1745147538 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6cad7e98-14c8-4217-a1ad-c1c1378c0c1c", "id": 3} v 0) -- 0x7efed00d17d0 con 0x7efed00d3830 2024-04-28T13:30:59.170 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.170+0000 7efeccc00640 1 -- 172.21.15.203:0/1745147538 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6cad7e98-14c8-4217-a1ad-c1c1378c0c1c", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7efebc03bb60 con 0x7efed00d3830 2024-04-28T13:30:59.170 INFO:teuthology.orchestra.run.smithi203.stdout:3 2024-04-28T13:30:59.172 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.171+0000 7efed6000640 1 -- 172.21.15.203:0/1745147538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efed00d3830 msgr2=0x7efed014f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:59.172 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.171+0000 7efed6000640 1 --2- 172.21.15.203:0/1745147538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efed00d3830 0x7efed014f550 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efebc035040 tx=0x7efebc0046c0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.172 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.171+0000 7efed6000640 1 -- 172.21.15.203:0/1745147538 shutdown_connections 2024-04-28T13:30:59.172 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.171+0000 7efed6000640 1 --2- 172.21.15.203:0/1745147538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efed00d3c10 0x7efed00b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.172 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.171+0000 7efed6000640 1 --2- 172.21.15.203:0/1745147538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efed00d3830 0x7efed014f550 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.173 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.171+0000 7efed6000640 1 --2- 172.21.15.203:0/1745147538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed00b8370 0x7efed014f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.173 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.171+0000 7efed6000640 1 -- 172.21.15.203:0/1745147538 >> 172.21.15.203:0/1745147538 conn(0x7efed00d4fe0 msgr2=0x7efed00d54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:30:59.173 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.172+0000 7efed6000640 1 -- 172.21.15.203:0/1745147538 shutdown_connections 2024-04-28T13:30:59.173 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.172+0000 7efed6000640 1 -- 172.21.15.203:0/1745147538 wait complete. 2024-04-28T13:30:59.189 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 2092249e-3aa5-4a4b-a5e9-13843e365ad3 4 2024-04-28T13:30:59.245 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:30:59.245+0000 7f3c2a23b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-28T13:30:59.304 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:30:59.304+0000 7f3c2a23b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-28T13:30:59.368 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:30:59.368+0000 7f3c2a23b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-28T13:30:59.371 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.370+0000 7fd805800640 1 Processor -- start 2024-04-28T13:30:59.371 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.371+0000 7fd805800640 1 -- start start 2024-04-28T13:30:59.373 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.372+0000 7fd805800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8000d3d40 0x7fd8000d5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:59.373 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.372+0000 7fd805800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd8000d1ff0 0x7fd80006cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:59.373 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.372+0000 7fd805800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8000d27f0 con 0x7fd80015f320 2024-04-28T13:30:59.373 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.372+0000 7fd805800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd8000d0630 con 0x7fd8000d1ff0 2024-04-28T13:30:59.373 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.372+0000 7fd805800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd8000d0ed0 con 0x7fd8000d3d40 2024-04-28T13:30:59.373 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.372+0000 7fd7f7e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8000d3d40 0x7fd8000d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:59.373 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.372+0000 7fd7ffe00640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd80015f320 0x7fd8000d18e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:59444/0 (socket says 172.21.15.203:59444) 2024-04-28T13:30:59.373 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.372+0000 7fd7ffe00640 1 -- 172.21.15.203:0/1561394999 learned_addr learned my addr 172.21.15.203:0/1561394999 (peer_addr_for_me v1:172.21.15.203:0/0) 2024-04-28T13:30:59.374 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.372+0000 7fd804e00640 1 --2- 172.21.15.203:0/1561394999 >> v2:172.21.15.196:3300/0 conn(0x7fd8000d1ff0 0x7fd80006cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:59.374 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.372+0000 7fd7ff400640 1 -- 172.21.15.203:0/1561394999 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4233895137 0 0) 0x7fd8000d27f0 con 0x7fd80015f320 2024-04-28T13:30:59.374 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.373+0000 7fd7f7e00640 1 -- 172.21.15.203:0/1561394999 >> v2:172.21.15.196:3300/0 conn(0x7fd8000d1ff0 msgr2=0x7fd80006cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:30:59.374 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.373+0000 7fd7f7e00640 1 --2- 172.21.15.203:0/1561394999 >> v2:172.21.15.196:3300/0 conn(0x7fd8000d1ff0 0x7fd80006cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.374 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.373+0000 7fd7f7e00640 1 -- 172.21.15.203:0/1561394999 >> v1:172.21.15.203:6789/0 conn(0x7fd80015f320 legacy=0x7fd8000d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:59.374 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.373+0000 7fd7f7e00640 1 -- 172.21.15.203:0/1561394999 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8000d0880 con 0x7fd8000d3d40 2024-04-28T13:30:59.375 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.374+0000 7fd7f7e00640 1 --2- 172.21.15.203:0/1561394999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8000d3d40 0x7fd8000d5b30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80097a0 tx=0x7fd7e802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3fa536f7f11e8b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:30:59.375 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.374+0000 7fd7ff400640 1 -- 172.21.15.203:0/1561394999 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8037070 con 0x7fd8000d3d40 2024-04-28T13:30:59.375 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.374+0000 7fd7ff400640 1 -- 172.21.15.203:0/1561394999 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e8032440 con 0x7fd8000d3d40 2024-04-28T13:30:59.375 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.374+0000 7fd7ff400640 1 -- 172.21.15.203:0/1561394999 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8032810 con 0x7fd8000d3d40 2024-04-28T13:30:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.374+0000 7fd805800640 1 -- 172.21.15.203:0/1561394999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8000d3d40 msgr2=0x7fd8000d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.374+0000 7fd805800640 1 --2- 172.21.15.203:0/1561394999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8000d3d40 0x7fd8000d5b30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80097a0 tx=0x7fd7e802a180 comp rx=0 tx=0).stop 2024-04-28T13:30:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.375+0000 7fd805800640 1 -- 172.21.15.203:0/1561394999 shutdown_connections 2024-04-28T13:30:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.375+0000 7fd805800640 1 --2- 172.21.15.203:0/1561394999 >> v2:172.21.15.196:3300/0 conn(0x7fd8000d1ff0 0x7fd80006cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.375+0000 7fd805800640 1 --2- 172.21.15.203:0/1561394999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8000d3d40 0x7fd8000d5b30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.375+0000 7fd805800640 1 -- 172.21.15.203:0/1561394999 >> 172.21.15.203:0/1561394999 conn(0x7fd8000d5010 msgr2=0x7fd8000d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:30:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.375+0000 7fd805800640 1 -- 172.21.15.203:0/1561394999 shutdown_connections 2024-04-28T13:30:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.375+0000 7fd805800640 1 -- 172.21.15.203:0/1561394999 wait complete. 2024-04-28T13:30:59.377 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.376+0000 7fd805800640 1 Processor -- start 2024-04-28T13:30:59.377 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.376+0000 7fd805800640 1 -- start start 2024-04-28T13:30:59.378 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.377+0000 7fd805800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8000d1ff0 0x7fd800160260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:59.378 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.377+0000 7fd805800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8000d3d40 0x7fd8000ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:59.379 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.377+0000 7fd805800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd80015f320 0x7fd800161ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:59.379 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.377+0000 7fd805800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd80015f700 con 0x7fd8000d1ff0 2024-04-28T13:30:59.379 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.377+0000 7fd805800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd8000d2480 con 0x7fd8000d3d40 2024-04-28T13:30:59.379 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.377+0000 7fd805800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd80006d430 con 0x7fd80015f320 2024-04-28T13:30:59.379 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.377+0000 7fd804e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd80015f320 0x7fd800161ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:59.379 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.377+0000 7fd7f7e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8000d3d40 0x7fd8000ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:59.379 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.377+0000 7fd804e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd80015f320 0x7fd800161ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:50726/0 (socket says 172.21.15.203:50726) 2024-04-28T13:30:59.379 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.378+0000 7fd804e00640 1 -- 172.21.15.203:0/1009831425 learned_addr learned my addr 172.21.15.203:0/1009831425 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-28T13:30:59.379 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.378+0000 7fd7ffe00640 1 --2- 172.21.15.203:0/1009831425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8000d1ff0 0x7fd800160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:59.379 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.378+0000 7fd7f7e00640 1 -- 172.21.15.203:0/1009831425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd80015f320 msgr2=0x7fd800161ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:59.379 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.378+0000 7fd7f7e00640 1 --2- 172.21.15.203:0/1009831425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd80015f320 0x7fd800161ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.379 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.378+0000 7fd7f7e00640 1 -- 172.21.15.203:0/1009831425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8000d1ff0 msgr2=0x7fd800160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:59.380 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.378+0000 7fd7f7e00640 1 --2- 172.21.15.203:0/1009831425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8000d1ff0 0x7fd800160260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.380 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.378+0000 7fd7f7e00640 1 -- 172.21.15.203:0/1009831425 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8000ca720 con 0x7fd8000d3d40 2024-04-28T13:30:59.380 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.378+0000 7fd7f7e00640 1 --2- 172.21.15.203:0/1009831425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8000d3d40 0x7fd8000ce8f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8031000 tx=0x7fd7e80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:30:59.381 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.379+0000 7fd7fcc00640 1 -- 172.21.15.203:0/1009831425 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8041070 con 0x7fd8000d3d40 2024-04-28T13:30:59.381 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.379+0000 7fd7fcc00640 1 -- 172.21.15.203:0/1009831425 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e803c870 con 0x7fd8000d3d40 2024-04-28T13:30:59.381 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.379+0000 7fd7fcc00640 1 -- 172.21.15.203:0/1009831425 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8037040 con 0x7fd8000d3d40 2024-04-28T13:30:59.381 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.379+0000 7fd805800640 1 -- 172.21.15.203:0/1009831425 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8000caa00 con 0x7fd8000d3d40 2024-04-28T13:30:59.381 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.379+0000 7fd805800640 1 -- 172.21.15.203:0/1009831425 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8000cafc0 con 0x7fd8000d3d40 2024-04-28T13:30:59.381 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.379+0000 7fd7fcc00640 1 -- 172.21.15.203:0/1009831425 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd7e8044070 con 0x7fd8000d3d40 2024-04-28T13:30:59.381 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.379+0000 7fd7fcc00640 1 -- 172.21.15.203:0/1009831425 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fd7e803cc60 con 0x7fd8000d3d40 2024-04-28T13:30:59.382 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.380+0000 7fd805800640 1 -- 172.21.15.203:0/1009831425 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd800050730 con 0x7fd8000d3d40 2024-04-28T13:30:59.382 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.382+0000 7fd7fcc00640 1 -- 172.21.15.203:0/1009831425 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd7e8046450 con 0x7fd8000d3d40 2024-04-28T13:30:59.462 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.461+0000 7fd805800640 1 -- 172.21.15.203:0/1009831425 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2092249e-3aa5-4a4b-a5e9-13843e365ad3", "id": 4} v 0) -- 0x7fd80006de70 con 0x7fd8000d3d40 2024-04-28T13:30:59.475 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.474+0000 7fd7fcc00640 1 -- 172.21.15.203:0/1009831425 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2092249e-3aa5-4a4b-a5e9-13843e365ad3", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fd7e8046630 con 0x7fd8000d3d40 2024-04-28T13:30:59.475 INFO:teuthology.orchestra.run.smithi203.stdout:4 2024-04-28T13:30:59.477 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.476+0000 7fd805800640 1 -- 172.21.15.203:0/1009831425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8000d3d40 msgr2=0x7fd8000ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:59.477 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.476+0000 7fd805800640 1 --2- 172.21.15.203:0/1009831425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8000d3d40 0x7fd8000ce8f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8031000 tx=0x7fd7e80046c0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.477 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.476+0000 7fd805800640 1 -- 172.21.15.203:0/1009831425 shutdown_connections 2024-04-28T13:30:59.477 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.476+0000 7fd805800640 1 --2- 172.21.15.203:0/1009831425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd80015f320 0x7fd800161ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.477 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.476+0000 7fd805800640 1 --2- 172.21.15.203:0/1009831425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8000d3d40 0x7fd8000ce8f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.477 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.476+0000 7fd805800640 1 --2- 172.21.15.203:0/1009831425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8000d1ff0 0x7fd800160260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.477 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.476+0000 7fd805800640 1 -- 172.21.15.203:0/1009831425 >> 172.21.15.203:0/1009831425 conn(0x7fd8000d5010 msgr2=0x7fd8000cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:30:59.477 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.476+0000 7fd805800640 1 -- 172.21.15.203:0/1009831425 shutdown_connections 2024-04-28T13:30:59.478 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.476+0000 7fd805800640 1 -- 172.21.15.203:0/1009831425 wait complete. 2024-04-28T13:30:59.491 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 2201e71b-0237-4734-818e-a6a9725d1f80 5 2024-04-28T13:30:59.503 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:30:59.503+0000 7f3c2a23b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-28T13:30:59.662 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.661+0000 7fb805c00640 1 Processor -- start 2024-04-28T13:30:59.662 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.661+0000 7fb805c00640 1 -- start start 2024-04-28T13:30:59.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.663+0000 7fb805c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8000d3c10 0x7fb8000b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:59.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.663+0000 7fb805c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb8000b8370 0x7fb8000d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:59.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.663+0000 7fb805c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8000cfb80 con 0x7fb8000d3830 2024-04-28T13:30:59.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.663+0000 7fb805c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb8000d2ee0 con 0x7fb8000b8370 2024-04-28T13:30:59.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.663+0000 7fb805c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb8000d31c0 con 0x7fb8000d3c10 2024-04-28T13:30:59.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.663+0000 7fb7f7e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8000d3c10 0x7fb8000b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:59.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.663+0000 7fb7f7e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8000d3c10 0x7fb8000b7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:50732/0 (socket says 172.21.15.203:50732) 2024-04-28T13:30:59.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.663+0000 7fb7f7e00640 1 -- 172.21.15.203:0/1784372968 learned_addr learned my addr 172.21.15.203:0/1784372968 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-28T13:30:59.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.663+0000 7fb805200640 1 --2- 172.21.15.203:0/1784372968 >> v2:172.21.15.196:3300/0 conn(0x7fb8000b8370 0x7fb8000d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:59.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.664+0000 7fb7f7e00640 1 -- 172.21.15.203:0/1784372968 >> v2:172.21.15.196:3300/0 conn(0x7fb8000b8370 msgr2=0x7fb8000d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:30:59.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.664+0000 7fb7f7e00640 1 --2- 172.21.15.203:0/1784372968 >> v2:172.21.15.196:3300/0 conn(0x7fb8000b8370 0x7fb8000d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.664+0000 7fb7f7e00640 1 -- 172.21.15.203:0/1784372968 >> v1:172.21.15.196:6789/0 conn(0x7fb8000d3830 legacy=0x7fb8000d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:30:59.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.664+0000 7fb7f7e00640 1 -- 172.21.15.203:0/1784372968 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8000cfe00 con 0x7fb8000d3c10 2024-04-28T13:30:59.666 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.665+0000 7fb7f7e00640 1 --2- 172.21.15.203:0/1784372968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8000d3c10 0x7fb8000b7e30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009870 tx=0x7fb7e802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60fa28130ad2fa36 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:30:59.666 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.665+0000 7fb7ff400640 1 -- 172.21.15.203:0/1784372968 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8037070 con 0x7fb8000d3c10 2024-04-28T13:30:59.666 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.665+0000 7fb7ff400640 1 -- 172.21.15.203:0/1784372968 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e8032440 con 0x7fb8000d3c10 2024-04-28T13:30:59.667 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.665+0000 7fb7ff400640 1 -- 172.21.15.203:0/1784372968 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8032810 con 0x7fb8000d3c10 2024-04-28T13:30:59.667 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.666+0000 7fb805c00640 1 -- 172.21.15.203:0/1784372968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8000d3c10 msgr2=0x7fb8000b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:59.667 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.666+0000 7fb805c00640 1 --2- 172.21.15.203:0/1784372968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8000d3c10 0x7fb8000b7e30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009870 tx=0x7fb7e802a210 comp rx=0 tx=0).stop 2024-04-28T13:30:59.667 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.666+0000 7fb805c00640 1 -- 172.21.15.203:0/1784372968 shutdown_connections 2024-04-28T13:30:59.667 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.666+0000 7fb805c00640 1 --2- 172.21.15.203:0/1784372968 >> v2:172.21.15.196:3300/0 conn(0x7fb8000b8370 0x7fb8000d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.667 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.666+0000 7fb805c00640 1 --2- 172.21.15.203:0/1784372968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8000d3c10 0x7fb8000b7e30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.667 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.666+0000 7fb805c00640 1 -- 172.21.15.203:0/1784372968 >> 172.21.15.203:0/1784372968 conn(0x7fb8000d4fe0 msgr2=0x7fb8000d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:30:59.667 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.666+0000 7fb805c00640 1 -- 172.21.15.203:0/1784372968 shutdown_connections 2024-04-28T13:30:59.667 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.666+0000 7fb805c00640 1 -- 172.21.15.203:0/1784372968 wait complete. 2024-04-28T13:30:59.668 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.668+0000 7fb805c00640 1 Processor -- start 2024-04-28T13:30:59.668 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.668+0000 7fb805c00640 1 -- start start 2024-04-28T13:30:59.670 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb805c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8000b8370 0x7fb8000cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:59.670 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb805c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8000d3830 0x7fb8001f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:59.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb805c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8000d3c10 0x7fb800162360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:30:59.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb805c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb8000d2bb0 con 0x7fb8000b8370 2024-04-28T13:30:59.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb805c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb8000d5d90 con 0x7fb8000d3c10 2024-04-28T13:30:59.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb805c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb8000d0b40 con 0x7fb8000d3830 2024-04-28T13:30:59.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb7f7e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8000d3830 0x7fb8001f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:59.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb7f7e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8000d3830 0x7fb8001f85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:50738/0 (socket says 172.21.15.203:50738) 2024-04-28T13:30:59.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb7f7e00640 1 -- 172.21.15.203:0/2437247371 learned_addr learned my addr 172.21.15.203:0/2437247371 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-28T13:30:59.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb805200640 1 --2- 172.21.15.203:0/2437247371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8000d3c10 0x7fb800162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:59.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb7ffe00640 1 --2- 172.21.15.203:0/2437247371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8000b8370 0x7fb8000cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:30:59.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb7f7e00640 1 -- 172.21.15.203:0/2437247371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8000d3c10 msgr2=0x7fb800162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:59.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb7f7e00640 1 --2- 172.21.15.203:0/2437247371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8000d3c10 0x7fb800162360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb7f7e00640 1 -- 172.21.15.203:0/2437247371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8000b8370 msgr2=0x7fb8000cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:59.671 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb7f7e00640 1 --2- 172.21.15.203:0/2437247371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8000b8370 0x7fb8000cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.669+0000 7fb7f7e00640 1 -- 172.21.15.203:0/2437247371 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8001f9500 con 0x7fb8000d3830 2024-04-28T13:30:59.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.670+0000 7fb7f7e00640 1 --2- 172.21.15.203:0/2437247371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8000d3830 0x7fb8001f85f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009870 tx=0x7fb7e80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:30:59.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.670+0000 7fb7fcc00640 1 -- 172.21.15.203:0/2437247371 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8041020 con 0x7fb8000d3830 2024-04-28T13:30:59.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.670+0000 7fb7fcc00640 1 -- 172.21.15.203:0/2437247371 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e80306c0 con 0x7fb8000d3830 2024-04-28T13:30:59.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.670+0000 7fb7fcc00640 1 -- 172.21.15.203:0/2437247371 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8037040 con 0x7fb8000d3830 2024-04-28T13:30:59.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.670+0000 7fb805c00640 1 -- 172.21.15.203:0/2437247371 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8001f8f40 con 0x7fb8000d3830 2024-04-28T13:30:59.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.670+0000 7fb805c00640 1 -- 172.21.15.203:0/2437247371 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb80015fe90 con 0x7fb8000d3830 2024-04-28T13:30:59.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.670+0000 7fb7fcc00640 1 -- 172.21.15.203:0/2437247371 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb7e8044020 con 0x7fb8000d3830 2024-04-28T13:30:59.672 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.671+0000 7fb7fcc00640 1 -- 172.21.15.203:0/2437247371 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fb7e803dc90 con 0x7fb8000d3830 2024-04-28T13:30:59.673 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.671+0000 7fb805c00640 1 -- 172.21.15.203:0/2437247371 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb800050730 con 0x7fb8000d3830 2024-04-28T13:30:59.674 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.674+0000 7fb7fcc00640 1 -- 172.21.15.203:0/2437247371 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb7e804b030 con 0x7fb8000d3830 2024-04-28T13:30:59.752 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.752+0000 7fb805c00640 1 -- 172.21.15.203:0/2437247371 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2201e71b-0237-4734-818e-a6a9725d1f80", "id": 5} v 0) -- 0x7fb8000ba760 con 0x7fb8000d3830 2024-04-28T13:30:59.763 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.763+0000 7fb7fcc00640 1 -- 172.21.15.203:0/2437247371 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2201e71b-0237-4734-818e-a6a9725d1f80", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fb8000cfb80 con 0x7fb8000d3830 2024-04-28T13:30:59.763 INFO:teuthology.orchestra.run.smithi203.stdout:5 2024-04-28T13:30:59.764 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.764+0000 7fb805c00640 1 -- 172.21.15.203:0/2437247371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8000d3830 msgr2=0x7fb8001f85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:30:59.765 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.764+0000 7fb805c00640 1 --2- 172.21.15.203:0/2437247371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8000d3830 0x7fb8001f85f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009870 tx=0x7fb7e80046c0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.765 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.764+0000 7fb805c00640 1 -- 172.21.15.203:0/2437247371 shutdown_connections 2024-04-28T13:30:59.765 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.764+0000 7fb805c00640 1 --2- 172.21.15.203:0/2437247371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8000d3c10 0x7fb800162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.765 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.764+0000 7fb805c00640 1 --2- 172.21.15.203:0/2437247371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8000d3830 0x7fb8001f85f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.765 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.764+0000 7fb805c00640 1 --2- 172.21.15.203:0/2437247371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8000b8370 0x7fb8000cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:30:59.765 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.764+0000 7fb805c00640 1 -- 172.21.15.203:0/2437247371 >> 172.21.15.203:0/2437247371 conn(0x7fb8000d4fe0 msgr2=0x7fb8000d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:30:59.765 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.764+0000 7fb805c00640 1 -- 172.21.15.203:0/2437247371 shutdown_connections 2024-04-28T13:30:59.765 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:30:59.764+0000 7fb805c00640 1 -- 172.21.15.203:0/2437247371 wait complete. 2024-04-28T13:30:59.781 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new a0a083ff-9305-48ca-b1c2-5d5f6736a47f 6 2024-04-28T13:30:59.804 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:30:59.803+0000 7f192bfec040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-28T13:30:59.905 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:30:59.904+0000 7f192bfec040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-28T13:30:59.955 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:30:59.954+0000 7f192bfec040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-28T13:30:59.981 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:30:59.980+0000 7f3c2a23b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-28T13:31:00.006 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.005+0000 7efd9f200640 1 Processor -- start 2024-04-28T13:31:00.006 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.005+0000 7efd9f200640 1 -- start start 2024-04-28T13:31:00.008 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.007+0000 7efd9f200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efd980d3c10 0x7efd980b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:00.008 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.007+0000 7efd9f200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd980b8370 0x7efd980d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:00.008 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.007+0000 7efd9f200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd980cfb80 con 0x7efd980d3830 2024-04-28T13:31:00.008 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.007+0000 7efd9f200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7efd980d2ee0 con 0x7efd980d3c10 2024-04-28T13:31:00.008 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.007+0000 7efd9f200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efd980d31c0 con 0x7efd980b8370 2024-04-28T13:31:00.008 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.007+0000 7efd9e800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd980b8370 0x7efd980d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:00.008 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.007+0000 7efd9d400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efd980d3c10 0x7efd980b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:00.008 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.007+0000 7efd9e800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd980b8370 0x7efd980d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:50748/0 (socket says 172.21.15.203:50748) 2024-04-28T13:31:00.008 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.007+0000 7efd9e800640 1 -- 172.21.15.203:0/1620461235 learned_addr learned my addr 172.21.15.203:0/1620461235 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-28T13:31:00.009 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.008+0000 7efd9e800640 1 -- 172.21.15.203:0/1620461235 >> v2:172.21.15.203:3300/0 conn(0x7efd980d3c10 msgr2=0x7efd980b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:00.009 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.008+0000 7efd9e800640 1 --2- 172.21.15.203:0/1620461235 >> v2:172.21.15.203:3300/0 conn(0x7efd980d3c10 0x7efd980b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.009 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.008+0000 7efd9e800640 1 -- 172.21.15.203:0/1620461235 >> v1:172.21.15.196:6789/0 conn(0x7efd980d3830 legacy=0x7efd980d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:00.009 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.008+0000 7efd9e800640 1 -- 172.21.15.203:0/1620461235 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd980cfe00 con 0x7efd980b8370 2024-04-28T13:31:00.010 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.009+0000 7efd9e800640 1 --2- 172.21.15.203:0/1620461235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd980b8370 0x7efd980d0600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efd94009e50 tx=0x7efd9402a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a43415821e19a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:00.010 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.009+0000 7efd9ca00640 1 -- 172.21.15.203:0/1620461235 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94036070 con 0x7efd980b8370 2024-04-28T13:31:00.010 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.009+0000 7efd9ca00640 1 -- 172.21.15.203:0/1620461235 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd94031600 con 0x7efd980b8370 2024-04-28T13:31:00.010 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.009+0000 7efd9ca00640 1 -- 172.21.15.203:0/1620461235 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd940319d0 con 0x7efd980b8370 2024-04-28T13:31:00.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.010+0000 7efd9f200640 1 -- 172.21.15.203:0/1620461235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd980b8370 msgr2=0x7efd980d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:00.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.010+0000 7efd9f200640 1 --2- 172.21.15.203:0/1620461235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd980b8370 0x7efd980d0600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efd94009e50 tx=0x7efd9402a6c0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.010+0000 7efd9f200640 1 -- 172.21.15.203:0/1620461235 shutdown_connections 2024-04-28T13:31:00.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.010+0000 7efd9f200640 1 --2- 172.21.15.203:0/1620461235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd980b8370 0x7efd980d0600 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.010+0000 7efd9f200640 1 --2- 172.21.15.203:0/1620461235 >> v2:172.21.15.203:3300/0 conn(0x7efd980d3c10 0x7efd980b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.010+0000 7efd9f200640 1 -- 172.21.15.203:0/1620461235 >> 172.21.15.203:0/1620461235 conn(0x7efd980d4fe0 msgr2=0x7efd980d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:00.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.010+0000 7efd9f200640 1 -- 172.21.15.203:0/1620461235 shutdown_connections 2024-04-28T13:31:00.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.010+0000 7efd9f200640 1 -- 172.21.15.203:0/1620461235 wait complete. 2024-04-28T13:31:00.012 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.012+0000 7efd9f200640 1 Processor -- start 2024-04-28T13:31:00.012 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.012+0000 7efd9f200640 1 -- start start 2024-04-28T13:31:00.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9f200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd980b8370 0x7efd980cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:00.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9f200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd980d3830 0x7efd981f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:00.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9f200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd980d3c10 0x7efd9815ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:00.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9f200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd980d2bb0 con 0x7efd980d3830 2024-04-28T13:31:00.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9f200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efd980d5d90 con 0x7efd980d3c10 2024-04-28T13:31:00.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9f200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efd980d0b40 con 0x7efd980b8370 2024-04-28T13:31:00.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9de00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd980b8370 0x7efd980cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:00.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9de00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd980b8370 0x7efd980cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:50764/0 (socket says 172.21.15.203:50764) 2024-04-28T13:31:00.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9e800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd980d3c10 0x7efd9815ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:00.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9de00640 1 -- 172.21.15.203:0/2668584780 learned_addr learned my addr 172.21.15.203:0/2668584780 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-28T13:31:00.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9d400640 1 --2- 172.21.15.203:0/2668584780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd980d3830 0x7efd981f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:00.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9de00640 1 -- 172.21.15.203:0/2668584780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd980d3c10 msgr2=0x7efd9815ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:00.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9de00640 1 --2- 172.21.15.203:0/2668584780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd980d3c10 0x7efd9815ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.016 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9de00640 1 -- 172.21.15.203:0/2668584780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd980d3830 msgr2=0x7efd981f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:00.016 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9de00640 1 --2- 172.21.15.203:0/2668584780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd980d3830 0x7efd981f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.016 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9de00640 1 -- 172.21.15.203:0/2668584780 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd98158f50 con 0x7efd980b8370 2024-04-28T13:31:00.016 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.013+0000 7efd9e800640 1 --2- 172.21.15.203:0/2668584780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd980d3c10 0x7efd9815ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:31:00.016 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.014+0000 7efd9de00640 1 --2- 172.21.15.203:0/2668584780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd980b8370 0x7efd980cb390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c460 tx=0x7efd8c00c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:00.016 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.014+0000 7efd92000640 1 -- 172.21.15.203:0/2668584780 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c016020 con 0x7efd980b8370 2024-04-28T13:31:00.016 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.014+0000 7efd92000640 1 -- 172.21.15.203:0/2668584780 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd8c012870 con 0x7efd980b8370 2024-04-28T13:31:00.016 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.014+0000 7efd92000640 1 -- 172.21.15.203:0/2668584780 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c012c60 con 0x7efd980b8370 2024-04-28T13:31:00.016 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.014+0000 7efd9f200640 1 -- 172.21.15.203:0/2668584780 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd98159230 con 0x7efd980b8370 2024-04-28T13:31:00.016 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.014+0000 7efd9f200640 1 -- 172.21.15.203:0/2668584780 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efd981f5040 con 0x7efd980b8370 2024-04-28T13:31:00.016 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.014+0000 7efd92000640 1 -- 172.21.15.203:0/2668584780 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efd8c019020 con 0x7efd980b8370 2024-04-28T13:31:00.016 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.014+0000 7efd92000640 1 -- 172.21.15.203:0/2668584780 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7efd8c012e00 con 0x7efd980b8370 2024-04-28T13:31:00.017 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.015+0000 7efd9f200640 1 -- 172.21.15.203:0/2668584780 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd98050730 con 0x7efd980b8370 2024-04-28T13:31:00.018 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.018+0000 7efd92000640 1 -- 172.21.15.203:0/2668584780 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7efd8c020030 con 0x7efd980b8370 2024-04-28T13:31:00.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.096+0000 7efd9f200640 1 -- 172.21.15.203:0/2668584780 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a0a083ff-9305-48ca-b1c2-5d5f6736a47f", "id": 6} v 0) -- 0x7efd981f5210 con 0x7efd980b8370 2024-04-28T13:31:00.108 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.108+0000 7efd92000640 1 -- 172.21.15.203:0/2668584780 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a0a083ff-9305-48ca-b1c2-5d5f6736a47f", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7efd8c01b790 con 0x7efd980b8370 2024-04-28T13:31:00.108 INFO:teuthology.orchestra.run.smithi203.stdout:6 2024-04-28T13:31:00.110 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.109+0000 7efd9f200640 1 -- 172.21.15.203:0/2668584780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd980b8370 msgr2=0x7efd980cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:00.110 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.109+0000 7efd9f200640 1 --2- 172.21.15.203:0/2668584780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd980b8370 0x7efd980cb390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c460 tx=0x7efd8c00c920 comp rx=0 tx=0).stop 2024-04-28T13:31:00.110 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.109+0000 7efd9f200640 1 -- 172.21.15.203:0/2668584780 shutdown_connections 2024-04-28T13:31:00.110 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.109+0000 7efd9f200640 1 --2- 172.21.15.203:0/2668584780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd980d3c10 0x7efd9815ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.110 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.109+0000 7efd9f200640 1 --2- 172.21.15.203:0/2668584780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd980d3830 0x7efd981f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.110 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.109+0000 7efd9f200640 1 --2- 172.21.15.203:0/2668584780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd980b8370 0x7efd980cb390 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.110 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.109+0000 7efd9f200640 1 -- 172.21.15.203:0/2668584780 >> 172.21.15.203:0/2668584780 conn(0x7efd980d4fe0 msgr2=0x7efd9815f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:00.110 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.110+0000 7efd9f200640 1 -- 172.21.15.203:0/2668584780 shutdown_connections 2024-04-28T13:31:00.110 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.110+0000 7efd9f200640 1 -- 172.21.15.203:0/2668584780 wait complete. 2024-04-28T13:31:00.125 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new eed708bd-0a80-4194-ad72-435b034cf05d 7 2024-04-28T13:31:00.146 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:00.145+0000 7f192bfec040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-28T13:31:00.221 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:00.219+0000 7f192bfec040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-28T13:31:00.281 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:00.281+0000 7f192bfec040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-28T13:31:00.291 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:31:00.291+0000 7f3c2a23b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-28T13:31:00.299 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.299+0000 7f311da00640 1 Processor -- start 2024-04-28T13:31:00.299 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.299+0000 7f311da00640 1 -- start start 2024-04-28T13:31:00.302 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.301+0000 7f311da00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f311815d020 0x7f3118073dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:00.302 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.301+0000 7f311da00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f311806c9b0 0x7f31180cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:00.302 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.301+0000 7f311da00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31180d1310 con 0x7f31180cf0f0 2024-04-28T13:31:00.302 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.301+0000 7f311da00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f31180ce6a0 con 0x7f311806c9b0 2024-04-28T13:31:00.302 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.301+0000 7f311da00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f31180cec60 con 0x7f311815d020 2024-04-28T13:31:00.302 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.301+0000 7f3117e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f311815d020 0x7f3118073dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:00.302 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.301+0000 7f3117e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f311815d020 0x7f3118073dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50344/0 (socket says 172.21.15.203:50344) 2024-04-28T13:31:00.302 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.301+0000 7f3117e00640 1 -- 172.21.15.203:0/2278349075 learned_addr learned my addr 172.21.15.203:0/2278349075 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-28T13:31:00.303 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.301+0000 7f311d000640 1 --2- 172.21.15.203:0/2278349075 >> v2:172.21.15.196:3300/0 conn(0x7f311806c9b0 0x7f31180cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:00.303 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.301+0000 7f3117e00640 1 -- 172.21.15.203:0/2278349075 >> v2:172.21.15.196:3300/0 conn(0x7f311806c9b0 msgr2=0x7f31180cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:00.303 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.301+0000 7f3117e00640 1 --2- 172.21.15.203:0/2278349075 >> v2:172.21.15.196:3300/0 conn(0x7f311806c9b0 0x7f31180cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.303 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.302+0000 7f3117e00640 1 -- 172.21.15.203:0/2278349075 >> v1:172.21.15.196:6789/0 conn(0x7f31180cf0f0 legacy=0x7f311806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:00.303 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.302+0000 7f3117e00640 1 -- 172.21.15.203:0/2278349075 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31180d15f0 con 0x7f311815d020 2024-04-28T13:31:00.304 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.303+0000 7f3117e00640 1 --2- 172.21.15.203:0/2278349075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311815d020 0x7f3118073dc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3118070ed0 tx=0x7f311002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b03cdd1816d797c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:00.304 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.303+0000 7f3116a00640 1 -- 172.21.15.203:0/2278349075 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110036070 con 0x7f311815d020 2024-04-28T13:31:00.304 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.303+0000 7f3116a00640 1 -- 172.21.15.203:0/2278349075 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3110031600 con 0x7f311815d020 2024-04-28T13:31:00.304 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.303+0000 7f3116a00640 1 -- 172.21.15.203:0/2278349075 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31100319d0 con 0x7f311815d020 2024-04-28T13:31:00.304 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.304+0000 7f311da00640 1 -- 172.21.15.203:0/2278349075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311815d020 msgr2=0x7f3118073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:00.304 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.304+0000 7f311da00640 1 --2- 172.21.15.203:0/2278349075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311815d020 0x7f3118073dc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3118070ed0 tx=0x7f311002a5d0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.304 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.304+0000 7f311da00640 1 -- 172.21.15.203:0/2278349075 shutdown_connections 2024-04-28T13:31:00.305 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.304+0000 7f311da00640 1 --2- 172.21.15.203:0/2278349075 >> v2:172.21.15.196:3300/0 conn(0x7f311806c9b0 0x7f31180cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.305 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.304+0000 7f311da00640 1 --2- 172.21.15.203:0/2278349075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311815d020 0x7f3118073dc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.305 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.304+0000 7f311da00640 1 -- 172.21.15.203:0/2278349075 >> 172.21.15.203:0/2278349075 conn(0x7f31180b3820 msgr2=0x7f31180b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:00.305 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.304+0000 7f311da00640 1 -- 172.21.15.203:0/2278349075 shutdown_connections 2024-04-28T13:31:00.305 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.304+0000 7f311da00640 1 -- 172.21.15.203:0/2278349075 wait complete. 2024-04-28T13:31:00.306 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.305+0000 7f311da00640 1 Processor -- start 2024-04-28T13:31:00.306 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.306+0000 7f311da00640 1 -- start start 2024-04-28T13:31:00.308 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.306+0000 7f311da00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311806c9b0 0x7f3118155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:00.308 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.306+0000 7f311da00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31180cf0f0 0x7f31180c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:00.308 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.306+0000 7f311da00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f311815d020 0x7f31180ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:00.308 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.306+0000 7f311da00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31180ce8f0 con 0x7f31180cf0f0 2024-04-28T13:31:00.308 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.307+0000 7f311da00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f311815d400 con 0x7f311806c9b0 2024-04-28T13:31:00.309 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.307+0000 7f311da00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3118160ac0 con 0x7f311815d020 2024-04-28T13:31:00.309 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.307+0000 7f3117e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311806c9b0 0x7f3118155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:00.309 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.307+0000 7f3117e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311806c9b0 0x7f3118155fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:50358/0 (socket says 172.21.15.203:50358) 2024-04-28T13:31:00.309 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.307+0000 7f3117e00640 1 -- 172.21.15.203:0/2908764432 learned_addr learned my addr 172.21.15.203:0/2908764432 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-28T13:31:00.309 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.307+0000 7f311d000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f311815d020 0x7f31180ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:00.309 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.307+0000 7f3117400640 1 --2- 172.21.15.203:0/2908764432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31180cf0f0 0x7f31180c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:00.309 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.307+0000 7f3117e00640 1 -- 172.21.15.203:0/2908764432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f311815d020 msgr2=0x7f31180ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:00.309 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.307+0000 7f3117e00640 1 --2- 172.21.15.203:0/2908764432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f311815d020 0x7f31180ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.309 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.307+0000 7f3117e00640 1 -- 172.21.15.203:0/2908764432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31180cf0f0 msgr2=0x7f31180c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:00.309 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.307+0000 7f3117e00640 1 --2- 172.21.15.203:0/2908764432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31180cf0f0 0x7f31180c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.309 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.307+0000 7f3117e00640 1 -- 172.21.15.203:0/2908764432 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31181fcd70 con 0x7f311806c9b0 2024-04-28T13:31:00.309 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.307+0000 7f3117400640 1 --2- 172.21.15.203:0/2908764432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31180cf0f0 0x7f31180c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:00.309 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.307+0000 7f311d000640 1 --2- 172.21.15.203:0/2908764432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f311815d020 0x7f31180ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:31:00.310 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.308+0000 7f3117e00640 1 --2- 172.21.15.203:0/2908764432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311806c9b0 0x7f3118155fc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3110000f80 tx=0x7f3110004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:00.310 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.308+0000 7f310fe00640 1 -- 172.21.15.203:0/2908764432 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110041020 con 0x7f311806c9b0 2024-04-28T13:31:00.310 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.308+0000 7f310fe00640 1 -- 172.21.15.203:0/2908764432 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3110007820 con 0x7f311806c9b0 2024-04-28T13:31:00.310 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.308+0000 7f310fe00640 1 -- 172.21.15.203:0/2908764432 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110036040 con 0x7f311806c9b0 2024-04-28T13:31:00.310 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.308+0000 7f311da00640 1 -- 172.21.15.203:0/2908764432 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31181fd050 con 0x7f311806c9b0 2024-04-28T13:31:00.310 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.308+0000 7f311da00640 1 -- 172.21.15.203:0/2908764432 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31181fd610 con 0x7f311806c9b0 2024-04-28T13:31:00.310 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.308+0000 7f310fe00640 1 -- 172.21.15.203:0/2908764432 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3110044020 con 0x7f311806c9b0 2024-04-28T13:31:00.311 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.308+0000 7f310fe00640 1 -- 172.21.15.203:0/2908764432 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f3110007c10 con 0x7f311806c9b0 2024-04-28T13:31:00.311 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.309+0000 7f311da00640 1 -- 172.21.15.203:0/2908764432 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3118050730 con 0x7f311806c9b0 2024-04-28T13:31:00.311 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.311+0000 7f310fe00640 1 -- 172.21.15.203:0/2908764432 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f311004b030 con 0x7f311806c9b0 2024-04-28T13:31:00.355 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:31:00.355+0000 7f3c2a23b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-28T13:31:00.391 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.391+0000 7f311da00640 1 -- 172.21.15.203:0/2908764432 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "eed708bd-0a80-4194-ad72-435b034cf05d", "id": 7} v 0) -- 0x7f3118073690 con 0x7f311806c9b0 2024-04-28T13:31:00.401 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.401+0000 7f310fe00640 1 -- 172.21.15.203:0/2908764432 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "eed708bd-0a80-4194-ad72-435b034cf05d", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f311003c7e0 con 0x7f311806c9b0 2024-04-28T13:31:00.401 INFO:teuthology.orchestra.run.smithi203.stdout:7 2024-04-28T13:31:00.403 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.402+0000 7f311da00640 1 -- 172.21.15.203:0/2908764432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311806c9b0 msgr2=0x7f3118155fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:00.403 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.402+0000 7f311da00640 1 --2- 172.21.15.203:0/2908764432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311806c9b0 0x7f3118155fc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3110000f80 tx=0x7f3110004a40 comp rx=0 tx=0).stop 2024-04-28T13:31:00.403 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.402+0000 7f311da00640 1 -- 172.21.15.203:0/2908764432 shutdown_connections 2024-04-28T13:31:00.403 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.402+0000 7f311da00640 1 --2- 172.21.15.203:0/2908764432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f311815d020 0x7f31180ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.403 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.402+0000 7f311da00640 1 --2- 172.21.15.203:0/2908764432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31180cf0f0 0x7f31180c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.403 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.402+0000 7f311da00640 1 --2- 172.21.15.203:0/2908764432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311806c9b0 0x7f3118155fc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.403 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.402+0000 7f311da00640 1 -- 172.21.15.203:0/2908764432 >> 172.21.15.203:0/2908764432 conn(0x7f31180b3820 msgr2=0x7f31180b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:00.403 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.402+0000 7f311da00640 1 -- 172.21.15.203:0/2908764432 shutdown_connections 2024-04-28T13:31:00.404 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:00.403+0000 7f311da00640 1 -- 172.21.15.203:0/2908764432 wait complete. 2024-04-28T13:31:00.409 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:00.409+0000 7f192bfec040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-28T13:31:00.420 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-28T13:31:00.420 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-28T13:31:00.427 INFO:tasks.ceph.osd.0:Started 2024-04-28T13:31:00.427 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-28T13:31:00.427 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-28T13:31:00.436 INFO:tasks.ceph.osd.1:Started 2024-04-28T13:31:00.437 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-28T13:31:00.437 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-28T13:31:00.442 INFO:tasks.ceph.osd.2:Started 2024-04-28T13:31:00.442 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-28T13:31:00.442 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-28T13:31:00.446 INFO:tasks.ceph.osd.3:Started 2024-04-28T13:31:00.447 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-28T13:31:00.447 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-28T13:31:00.453 INFO:tasks.ceph.osd.4:Started 2024-04-28T13:31:00.454 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-28T13:31:00.454 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-28T13:31:00.460 INFO:tasks.ceph.osd.5:Started 2024-04-28T13:31:00.460 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-28T13:31:00.460 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-28T13:31:00.464 INFO:tasks.ceph.osd.6:Started 2024-04-28T13:31:00.464 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-28T13:31:00.464 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-28T13:31:00.470 INFO:tasks.ceph.osd.7:Started 2024-04-28T13:31:00.470 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:00.501 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:31:00.500+0000 7f3c2a23b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-28T13:31:00.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.677+0000 7feb5ee00640 1 Processor -- start 2024-04-28T13:31:00.678 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.678+0000 7feb5ee00640 1 -- start start 2024-04-28T13:31:00.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.679+0000 7feb5ee00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feb5815f4d0 0x7feb580b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:00.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.679+0000 7feb5ee00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb580b2550 con 0x7feb580d2e20 2024-04-28T13:31:00.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.679+0000 7feb5ee00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb580b2830 con 0x7feb580d5810 2024-04-28T13:31:00.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.679+0000 7feb5ee00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7feb580b6180 con 0x7feb5815f4d0 2024-04-28T13:31:00.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.679+0000 7feb57400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7feb580d2e20 0x7feb580d5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:37506/0 (socket says 172.21.15.196:37506) 2024-04-28T13:31:00.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.679+0000 7feb57400640 1 -- 172.21.15.196:0/548063682 learned_addr learned my addr 172.21.15.196:0/548063682 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:00.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.679+0000 7feb57e00640 1 --2- 172.21.15.196:0/548063682 >> v2:172.21.15.203:3300/0 conn(0x7feb5815f4d0 0x7feb580b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:00.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.680+0000 7feb56a00640 1 -- 172.21.15.196:0/548063682 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1756003509 0 0) 0x7feb580b2550 con 0x7feb580d2e20 2024-04-28T13:31:00.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.680+0000 7feb57e00640 1 -- 172.21.15.196:0/548063682 >> v1:172.21.15.203:6789/0 conn(0x7feb580d5810 legacy=0x7feb580b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:00.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.680+0000 7feb57e00640 1 -- 172.21.15.196:0/548063682 >> v1:172.21.15.196:6789/0 conn(0x7feb580d2e20 legacy=0x7feb580d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:00.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.680+0000 7feb57e00640 1 -- 172.21.15.196:0/548063682 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb580b2b10 con 0x7feb5815f4d0 2024-04-28T13:31:00.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.681+0000 7feb57e00640 1 --2- 172.21.15.196:0/548063682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb5815f4d0 0x7feb580b1ee0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feb480097a0 tx=0x7feb4802a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aed23ed21f1559d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:00.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.681+0000 7feb56a00640 1 -- 172.21.15.196:0/548063682 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48037070 con 0x7feb5815f4d0 2024-04-28T13:31:00.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.681+0000 7feb56a00640 1 -- 172.21.15.196:0/548063682 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb4802adc0 con 0x7feb5815f4d0 2024-04-28T13:31:00.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.681+0000 7feb56a00640 1 -- 172.21.15.196:0/548063682 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48032660 con 0x7feb5815f4d0 2024-04-28T13:31:00.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.682+0000 7feb5ee00640 1 -- 172.21.15.196:0/548063682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb5815f4d0 msgr2=0x7feb580b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:00.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.682+0000 7feb5ee00640 1 --2- 172.21.15.196:0/548063682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb5815f4d0 0x7feb580b1ee0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feb480097a0 tx=0x7feb4802a0b0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.682+0000 7feb5ee00640 1 -- 172.21.15.196:0/548063682 shutdown_connections 2024-04-28T13:31:00.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.682+0000 7feb5ee00640 1 --2- 172.21.15.196:0/548063682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb5815f4d0 0x7feb580b1ee0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.682+0000 7feb5ee00640 1 -- 172.21.15.196:0/548063682 >> 172.21.15.196:0/548063682 conn(0x7feb5808ebf0 msgr2=0x7feb5809e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:00.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.682+0000 7feb5ee00640 1 -- 172.21.15.196:0/548063682 shutdown_connections 2024-04-28T13:31:00.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.682+0000 7feb5ee00640 1 -- 172.21.15.196:0/548063682 wait complete. 2024-04-28T13:31:00.683 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.684+0000 7feb5ee00640 1 Processor -- start 2024-04-28T13:31:00.683 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.684+0000 7feb5ee00640 1 -- start start 2024-04-28T13:31:00.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb5ee00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb580d2e20 0x7feb58150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:00.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb5ee00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb580d5810 0x7feb580cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:00.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb5ee00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb5815f4d0 0x7feb580b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:00.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb5ee00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feb580b3d00 con 0x7feb580d5810 2024-04-28T13:31:00.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb5ee00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feb580b30d0 con 0x7feb580d2e20 2024-04-28T13:31:00.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb5ee00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feb580b3e80 con 0x7feb5815f4d0 2024-04-28T13:31:00.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb57400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb580d5810 0x7feb580cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:00.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb57400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb580d5810 0x7feb580cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:53344/0 (socket says 172.21.15.196:53344) 2024-04-28T13:31:00.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb57400640 1 -- 172.21.15.196:0/425496068 learned_addr learned my addr 172.21.15.196:0/425496068 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:00.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb57e00640 1 --2- 172.21.15.196:0/425496068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb580d2e20 0x7feb58150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:00.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb5ce00640 1 --2- 172.21.15.196:0/425496068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb5815f4d0 0x7feb580b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:00.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb57400640 1 -- 172.21.15.196:0/425496068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb5815f4d0 msgr2=0x7feb580b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:00.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb57400640 1 --2- 172.21.15.196:0/425496068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb5815f4d0 0x7feb580b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb57400640 1 -- 172.21.15.196:0/425496068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb580d2e20 msgr2=0x7feb58150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:00.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb57400640 1 --2- 172.21.15.196:0/425496068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb580d2e20 0x7feb58150570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.685+0000 7feb57400640 1 -- 172.21.15.196:0/425496068 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb580ca5a0 con 0x7feb580d5810 2024-04-28T13:31:00.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.686+0000 7feb57400640 1 --2- 172.21.15.196:0/425496068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb580d5810 0x7feb580cb640 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7feb4400c4d0 tx=0x7feb4400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:00.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.686+0000 7feb4fe00640 1 -- 172.21.15.196:0/425496068 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44018070 con 0x7feb580d5810 2024-04-28T13:31:00.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.686+0000 7feb4fe00640 1 -- 172.21.15.196:0/425496068 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb44007500 con 0x7feb580d5810 2024-04-28T13:31:00.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.686+0000 7feb4fe00640 1 -- 172.21.15.196:0/425496068 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44004a20 con 0x7feb580d5810 2024-04-28T13:31:00.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.687+0000 7feb5ee00640 1 -- 172.21.15.196:0/425496068 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb580ca880 con 0x7feb580d5810 2024-04-28T13:31:00.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.687+0000 7feb5ee00640 1 -- 172.21.15.196:0/425496068 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb580cae40 con 0x7feb580d5810 2024-04-28T13:31:00.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.687+0000 7feb4fe00640 1 -- 172.21.15.196:0/425496068 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feb44004bc0 con 0x7feb580d5810 2024-04-28T13:31:00.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.687+0000 7feb4fe00640 1 -- 172.21.15.196:0/425496068 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7feb4401ba30 con 0x7feb580d5810 2024-04-28T13:31:00.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.688+0000 7feb5ee00640 1 -- 172.21.15.196:0/425496068 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb5806a290 con 0x7feb580d5810 2024-04-28T13:31:00.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.690+0000 7feb4fe00640 1 -- 172.21.15.196:0/425496068 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7feb4401bc80 con 0x7feb580d5810 2024-04-28T13:31:00.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.755+0000 7feb5ee00640 1 -- 172.21.15.196:0/425496068 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb580b5cb0 con 0x7feb580d5810 2024-04-28T13:31:00.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.756+0000 7feb4fe00640 1 -- 172.21.15.196:0/425496068 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7feb4401b360 con 0x7feb580d5810 2024-04-28T13:31:00.756 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:00.756 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":10,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:00.393424+0000","last_up_change":"0.000000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:00.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.757+0000 7feb5ee00640 1 -- 172.21.15.196:0/425496068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb580d5810 msgr2=0x7feb580cb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:00.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.757+0000 7feb5ee00640 1 --2- 172.21.15.196:0/425496068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb580d5810 0x7feb580cb640 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7feb4400c4d0 tx=0x7feb4400c990 comp rx=0 tx=0).stop 2024-04-28T13:31:00.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.757+0000 7feb5ee00640 1 -- 172.21.15.196:0/425496068 shutdown_connections 2024-04-28T13:31:00.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.757+0000 7feb5ee00640 1 --2- 172.21.15.196:0/425496068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb5815f4d0 0x7feb580b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.757+0000 7feb5ee00640 1 --2- 172.21.15.196:0/425496068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb580d5810 0x7feb580cb640 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.757+0000 7feb5ee00640 1 --2- 172.21.15.196:0/425496068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb580d2e20 0x7feb58150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:00.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.757+0000 7feb5ee00640 1 -- 172.21.15.196:0/425496068 >> 172.21.15.196:0/425496068 conn(0x7feb5808ebf0 msgr2=0x7feb580d5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:00.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.757+0000 7feb5ee00640 1 -- 172.21.15.196:0/425496068 shutdown_connections 2024-04-28T13:31:00.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:00.757+0000 7feb5ee00640 1 -- 172.21.15.196:0/425496068 wait complete. 2024-04-28T13:31:00.774 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-28T13:31:00.774 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-28T13:31:00.876 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:31:00.875+0000 7f3c2a23b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-28T13:31:00.930 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:00.930+0000 7f192bfec040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-28T13:31:00.933 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:31:00.933+0000 7f3c2a23b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-28T13:31:01.114 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:31:01.114+0000 7f3c2a23b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-28T13:31:01.149 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-28T13:31:01.149+0000 7f731a116640 -1 Falling back to public interface 2024-04-28T13:31:01.150 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-28T13:31:01.149+0000 7f051a618640 -1 Falling back to public interface 2024-04-28T13:31:01.150 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-28T13:31:01.149+0000 7fe815618640 -1 Falling back to public interface 2024-04-28T13:31:01.160 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-28T13:31:01.160+0000 7f4327e0c640 -1 Falling back to public interface 2024-04-28T13:31:01.161 INFO:tasks.ceph.osd.0.smithi196.stderr:2024-04-28T13:31:01.161+0000 7efc122d9640 -1 Falling back to public interface 2024-04-28T13:31:01.163 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-28T13:31:01.164+0000 7fcd84dad640 -1 Falling back to public interface 2024-04-28T13:31:01.171 INFO:tasks.ceph.osd.3.smithi196.stderr:2024-04-28T13:31:01.171+0000 7f3f5ca43640 -1 Falling back to public interface 2024-04-28T13:31:01.175 INFO:tasks.ceph.osd.1.smithi196.stderr:2024-04-28T13:31:01.176+0000 7f2512e2c640 -1 Falling back to public interface 2024-04-28T13:31:01.235 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:01.235+0000 7f192bfec040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-28T13:31:01.288 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:31:01.288+0000 7f3c2a23b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-28T13:31:01.296 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:01.296+0000 7f192bfec040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-28T13:31:01.346 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:31:01.345+0000 7f3c2a23b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-28T13:31:01.426 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:31:01.425+0000 7f3c2a23b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-28T13:31:01.427 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:01.427+0000 7f192bfec040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-28T13:31:01.481 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:31:01.480+0000 7f3c2a23b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-28T13:31:01.579 DEBUG:teuthology.orchestra.run.smithi196:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:01.737 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.737+0000 7ff533000640 1 Processor -- start 2024-04-28T13:31:01.737 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.737+0000 7ff533000640 1 -- start start 2024-04-28T13:31:01.739 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.738+0000 7ff533000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff52c15f190 0x7ff52c0d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:01.739 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.738+0000 7ff533000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff52c0d0a70 0x7ff52c0d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:01.739 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.738+0000 7ff533000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff52c15f570 con 0x7ff52c0d1d00 2024-04-28T13:31:01.739 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.738+0000 7ff533000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff52c0b6820 con 0x7ff52c15f190 2024-04-28T13:31:01.739 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.738+0000 7ff533000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff52c0d2c40 con 0x7ff52c0d0a70 2024-04-28T13:31:01.739 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.739+0000 7ff531c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff52c15f190 0x7ff52c0d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:01.739 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.739+0000 7ff531c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff52c15f190 0x7ff52c0d0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:53348/0 (socket says 172.21.15.196:53348) 2024-04-28T13:31:01.739 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.739+0000 7ff531c00640 1 -- 172.21.15.196:0/3924086658 learned_addr learned my addr 172.21.15.196:0/3924086658 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:01.739 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.739+0000 7ff531200640 1 --2- 172.21.15.196:0/3924086658 >> v2:172.21.15.203:3300/0 conn(0x7ff52c0d0a70 0x7ff52c0d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:01.740 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.739+0000 7ff531c00640 1 -- 172.21.15.196:0/3924086658 >> v2:172.21.15.203:3300/0 conn(0x7ff52c0d0a70 msgr2=0x7ff52c0d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:01.740 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.739+0000 7ff531c00640 1 --2- 172.21.15.196:0/3924086658 >> v2:172.21.15.203:3300/0 conn(0x7ff52c0d0a70 0x7ff52c0d17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:01.740 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.739+0000 7ff531c00640 1 -- 172.21.15.196:0/3924086658 >> v1:172.21.15.196:6789/0 conn(0x7ff52c0d1d00 legacy=0x7ff52c162c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:01.740 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.739+0000 7ff531c00640 1 -- 172.21.15.196:0/3924086658 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c0d2f20 con 0x7ff52c15f190 2024-04-28T13:31:01.740 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.740+0000 7ff531c00640 1 --2- 172.21.15.196:0/3924086658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff52c15f190 0x7ff52c0d0530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0d4e50 tx=0x7ff51c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d4fa4a8ba84755 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:01.741 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.741+0000 7ff52be00640 1 -- 172.21.15.196:0/3924086658 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c037070 con 0x7ff52c15f190 2024-04-28T13:31:01.741 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.741+0000 7ff52be00640 1 -- 172.21.15.196:0/3924086658 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff51c032440 con 0x7ff52c15f190 2024-04-28T13:31:01.741 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.741+0000 7ff52be00640 1 -- 172.21.15.196:0/3924086658 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c032810 con 0x7ff52c15f190 2024-04-28T13:31:01.742 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.742+0000 7ff533000640 1 -- 172.21.15.196:0/3924086658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff52c15f190 msgr2=0x7ff52c0d0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:01.742 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.742+0000 7ff533000640 1 --2- 172.21.15.196:0/3924086658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff52c15f190 0x7ff52c0d0530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0d4e50 tx=0x7ff51c02a270 comp rx=0 tx=0).stop 2024-04-28T13:31:01.742 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.742+0000 7ff533000640 1 -- 172.21.15.196:0/3924086658 shutdown_connections 2024-04-28T13:31:01.742 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.742+0000 7ff533000640 1 --2- 172.21.15.196:0/3924086658 >> v2:172.21.15.203:3300/0 conn(0x7ff52c0d0a70 0x7ff52c0d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:01.742 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.742+0000 7ff533000640 1 --2- 172.21.15.196:0/3924086658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff52c15f190 0x7ff52c0d0530 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:01.742 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.742+0000 7ff533000640 1 -- 172.21.15.196:0/3924086658 >> 172.21.15.196:0/3924086658 conn(0x7ff52c0d4ef0 msgr2=0x7ff52c0d5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:01.742 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.742+0000 7ff533000640 1 -- 172.21.15.196:0/3924086658 shutdown_connections 2024-04-28T13:31:01.742 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.742+0000 7ff533000640 1 -- 172.21.15.196:0/3924086658 wait complete. 2024-04-28T13:31:01.743 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.744+0000 7ff533000640 1 Processor -- start 2024-04-28T13:31:01.743 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.744+0000 7ff533000640 1 -- start start 2024-04-28T13:31:01.745 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff533000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff52c0d0a70 0x7ff52c1575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:01.745 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff533000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff52c0d1d00 0x7ff52c1f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:01.745 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff533000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff52c15f190 0x7ff52c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:01.745 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff533000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c0d37c0 con 0x7ff52c0d1d00 2024-04-28T13:31:01.745 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff533000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c0d34e0 con 0x7ff52c15f190 2024-04-28T13:31:01.745 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff533000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff52c0d3ce0 con 0x7ff52c0d0a70 2024-04-28T13:31:01.746 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff531200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff52c0d1d00 0x7ff52c1f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:01.746 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff531200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff52c0d1d00 0x7ff52c1f8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:53352/0 (socket says 172.21.15.196:53352) 2024-04-28T13:31:01.746 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff531200640 1 -- 172.21.15.196:0/2684534760 learned_addr learned my addr 172.21.15.196:0/2684534760 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:01.746 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff531c00640 1 --2- 172.21.15.196:0/2684534760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff52c0d0a70 0x7ff52c1575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:01.746 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff532600640 1 --2- 172.21.15.196:0/2684534760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff52c15f190 0x7ff52c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:01.746 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff531200640 1 -- 172.21.15.196:0/2684534760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff52c0d0a70 msgr2=0x7ff52c1575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:01.746 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff531200640 1 --2- 172.21.15.196:0/2684534760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff52c0d0a70 0x7ff52c1575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:01.746 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff531200640 1 -- 172.21.15.196:0/2684534760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff52c15f190 msgr2=0x7ff52c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:01.746 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff531200640 1 --2- 172.21.15.196:0/2684534760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff52c15f190 0x7ff52c0cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:01.746 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff531200640 1 -- 172.21.15.196:0/2684534760 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c15a340 con 0x7ff52c0d1d00 2024-04-28T13:31:01.746 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff532600640 1 --2- 172.21.15.196:0/2684534760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff52c15f190 0x7ff52c0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:31:01.746 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.745+0000 7ff531200640 1 --2- 172.21.15.196:0/2684534760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff52c0d1d00 0x7ff52c1f8eb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff51800e9e0 tx=0x7ff51800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:01.747 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.746+0000 7ff529600640 1 -- 172.21.15.196:0/2684534760 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518017070 con 0x7ff52c0d1d00 2024-04-28T13:31:01.747 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.746+0000 7ff529600640 1 -- 172.21.15.196:0/2684534760 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff51800c9d0 con 0x7ff52c0d1d00 2024-04-28T13:31:01.747 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.746+0000 7ff533000640 1 -- 172.21.15.196:0/2684534760 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52c15a620 con 0x7ff52c0d1d00 2024-04-28T13:31:01.747 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.746+0000 7ff529600640 1 -- 172.21.15.196:0/2684534760 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51800cdc0 con 0x7ff52c0d1d00 2024-04-28T13:31:01.747 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.746+0000 7ff533000640 1 -- 172.21.15.196:0/2684534760 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff52c0cd1b0 con 0x7ff52c0d1d00 2024-04-28T13:31:01.748 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.746+0000 7ff529600640 1 -- 172.21.15.196:0/2684534760 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff51801a070 con 0x7ff52c0d1d00 2024-04-28T13:31:01.748 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.746+0000 7ff529600640 1 -- 172.21.15.196:0/2684534760 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7ff51801cc60 con 0x7ff52c0d1d00 2024-04-28T13:31:01.748 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.747+0000 7ff533000640 1 -- 172.21.15.196:0/2684534760 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff52c076870 con 0x7ff52c0d1d00 2024-04-28T13:31:01.749 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.749+0000 7ff529600640 1 -- 172.21.15.196:0/2684534760 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff51801c590 con 0x7ff52c0d1d00 2024-04-28T13:31:01.773 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:01.773+0000 7f192bfec040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-28T13:31:01.814 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.814+0000 7ff533000640 1 -- 172.21.15.196:0/2684534760 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff52c08f030 con 0x7ff52c0d1d00 2024-04-28T13:31:01.815 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.816+0000 7ff529600640 1 -- 172.21.15.196:0/2684534760 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7ff51801c830 con 0x7ff52c0d1d00 2024-04-28T13:31:01.816 INFO:teuthology.misc.health.smithi196.stdout: 2024-04-28T13:31:01.816 INFO:teuthology.misc.health.smithi196.stdout:{"epoch":10,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:00.393424+0000","last_up_change":"0.000000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:01.817 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.817+0000 7ff533000640 1 -- 172.21.15.196:0/2684534760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff52c0d1d00 msgr2=0x7ff52c1f8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:01.817 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.817+0000 7ff533000640 1 --2- 172.21.15.196:0/2684534760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff52c0d1d00 0x7ff52c1f8eb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff51800e9e0 tx=0x7ff51800eea0 comp rx=0 tx=0).stop 2024-04-28T13:31:01.817 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.817+0000 7ff533000640 1 -- 172.21.15.196:0/2684534760 shutdown_connections 2024-04-28T13:31:01.817 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.817+0000 7ff533000640 1 --2- 172.21.15.196:0/2684534760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff52c15f190 0x7ff52c0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:01.817 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.817+0000 7ff533000640 1 --2- 172.21.15.196:0/2684534760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff52c0d1d00 0x7ff52c1f8eb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:01.817 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.817+0000 7ff533000640 1 --2- 172.21.15.196:0/2684534760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff52c0d0a70 0x7ff52c1575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:01.817 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.817+0000 7ff533000640 1 -- 172.21.15.196:0/2684534760 >> 172.21.15.196:0/2684534760 conn(0x7ff52c0d4ef0 msgr2=0x7ff52c0b9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:01.817 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.817+0000 7ff533000640 1 -- 172.21.15.196:0/2684534760 shutdown_connections 2024-04-28T13:31:01.817 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:01.817+0000 7ff533000640 1 -- 172.21.15.196:0/2684534760 wait complete. 2024-04-28T13:31:01.829 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-28T13:31:01.832 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:01.833+0000 7f192bfec040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-28T13:31:02.015 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:02.015+0000 7f192bfec040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-28T13:31:02.019 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:31:02.017+0000 7f3c2a23b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-28T13:31:02.149 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:31:02.148+0000 7f3c2a23b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-28T13:31:02.186 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:02.186+0000 7f192bfec040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-28T13:31:02.247 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:02.246+0000 7f192bfec040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-28T13:31:02.330 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:02.330+0000 7f192bfec040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-28T13:31:02.383 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:02.383+0000 7f192bfec040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-28T13:31:02.924 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:02.924+0000 7f192bfec040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-28T13:31:03.052 INFO:tasks.ceph.mgr.y.smithi196.stderr:2024-04-28T13:31:03.052+0000 7f192bfec040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-28T13:31:03.193 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:31:03.193+0000 7f3bca800640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-28T13:31:03.196 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:31:03.195+0000 7f3bca800640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-28T13:31:04.534 INFO:tasks.ceph.osd.1.smithi196.stderr:2024-04-28T13:31:04.534+0000 7f2512e2c640 -1 osd.1 0 log_to_monitors true 2024-04-28T13:31:04.536 INFO:tasks.ceph.osd.3.smithi196.stderr:2024-04-28T13:31:04.536+0000 7f3f5ca43640 -1 osd.3 0 log_to_monitors true 2024-04-28T13:31:04.772 INFO:tasks.ceph.osd.0.smithi196.stderr:2024-04-28T13:31:04.772+0000 7efc122d9640 -1 osd.0 0 log_to_monitors true 2024-04-28T13:31:04.779 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-28T13:31:04.779+0000 7fcd84dad640 -1 osd.2 0 log_to_monitors true 2024-04-28T13:31:05.012 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-28T13:31:05.012+0000 7f051a618640 -1 osd.7 0 log_to_monitors true 2024-04-28T13:31:05.032 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-28T13:31:05.032+0000 7f731a116640 -1 osd.4 0 log_to_monitors true 2024-04-28T13:31:05.040 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-28T13:31:05.039+0000 7fe815618640 -1 osd.6 0 log_to_monitors true 2024-04-28T13:31:05.043 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-28T13:31:05.043+0000 7f4327e0c640 -1 osd.5 0 log_to_monitors true 2024-04-28T13:31:08.383 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-28T13:31:08.383+0000 7f7315a00640 -1 osd.4 0 waiting for initial osdmap 2024-04-28T13:31:08.393 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-28T13:31:08.392+0000 7f730b000640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T13:31:08.438 INFO:tasks.ceph.osd.3.smithi196.stderr:2024-04-28T13:31:08.438+0000 7f3f57800640 -1 osd.3 0 waiting for initial osdmap 2024-04-28T13:31:08.445 INFO:tasks.ceph.osd.3.smithi196.stderr:2024-04-28T13:31:08.444+0000 7f3f4d800640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T13:31:08.463 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-28T13:31:08.463+0000 7fe810a00640 -1 osd.6 0 waiting for initial osdmap 2024-04-28T13:31:08.471 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-28T13:31:08.470+0000 7fe806000640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T13:31:08.529 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-28T13:31:08.529+0000 7f0514e00640 -1 osd.7 0 waiting for initial osdmap 2024-04-28T13:31:08.537 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-28T13:31:08.536+0000 7f050ae00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T13:31:08.543 INFO:tasks.ceph.osd.1.smithi196.stderr:2024-04-28T13:31:08.544+0000 7f250de00640 -1 osd.1 0 waiting for initial osdmap 2024-04-28T13:31:08.550 INFO:tasks.ceph.osd.1.smithi196.stderr:2024-04-28T13:31:08.550+0000 7f2503400640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T13:31:08.558 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-28T13:31:08.558+0000 7f4322600640 -1 osd.5 0 waiting for initial osdmap 2024-04-28T13:31:08.564 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-28T13:31:08.564+0000 7f4318600640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T13:31:08.571 INFO:tasks.ceph.osd.0.smithi196.stderr:2024-04-28T13:31:08.572+0000 7efc0dc00640 -1 osd.0 0 waiting for initial osdmap 2024-04-28T13:31:08.578 INFO:tasks.ceph.osd.0.smithi196.stderr:2024-04-28T13:31:08.578+0000 7efc03200640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T13:31:08.580 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-28T13:31:08.580+0000 7fcd7fc00640 -1 osd.2 0 waiting for initial osdmap 2024-04-28T13:31:08.588 INFO:tasks.ceph.osd.2.smithi196.stderr:2024-04-28T13:31:08.588+0000 7fcd75c00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T13:31:08.635 DEBUG:teuthology.orchestra.run.smithi196:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:08.794 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.793+0000 7f9050400640 1 Processor -- start 2024-04-28T13:31:08.794 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.794+0000 7f9050400640 1 -- start start 2024-04-28T13:31:08.795 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.795+0000 7f9050400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9048074040 0x7f90480b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:08.795 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.795+0000 7f9050400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90480b58a0 con 0x7f90480b5200 2024-04-28T13:31:08.795 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.795+0000 7f9050400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9048072130 con 0x7f90481637a0 2024-04-28T13:31:08.795 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.795+0000 7f9050400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f90480b4210 con 0x7f9048074040 2024-04-28T13:31:08.795 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.795+0000 7f904e400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f90480b5200 0x7f9048163b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:57988/0 (socket says 172.21.15.196:57988) 2024-04-28T13:31:08.796 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.795+0000 7f904e400640 1 -- 172.21.15.196:0/3752495496 learned_addr learned my addr 172.21.15.196:0/3752495496 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:08.798 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f904d000640 1 --2- 172.21.15.196:0/3752495496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9048074040 0x7f90480b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:08.798 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f9047e00640 1 -- 172.21.15.196:0/3752495496 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 342741345 0 0) 0x7f90480b58a0 con 0x7f90480b5200 2024-04-28T13:31:08.798 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f9047e00640 1 -- 172.21.15.196:0/3752495496 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9030003520 con 0x7f90480b5200 2024-04-28T13:31:08.798 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f9047e00640 1 -- 172.21.15.196:0/3752495496 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3486296789 0 0) 0x7f9030003520 con 0x7f90480b5200 2024-04-28T13:31:08.798 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f9047e00640 1 -- 172.21.15.196:0/3752495496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9048074040 msgr2=0x7f90480b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:08.798 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f9047e00640 1 --2- 172.21.15.196:0/3752495496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9048074040 0x7f90480b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:08.798 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f9047e00640 1 -- 172.21.15.196:0/3752495496 >> v1:172.21.15.203:6789/0 conn(0x7f90481637a0 legacy=0x7f9048073930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:08.799 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f9047e00640 1 -- 172.21.15.196:0/3752495496 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9048072410 con 0x7f90480b5200 2024-04-28T13:31:08.799 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f9047e00640 1 -- 172.21.15.196:0/3752495496 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f90400019f0 con 0x7f90480b5200 2024-04-28T13:31:08.799 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f9047e00640 1 -- 172.21.15.196:0/3752495496 >> v1:172.21.15.196:6789/0 conn(0x7f90480b5200 legacy=0x7f9048163b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:08.799 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f9047e00640 1 --2- 172.21.15.196:0/3752495496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90300039e0 0x7f9030003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:08.799 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f9047e00640 1 --2- 172.21.15.196:0/3752495496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9048074040 0x7f9030004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:08.799 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f9047e00640 1 --2- 172.21.15.196:0/3752495496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9030004d40 0x7f90300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:08.799 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f9047e00640 1 -- 172.21.15.196:0/3752495496 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f903000bb50 con 0x7f90300039e0 2024-04-28T13:31:08.799 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f9047e00640 1 -- 172.21.15.196:0/3752495496 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f903000bcd0 con 0x7f9030004d40 2024-04-28T13:31:08.799 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.796+0000 7f9047e00640 1 -- 172.21.15.196:0/3752495496 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f903000bed0 con 0x7f9048074040 2024-04-28T13:31:08.799 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.797+0000 7f904da00640 1 --2- 172.21.15.196:0/3752495496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90300039e0 0x7f9030003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:08.799 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.797+0000 7f904e400640 1 --2- 172.21.15.196:0/3752495496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9030004d40 0x7f90300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:08.799 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.797+0000 7f904da00640 1 -- 172.21.15.196:0/3752495496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9048074040 msgr2=0x7f9030004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:08.799 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.797+0000 7f904da00640 1 --2- 172.21.15.196:0/3752495496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9048074040 0x7f9030004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:08.799 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.797+0000 7f904da00640 1 -- 172.21.15.196:0/3752495496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9030004d40 msgr2=0x7f90300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:08.800 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.797+0000 7f904da00640 1 --2- 172.21.15.196:0/3752495496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9030004d40 0x7f90300051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:08.800 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.797+0000 7f904da00640 1 -- 172.21.15.196:0/3752495496 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9030003520 con 0x7f90300039e0 2024-04-28T13:31:08.800 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.798+0000 7f904da00640 1 --2- 172.21.15.196:0/3752495496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90300039e0 0x7f9030003e00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f903c001750 tx=0x7f9030005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864d11cbc2649ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:08.800 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.798+0000 7f9047e00640 1 -- 172.21.15.196:0/3752495496 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c001370 con 0x7f90300039e0 2024-04-28T13:31:08.800 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.798+0000 7f9047e00640 1 -- 172.21.15.196:0/3752495496 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903c0052d0 con 0x7f90300039e0 2024-04-28T13:31:08.800 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.798+0000 7f9047e00640 1 -- 172.21.15.196:0/3752495496 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c02abe0 con 0x7f90300039e0 2024-04-28T13:31:08.802 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.799+0000 7f9050400640 1 -- 172.21.15.196:0/3752495496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90300039e0 msgr2=0x7f9030003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:08.803 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.799+0000 7f9050400640 1 --2- 172.21.15.196:0/3752495496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90300039e0 0x7f9030003e00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f903c001750 tx=0x7f9030005830 comp rx=0 tx=0).stop 2024-04-28T13:31:08.803 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.799+0000 7f904e400640 1 -- 172.21.15.196:0/3752495496 reap_dead start 2024-04-28T13:31:08.803 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.799+0000 7f9050400640 1 -- 172.21.15.196:0/3752495496 shutdown_connections 2024-04-28T13:31:08.803 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.799+0000 7f9050400640 1 -- 172.21.15.196:0/3752495496 >> 172.21.15.196:0/3752495496 conn(0x7f90480d0e10 msgr2=0x7f90480d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:08.803 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.799+0000 7f9050400640 1 -- 172.21.15.196:0/3752495496 shutdown_connections 2024-04-28T13:31:08.803 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.799+0000 7f9050400640 1 -- 172.21.15.196:0/3752495496 wait complete. 2024-04-28T13:31:08.803 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.800+0000 7f9050400640 1 Processor -- start 2024-04-28T13:31:08.803 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.800+0000 7f9050400640 1 -- start start 2024-04-28T13:31:08.803 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.801+0000 7f9050400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90300039e0 0x7f9048163b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:08.803 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.801+0000 7f9050400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9048145990 0x7f9048148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:08.803 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.801+0000 7f9050400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9048145df0 0x7f90481488c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:08.803 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.801+0000 7f9050400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9048072ef0 con 0x7f9048145df0 2024-04-28T13:31:08.804 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.801+0000 7f9050400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9048073070 con 0x7f90300039e0 2024-04-28T13:31:08.804 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.801+0000 7f9050400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f90480b4cc0 con 0x7f9048145990 2024-04-28T13:31:08.804 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.801+0000 7f904e400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9048145df0 0x7f90481488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:08.804 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.802+0000 7f904e400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9048145df0 0x7f90481488c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:42804/0 (socket says 172.21.15.196:42804) 2024-04-28T13:31:08.804 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.802+0000 7f904e400640 1 -- 172.21.15.196:0/2592498065 learned_addr learned my addr 172.21.15.196:0/2592498065 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:08.804 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.802+0000 7f904d000640 1 --2- 172.21.15.196:0/2592498065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9048145990 0x7f9048148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:08.804 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.802+0000 7f904da00640 1 --2- 172.21.15.196:0/2592498065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90300039e0 0x7f9048163b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:08.804 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.802+0000 7f904e400640 1 -- 172.21.15.196:0/2592498065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9048145990 msgr2=0x7f9048148380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:08.804 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.802+0000 7f904e400640 1 --2- 172.21.15.196:0/2592498065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9048145990 0x7f9048148380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:08.804 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.802+0000 7f904e400640 1 -- 172.21.15.196:0/2592498065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90300039e0 msgr2=0x7f9048163b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:08.804 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.802+0000 7f904e400640 1 --2- 172.21.15.196:0/2592498065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90300039e0 0x7f9048163b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:08.804 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.802+0000 7f904e400640 1 -- 172.21.15.196:0/2592498065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90481640c0 con 0x7f9048145df0 2024-04-28T13:31:08.804 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.802+0000 7f904e400640 1 --2- 172.21.15.196:0/2592498065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9048145df0 0x7f90481488c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f90480747c0 tx=0x7f904000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:08.804 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.802+0000 7f9045600640 1 -- 172.21.15.196:0/2592498065 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040004d60 con 0x7f9048145df0 2024-04-28T13:31:08.805 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.802+0000 7f9045600640 1 -- 172.21.15.196:0/2592498065 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f904000bb80 con 0x7f9048145df0 2024-04-28T13:31:08.805 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.802+0000 7f9050400640 1 -- 172.21.15.196:0/2592498065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904815b8d0 con 0x7f9048145df0 2024-04-28T13:31:08.805 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.802+0000 7f9045600640 1 -- 172.21.15.196:0/2592498065 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040007110 con 0x7f9048145df0 2024-04-28T13:31:08.805 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.803+0000 7f9050400640 1 -- 172.21.15.196:0/2592498065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904815be90 con 0x7f9048145df0 2024-04-28T13:31:08.805 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.804+0000 7f9050400640 1 -- 172.21.15.196:0/2592498065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9048076870 con 0x7f9048145df0 2024-04-28T13:31:08.806 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.804+0000 7f9045600640 1 -- 172.21.15.196:0/2592498065 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f904001b020 con 0x7f9048145df0 2024-04-28T13:31:08.806 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.806+0000 7f9045600640 1 --2- 172.21.15.196:0/2592498065 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f902c06c860 0x7f902c06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:08.806 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.806+0000 7f9045600640 1 -- 172.21.15.196:0/2592498065 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f904008add0 con 0x7f9048145df0 2024-04-28T13:31:08.806 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.806+0000 7f904da00640 1 --2- 172.21.15.196:0/2592498065 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f902c06c860 0x7f902c06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:08.807 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.807+0000 7f904da00640 1 --2- 172.21.15.196:0/2592498065 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f902c06c860 0x7f902c06ed20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f903c032000 tx=0x7f903c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:08.810 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.810+0000 7f9045600640 1 -- 172.21.15.196:0/2592498065 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f904005a100 con 0x7f9048145df0 2024-04-28T13:31:08.994 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.994+0000 7f9050400640 1 -- 172.21.15.196:0/2592498065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f904806def0 con 0x7f9048145df0 2024-04-28T13:31:08.995 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.995+0000 7f9045600640 1 -- 172.21.15.196:0/2592498065 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f9040059a00 con 0x7f9048145df0 2024-04-28T13:31:08.996 INFO:teuthology.misc.health.smithi196.stdout: 2024-04-28T13:31:08.996 INFO:teuthology.misc.health.smithi196.stdout:{"epoch":12,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:06.201067+0000","last_up_change":"0.000000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:08.999 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.999+0000 7f9050400640 1 -- 172.21.15.196:0/2592498065 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f902c06c860 msgr2=0x7f902c06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:08.999 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.999+0000 7f9050400640 1 --2- 172.21.15.196:0/2592498065 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f902c06c860 0x7f902c06ed20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f903c032000 tx=0x7f903c02e000 comp rx=0 tx=0).stop 2024-04-28T13:31:08.999 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.999+0000 7f9050400640 1 -- 172.21.15.196:0/2592498065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9048145df0 msgr2=0x7f90481488c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:08.999 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.999+0000 7f9050400640 1 --2- 172.21.15.196:0/2592498065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9048145df0 0x7f90481488c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f90480747c0 tx=0x7f904000b4a0 comp rx=0 tx=0).stop 2024-04-28T13:31:08.999 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.999+0000 7f9050400640 1 -- 172.21.15.196:0/2592498065 shutdown_connections 2024-04-28T13:31:08.999 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.999+0000 7f9050400640 1 --2- 172.21.15.196:0/2592498065 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f902c06c860 0x7f902c06ed20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:08.999 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.999+0000 7f9050400640 1 --2- 172.21.15.196:0/2592498065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9048145df0 0x7f90481488c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:08.999 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.999+0000 7f9050400640 1 --2- 172.21.15.196:0/2592498065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9048145990 0x7f9048148380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:09.000 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.999+0000 7f9050400640 1 --2- 172.21.15.196:0/2592498065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90300039e0 0x7f9048163b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:09.000 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.999+0000 7f9050400640 1 -- 172.21.15.196:0/2592498065 >> 172.21.15.196:0/2592498065 conn(0x7f90480d0e10 msgr2=0x7f90480d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:09.000 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.999+0000 7f9050400640 1 -- 172.21.15.196:0/2592498065 shutdown_connections 2024-04-28T13:31:09.000 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:08.999+0000 7f9050400640 1 -- 172.21.15.196:0/2592498065 wait complete. 2024-04-28T13:31:09.012 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-28T13:31:09.388 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:31:09.396 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:31:15.819 DEBUG:teuthology.orchestra.run.smithi196:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:16.000 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:15.999+0000 7fc490a00640 1 Processor -- start 2024-04-28T13:31:16.000 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.000+0000 7fc490a00640 1 -- start start 2024-04-28T13:31:16.001 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.001+0000 7fc490a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc48c1401b0 0x7fc48c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.001 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.001+0000 7fc490a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc48c0ce0e0 0x7fc48c0b6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.001 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.001+0000 7fc490a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc48c160ee0 con 0x7fc48c0b61d0 2024-04-28T13:31:16.002 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.001+0000 7fc490a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc48c161af0 con 0x7fc48c0ce0e0 2024-04-28T13:31:16.002 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.001+0000 7fc490a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc48c1611c0 con 0x7fc48c1401b0 2024-04-28T13:31:16.002 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.002+0000 7fc489400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc48c0b61d0 0x7fc48c0cd9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:57854/0 (socket says 172.21.15.196:57854) 2024-04-28T13:31:16.002 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.002+0000 7fc489400640 1 -- 172.21.15.196:0/827085989 learned_addr learned my addr 172.21.15.196:0/827085989 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:16.004 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.002+0000 7fc489e00640 1 --2- 172.21.15.196:0/827085989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc48c1401b0 0x7fc48c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.004 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.002+0000 7fc488a00640 1 -- 172.21.15.196:0/827085989 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68332080 0 0) 0x7fc48c160ee0 con 0x7fc48c0b61d0 2024-04-28T13:31:16.004 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.002+0000 7fc488a00640 1 -- 172.21.15.196:0/827085989 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc470003520 con 0x7fc48c0b61d0 2024-04-28T13:31:16.004 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.002+0000 7fc48a800640 1 --2- 172.21.15.196:0/827085989 >> v2:172.21.15.203:3300/0 conn(0x7fc48c0ce0e0 0x7fc48c0b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.004 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.002+0000 7fc488a00640 1 -- 172.21.15.196:0/827085989 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2027955103 0 0) 0x7fc470003520 con 0x7fc48c0b61d0 2024-04-28T13:31:16.004 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.002+0000 7fc488a00640 1 -- 172.21.15.196:0/827085989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc48c1401b0 msgr2=0x7fc48c0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:16.004 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.002+0000 7fc488a00640 1 --2- 172.21.15.196:0/827085989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc48c1401b0 0x7fc48c0cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.004 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.002+0000 7fc488a00640 1 -- 172.21.15.196:0/827085989 >> v2:172.21.15.203:3300/0 conn(0x7fc48c0ce0e0 msgr2=0x7fc48c0b6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:16.004 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.002+0000 7fc488a00640 1 --2- 172.21.15.196:0/827085989 >> v2:172.21.15.203:3300/0 conn(0x7fc48c0ce0e0 0x7fc48c0b6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.004 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.002+0000 7fc488a00640 1 -- 172.21.15.196:0/827085989 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48c1614a0 con 0x7fc48c0b61d0 2024-04-28T13:31:16.004 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.002+0000 7fc488a00640 1 -- 172.21.15.196:0/827085989 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fc478001670 con 0x7fc48c0b61d0 2024-04-28T13:31:16.005 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.002+0000 7fc488a00640 1 -- 172.21.15.196:0/827085989 >> v1:172.21.15.196:6789/0 conn(0x7fc48c0b61d0 legacy=0x7fc48c0cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:16.005 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.003+0000 7fc488a00640 1 --2- 172.21.15.196:0/827085989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4700039b0 0x7fc470003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.005 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.003+0000 7fc488a00640 1 --2- 172.21.15.196:0/827085989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc470004800 0x7fc470004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.005 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.003+0000 7fc488a00640 1 --2- 172.21.15.196:0/827085989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc48c1401b0 0x7fc4700051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.005 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.003+0000 7fc488a00640 1 -- 172.21.15.196:0/827085989 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c161af0 con 0x7fc470004800 2024-04-28T13:31:16.005 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.003+0000 7fc488a00640 1 -- 172.21.15.196:0/827085989 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c1611c0 con 0x7fc4700039b0 2024-04-28T13:31:16.005 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.003+0000 7fc488a00640 1 -- 172.21.15.196:0/827085989 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc47000bb50 con 0x7fc48c1401b0 2024-04-28T13:31:16.005 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.003+0000 7fc489400640 1 --2- 172.21.15.196:0/827085989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc470004800 0x7fc470004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.005 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.003+0000 7fc489400640 1 -- 172.21.15.196:0/827085989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc48c1401b0 msgr2=0x7fc4700051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:16.005 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.003+0000 7fc489400640 1 --2- 172.21.15.196:0/827085989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc48c1401b0 0x7fc4700051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.005 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.003+0000 7fc489e00640 1 --2- 172.21.15.196:0/827085989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4700039b0 0x7fc470003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.005 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.003+0000 7fc489400640 1 -- 172.21.15.196:0/827085989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4700039b0 msgr2=0x7fc470003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:16.005 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.003+0000 7fc489400640 1 --2- 172.21.15.196:0/827085989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4700039b0 0x7fc470003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.005 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.003+0000 7fc489400640 1 -- 172.21.15.196:0/827085989 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc470003520 con 0x7fc470004800 2024-04-28T13:31:16.006 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.004+0000 7fc489400640 1 --2- 172.21.15.196:0/827085989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc470004800 0x7fc470004c80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc47800c160 tx=0x7fc470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7def6107e5daedbc server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:16.007 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.004+0000 7fc488a00640 1 -- 172.21.15.196:0/827085989 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478007c80 con 0x7fc470004800 2024-04-28T13:31:16.007 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.004+0000 7fc488a00640 1 -- 172.21.15.196:0/827085989 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc478005ad0 con 0x7fc470004800 2024-04-28T13:31:16.007 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.004+0000 7fc488a00640 1 -- 172.21.15.196:0/827085989 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478007070 con 0x7fc470004800 2024-04-28T13:31:16.007 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.005+0000 7fc490a00640 1 -- 172.21.15.196:0/827085989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc470004800 msgr2=0x7fc470004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:16.007 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.005+0000 7fc490a00640 1 --2- 172.21.15.196:0/827085989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc470004800 0x7fc470004c80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc47800c160 tx=0x7fc470005830 comp rx=0 tx=0).stop 2024-04-28T13:31:16.007 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.005+0000 7fc48a800640 1 -- 172.21.15.196:0/827085989 reap_dead start 2024-04-28T13:31:16.007 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.005+0000 7fc490a00640 1 -- 172.21.15.196:0/827085989 shutdown_connections 2024-04-28T13:31:16.007 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.005+0000 7fc490a00640 1 -- 172.21.15.196:0/827085989 >> 172.21.15.196:0/827085989 conn(0x7fc48c0d0e10 msgr2=0x7fc48c0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:16.007 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.005+0000 7fc490a00640 1 -- 172.21.15.196:0/827085989 shutdown_connections 2024-04-28T13:31:16.007 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.005+0000 7fc490a00640 1 -- 172.21.15.196:0/827085989 wait complete. 2024-04-28T13:31:16.007 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.007+0000 7fc490a00640 1 Processor -- start 2024-04-28T13:31:16.008 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.007+0000 7fc490a00640 1 -- start start 2024-04-28T13:31:16.009 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.008+0000 7fc490a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc470004800 0x7fc48c14f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.009 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.008+0000 7fc490a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc48c14faf0 0x7fc48c0b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.009 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.008+0000 7fc490a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc48c1510d0 0x7fc48c0b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.010 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.008+0000 7fc490a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c0cf900 con 0x7fc48c1510d0 2024-04-28T13:31:16.010 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.008+0000 7fc490a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c140590 con 0x7fc48c14faf0 2024-04-28T13:31:16.010 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.008+0000 7fc490a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc48c0cfa80 con 0x7fc470004800 2024-04-28T13:31:16.010 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.008+0000 7fc48a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc48c1510d0 0x7fc48c0b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.010 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.008+0000 7fc48a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc48c1510d0 0x7fc48c0b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:47904/0 (socket says 172.21.15.196:47904) 2024-04-28T13:31:16.010 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.008+0000 7fc48a800640 1 -- 172.21.15.196:0/1482047325 learned_addr learned my addr 172.21.15.196:0/1482047325 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:16.010 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.009+0000 7fc489400640 1 --2- 172.21.15.196:0/1482047325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc48c14faf0 0x7fc48c0b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.010 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.009+0000 7fc489e00640 1 --2- 172.21.15.196:0/1482047325 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc470004800 0x7fc48c14f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.010 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.009+0000 7fc48a800640 1 -- 172.21.15.196:0/1482047325 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc470004800 msgr2=0x7fc48c14f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:16.010 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.009+0000 7fc48a800640 1 --2- 172.21.15.196:0/1482047325 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc470004800 0x7fc48c14f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.010 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.009+0000 7fc48a800640 1 -- 172.21.15.196:0/1482047325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc48c14faf0 msgr2=0x7fc48c0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:16.010 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.009+0000 7fc48a800640 1 --2- 172.21.15.196:0/1482047325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc48c14faf0 0x7fc48c0b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.010 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.009+0000 7fc48a800640 1 -- 172.21.15.196:0/1482047325 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48c150030 con 0x7fc48c1510d0 2024-04-28T13:31:16.010 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.009+0000 7fc48a800640 1 --2- 172.21.15.196:0/1482047325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc48c1510d0 0x7fc48c0b4aa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc48c076320 tx=0x7fc48400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:16.011 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.009+0000 7fc482000640 1 -- 172.21.15.196:0/1482047325 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48400ed90 con 0x7fc48c1510d0 2024-04-28T13:31:16.011 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.009+0000 7fc482000640 1 -- 172.21.15.196:0/1482047325 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc484019070 con 0x7fc48c1510d0 2024-04-28T13:31:16.011 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.009+0000 7fc482000640 1 -- 172.21.15.196:0/1482047325 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484014660 con 0x7fc48c1510d0 2024-04-28T13:31:16.011 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.009+0000 7fc490a00640 1 -- 172.21.15.196:0/1482047325 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48c150310 con 0x7fc48c1510d0 2024-04-28T13:31:16.011 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.010+0000 7fc490a00640 1 -- 172.21.15.196:0/1482047325 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48c1508d0 con 0x7fc48c1510d0 2024-04-28T13:31:16.011 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.011+0000 7fc490a00640 1 -- 172.21.15.196:0/1482047325 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc48c076870 con 0x7fc48c1510d0 2024-04-28T13:31:16.011 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.012+0000 7fc482000640 1 -- 172.21.15.196:0/1482047325 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc484014800 con 0x7fc48c1510d0 2024-04-28T13:31:16.012 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.013+0000 7fc482000640 1 --2- 172.21.15.196:0/1482047325 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc46c06c8f0 0x7fc46c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.012 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.013+0000 7fc482000640 1 -- 172.21.15.196:0/1482047325 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fc48408bce0 con 0x7fc48c1510d0 2024-04-28T13:31:16.013 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.013+0000 7fc489e00640 1 --2- 172.21.15.196:0/1482047325 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc46c06c8f0 0x7fc46c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.014 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.014+0000 7fc489e00640 1 --2- 172.21.15.196:0/1482047325 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc46c06c8f0 0x7fc46c06edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc47c006fd0 tx=0x7fc47c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:16.018 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.018+0000 7fc482000640 1 -- 172.21.15.196:0/1482047325 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc48405a3c0 con 0x7fc48c1510d0 2024-04-28T13:31:16.217 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.217+0000 7fc490a00640 1 -- 172.21.15.196:0/1482047325 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc48c0cd300 con 0x7fc48c1510d0 2024-04-28T13:31:16.218 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.218+0000 7fc482000640 1 -- 172.21.15.196:0/1482047325 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7fc484083e20 con 0x7fc48c1510d0 2024-04-28T13:31:16.218 INFO:teuthology.misc.health.smithi196.stdout: 2024-04-28T13:31:16.218 INFO:teuthology.misc.health.smithi196.stdout:{"epoch":16,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:12.469573+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:16.221 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.221+0000 7fc490a00640 1 -- 172.21.15.196:0/1482047325 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc46c06c8f0 msgr2=0x7fc46c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:16.221 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.221+0000 7fc490a00640 1 --2- 172.21.15.196:0/1482047325 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc46c06c8f0 0x7fc46c06edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc47c006fd0 tx=0x7fc47c008040 comp rx=0 tx=0).stop 2024-04-28T13:31:16.221 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.221+0000 7fc490a00640 1 -- 172.21.15.196:0/1482047325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc48c1510d0 msgr2=0x7fc48c0b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:16.221 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.221+0000 7fc490a00640 1 --2- 172.21.15.196:0/1482047325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc48c1510d0 0x7fc48c0b4aa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc48c076320 tx=0x7fc48400e4b0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.221 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.221+0000 7fc490a00640 1 -- 172.21.15.196:0/1482047325 shutdown_connections 2024-04-28T13:31:16.221 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.221+0000 7fc490a00640 1 --2- 172.21.15.196:0/1482047325 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc46c06c8f0 0x7fc46c06edb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.222 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.221+0000 7fc490a00640 1 --2- 172.21.15.196:0/1482047325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc48c1510d0 0x7fc48c0b4aa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.222 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.221+0000 7fc490a00640 1 --2- 172.21.15.196:0/1482047325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc48c14faf0 0x7fc48c0b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.222 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.221+0000 7fc490a00640 1 --2- 172.21.15.196:0/1482047325 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc470004800 0x7fc48c14f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.222 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.221+0000 7fc490a00640 1 -- 172.21.15.196:0/1482047325 >> 172.21.15.196:0/1482047325 conn(0x7fc48c0d0e10 msgr2=0x7fc48c073260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:16.222 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.221+0000 7fc490a00640 1 -- 172.21.15.196:0/1482047325 shutdown_connections 2024-04-28T13:31:16.222 INFO:teuthology.misc.health.smithi196.stderr:2024-04-28T13:31:16.221+0000 7fc490a00640 1 -- 172.21.15.196:0/1482047325 wait complete. 2024-04-28T13:31:16.236 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-28T13:31:16.236 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-28T13:31:16.236 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-28T13:31:16.236 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-28T13:31:16.241 INFO:tasks.ceph.mds.a:Started 2024-04-28T13:31:16.241 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-28T13:31:16.241 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-28T13:31:16.246 INFO:tasks.ceph.mds.c:Started 2024-04-28T13:31:16.246 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-28T13:31:16.246 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-28T13:31:16.254 INFO:tasks.ceph.mds.b:Started 2024-04-28T13:31:16.254 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-28T13:31:16.254 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-28T13:31:16.261 INFO:tasks.ceph.mds.d:Started 2024-04-28T13:31:16.262 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-28T13:31:16.262 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:16.381 INFO:tasks.ceph.mds.a.smithi196.stdout:starting mds.a at 2024-04-28T13:31:16.381 INFO:tasks.ceph.mds.c.smithi196.stdout:starting mds.c at 2024-04-28T13:31:16.414 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-28T13:31:16.414 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-28T13:31:16.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.432+0000 7fc955800640 1 Processor -- start 2024-04-28T13:31:16.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.432+0000 7fc955800640 1 -- start start 2024-04-28T13:31:16.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.433+0000 7fc955800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc95015f4d0 0x7fc9500b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.433+0000 7fc955800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9500d2e20 0x7fc9500d5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.433+0000 7fc955800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9500d5ab0 con 0x7fc9500d5640 2024-04-28T13:31:16.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.433+0000 7fc955800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc9500b6180 con 0x7fc95015f4d0 2024-04-28T13:31:16.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.433+0000 7fc955800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc9500b2550 con 0x7fc9500d2e20 2024-04-28T13:31:16.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.434+0000 7fc94ea00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc95015f4d0 0x7fc9500b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.434+0000 7fc94f400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc9500d5640 0x7fc9500b4ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:57878/0 (socket says 172.21.15.196:57878) 2024-04-28T13:31:16.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.434+0000 7fc94f400640 1 -- 172.21.15.196:0/250070138 learned_addr learned my addr 172.21.15.196:0/250070138 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:16.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.434+0000 7fc94e000640 1 --2- 172.21.15.196:0/250070138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9500d2e20 0x7fc9500d5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.434+0000 7fc94ea00640 1 -- 172.21.15.196:0/250070138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9500d2e20 msgr2=0x7fc9500d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:16.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.434+0000 7fc94ea00640 1 --2- 172.21.15.196:0/250070138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9500d2e20 0x7fc9500d5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.434+0000 7fc94ea00640 1 -- 172.21.15.196:0/250070138 >> v1:172.21.15.196:6789/0 conn(0x7fc9500d5640 legacy=0x7fc9500b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:16.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.434+0000 7fc94ea00640 1 -- 172.21.15.196:0/250070138 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9500b2830 con 0x7fc95015f4d0 2024-04-28T13:31:16.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.435+0000 7fc94ea00640 1 --2- 172.21.15.196:0/250070138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95015f4d0 0x7fc9500b1ee0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc9400097a0 tx=0x7fc94002a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a24a2f5edfcb23b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:16.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.435+0000 7fc94d600640 1 -- 172.21.15.196:0/250070138 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940037070 con 0x7fc95015f4d0 2024-04-28T13:31:16.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.435+0000 7fc94d600640 1 -- 172.21.15.196:0/250070138 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc94002ac90 con 0x7fc95015f4d0 2024-04-28T13:31:16.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.435+0000 7fc94d600640 1 -- 172.21.15.196:0/250070138 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940032660 con 0x7fc95015f4d0 2024-04-28T13:31:16.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.436+0000 7fc955800640 1 -- 172.21.15.196:0/250070138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95015f4d0 msgr2=0x7fc9500b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:16.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.436+0000 7fc955800640 1 --2- 172.21.15.196:0/250070138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95015f4d0 0x7fc9500b1ee0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc9400097a0 tx=0x7fc94002a000 comp rx=0 tx=0).stop 2024-04-28T13:31:16.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.436+0000 7fc955800640 1 -- 172.21.15.196:0/250070138 shutdown_connections 2024-04-28T13:31:16.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.436+0000 7fc955800640 1 --2- 172.21.15.196:0/250070138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9500d2e20 0x7fc9500d5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.436+0000 7fc955800640 1 --2- 172.21.15.196:0/250070138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95015f4d0 0x7fc9500b1ee0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.436+0000 7fc955800640 1 -- 172.21.15.196:0/250070138 >> 172.21.15.196:0/250070138 conn(0x7fc95008ebf0 msgr2=0x7fc95009e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:16.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.436+0000 7fc955800640 1 -- 172.21.15.196:0/250070138 shutdown_connections 2024-04-28T13:31:16.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.436+0000 7fc955800640 1 -- 172.21.15.196:0/250070138 wait complete. 2024-04-28T13:31:16.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.438+0000 7fc955800640 1 Processor -- start 2024-04-28T13:31:16.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.438+0000 7fc955800640 1 -- start start 2024-04-28T13:31:16.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc955800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9500d2e20 0x7fc95015b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc955800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9500d5640 0x7fc95015f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc955800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc95015f4d0 0x7fc95015fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc955800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9500b30d0 con 0x7fc9500d2e20 2024-04-28T13:31:16.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc955800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc9500b2df0 con 0x7fc9500d5640 2024-04-28T13:31:16.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc955800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc9500b4600 con 0x7fc95015f4d0 2024-04-28T13:31:16.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc94ea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9500d2e20 0x7fc95015b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc94ea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9500d2e20 0x7fc95015b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:47964/0 (socket says 172.21.15.196:47964) 2024-04-28T13:31:16.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc94ea00640 1 -- 172.21.15.196:0/3471132254 learned_addr learned my addr 172.21.15.196:0/3471132254 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:16.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc94f400640 1 --2- 172.21.15.196:0/3471132254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc95015f4d0 0x7fc95015fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc94e000640 1 --2- 172.21.15.196:0/3471132254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9500d5640 0x7fc95015f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc94ea00640 1 -- 172.21.15.196:0/3471132254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc95015f4d0 msgr2=0x7fc95015fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:16.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc94ea00640 1 --2- 172.21.15.196:0/3471132254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc95015f4d0 0x7fc95015fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc94ea00640 1 -- 172.21.15.196:0/3471132254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9500d5640 msgr2=0x7fc95015f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:16.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc94ea00640 1 --2- 172.21.15.196:0/3471132254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9500d5640 0x7fc95015f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc94ea00640 1 -- 172.21.15.196:0/3471132254 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9500b83e0 con 0x7fc9500d2e20 2024-04-28T13:31:16.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.439+0000 7fc94ea00640 1 --2- 172.21.15.196:0/3471132254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9500d2e20 0x7fc95015b170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc940035040 tx=0x7fc9400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:16.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.440+0000 7fc946a00640 1 -- 172.21.15.196:0/3471132254 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940041030 con 0x7fc9500d2e20 2024-04-28T13:31:16.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.440+0000 7fc946a00640 1 -- 172.21.15.196:0/3471132254 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc94003d420 con 0x7fc9500d2e20 2024-04-28T13:31:16.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.440+0000 7fc946a00640 1 -- 172.21.15.196:0/3471132254 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940037040 con 0x7fc9500d2e20 2024-04-28T13:31:16.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.440+0000 7fc955800640 1 -- 172.21.15.196:0/3471132254 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9500b86c0 con 0x7fc9500d2e20 2024-04-28T13:31:16.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.440+0000 7fc955800640 1 -- 172.21.15.196:0/3471132254 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9500b8c80 con 0x7fc9500d2e20 2024-04-28T13:31:16.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.441+0000 7fc955800640 1 -- 172.21.15.196:0/3471132254 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc95006a290 con 0x7fc9500d2e20 2024-04-28T13:31:16.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.442+0000 7fc946a00640 1 -- 172.21.15.196:0/3471132254 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc940044070 con 0x7fc9500d2e20 2024-04-28T13:31:16.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.443+0000 7fc946a00640 1 --2- 172.21.15.196:0/3471132254 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc93006c8f0 0x7fc93006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.443+0000 7fc946a00640 1 -- 172.21.15.196:0/3471132254 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fc9400ab9e0 con 0x7fc9500d2e20 2024-04-28T13:31:16.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.443+0000 7fc94e000640 1 --2- 172.21.15.196:0/3471132254 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc93006c8f0 0x7fc93006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.444+0000 7fc94e000640 1 --2- 172.21.15.196:0/3471132254 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc93006c8f0 0x7fc93006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc9500b6f60 tx=0x7fc93c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:16.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.447+0000 7fc946a00640 1 -- 172.21.15.196:0/3471132254 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc94004b070 con 0x7fc9500d2e20 2024-04-28T13:31:16.644 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.644+0000 7fc955800640 1 -- 172.21.15.196:0/3471132254 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9500b9060 con 0x7fc9500d2e20 2024-04-28T13:31:16.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.645+0000 7fc946a00640 1 -- 172.21.15.196:0/3471132254 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7fc94007a0d0 con 0x7fc9500d2e20 2024-04-28T13:31:16.645 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:16.645 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":16,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:12.469573+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:16.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.647+0000 7fc955800640 1 -- 172.21.15.196:0/3471132254 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc93006c8f0 msgr2=0x7fc93006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:16.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.647+0000 7fc955800640 1 --2- 172.21.15.196:0/3471132254 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc93006c8f0 0x7fc93006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc9500b6f60 tx=0x7fc93c008040 comp rx=0 tx=0).stop 2024-04-28T13:31:16.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.647+0000 7fc955800640 1 -- 172.21.15.196:0/3471132254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9500d2e20 msgr2=0x7fc95015b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:16.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.647+0000 7fc955800640 1 --2- 172.21.15.196:0/3471132254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9500d2e20 0x7fc95015b170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc940035040 tx=0x7fc9400046c0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.647+0000 7fc955800640 1 -- 172.21.15.196:0/3471132254 shutdown_connections 2024-04-28T13:31:16.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.648+0000 7fc955800640 1 --2- 172.21.15.196:0/3471132254 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc93006c8f0 0x7fc93006edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.648+0000 7fc955800640 1 --2- 172.21.15.196:0/3471132254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc95015f4d0 0x7fc95015fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.648+0000 7fc955800640 1 --2- 172.21.15.196:0/3471132254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9500d5640 0x7fc95015f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.648+0000 7fc955800640 1 --2- 172.21.15.196:0/3471132254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9500d2e20 0x7fc95015b170 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.648+0000 7fc955800640 1 -- 172.21.15.196:0/3471132254 >> 172.21.15.196:0/3471132254 conn(0x7fc95008ebf0 msgr2=0x7fc9500cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:16.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.648+0000 7fc955800640 1 -- 172.21.15.196:0/3471132254 shutdown_connections 2024-04-28T13:31:16.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.648+0000 7fc955800640 1 -- 172.21.15.196:0/3471132254 wait complete. 2024-04-28T13:31:16.666 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:31:16.666 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:31:16.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.863+0000 7f006fe00640 1 Processor -- start 2024-04-28T13:31:16.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.863+0000 7f006fe00640 1 -- start start 2024-04-28T13:31:16.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.865+0000 7f006fe00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f00700d17b0 0x7f007006f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.865+0000 7f006fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f00700b5df0 0x7f0070072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.865+0000 7f006fe00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f007006df10 con 0x7f007015f150 2024-04-28T13:31:16.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.865+0000 7f006fe00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f007006fda0 con 0x7f00700b5df0 2024-04-28T13:31:16.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.865+0000 7f006fe00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f007006e1f0 con 0x7f00700d17b0 2024-04-28T13:31:16.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.865+0000 7f006f400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f00700b5df0 0x7f0070072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.865+0000 7f006f400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f00700b5df0 0x7f0070072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:47978/0 (socket says 172.21.15.196:47978) 2024-04-28T13:31:16.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.865+0000 7f006f400640 1 -- 172.21.15.196:0/4186911190 learned_addr learned my addr 172.21.15.196:0/4186911190 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:16.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.865+0000 7f006e000640 1 --2- 172.21.15.196:0/4186911190 >> v2:172.21.15.203:3300/0 conn(0x7f00700d17b0 0x7f007006f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.866+0000 7f006f400640 1 -- 172.21.15.196:0/4186911190 >> v2:172.21.15.203:3300/0 conn(0x7f00700d17b0 msgr2=0x7f007006f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:16.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.866+0000 7f006f400640 1 --2- 172.21.15.196:0/4186911190 >> v2:172.21.15.203:3300/0 conn(0x7f00700d17b0 0x7f007006f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.866+0000 7f006f400640 1 -- 172.21.15.196:0/4186911190 >> v1:172.21.15.203:6789/0 conn(0x7f007015f150 legacy=0x7f00700d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:16.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.866+0000 7f006f400640 1 -- 172.21.15.196:0/4186911190 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007006e4d0 con 0x7f00700b5df0 2024-04-28T13:31:16.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.867+0000 7f006f400640 1 --2- 172.21.15.196:0/4186911190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00700b5df0 0x7f0070072150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f00700703e0 tx=0x7f006802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349a6cf1ba640d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:16.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.867+0000 7f006d600640 1 -- 172.21.15.196:0/4186911190 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068036070 con 0x7f00700b5df0 2024-04-28T13:31:16.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.867+0000 7f006d600640 1 -- 172.21.15.196:0/4186911190 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0068031600 con 0x7f00700b5df0 2024-04-28T13:31:16.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.867+0000 7f006d600640 1 -- 172.21.15.196:0/4186911190 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00680319d0 con 0x7f00700b5df0 2024-04-28T13:31:16.868 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.868+0000 7f006fe00640 1 -- 172.21.15.196:0/4186911190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00700b5df0 msgr2=0x7f0070072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:16.868 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.868+0000 7f006fe00640 1 --2- 172.21.15.196:0/4186911190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00700b5df0 0x7f0070072150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f00700703e0 tx=0x7f006802a6f0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.868 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.868+0000 7f006fe00640 1 -- 172.21.15.196:0/4186911190 shutdown_connections 2024-04-28T13:31:16.868 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.868+0000 7f006fe00640 1 --2- 172.21.15.196:0/4186911190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00700b5df0 0x7f0070072150 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.868+0000 7f006fe00640 1 --2- 172.21.15.196:0/4186911190 >> v2:172.21.15.203:3300/0 conn(0x7f00700d17b0 0x7f007006f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.868+0000 7f006fe00640 1 -- 172.21.15.196:0/4186911190 >> 172.21.15.196:0/4186911190 conn(0x7f007006c6c0 msgr2=0x7f007006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:16.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.868+0000 7f006fe00640 1 -- 172.21.15.196:0/4186911190 shutdown_connections 2024-04-28T13:31:16.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.868+0000 7f006fe00640 1 -- 172.21.15.196:0/4186911190 wait complete. 2024-04-28T13:31:16.870 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.870+0000 7f006fe00640 1 Processor -- start 2024-04-28T13:31:16.870 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.870+0000 7f006fe00640 1 -- start start 2024-04-28T13:31:16.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006fe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00700b5df0 0x7f0070150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006fe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00700d17b0 0x7f007015a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f007015f150 0x7f007015a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006fe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f007015f530 con 0x7f007015f150 2024-04-28T13:31:16.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006fe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f007006ea90 con 0x7f00700b5df0 2024-04-28T13:31:16.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006fe00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0070072690 con 0x7f00700d17b0 2024-04-28T13:31:16.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006f400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f007015f150 0x7f007015a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006f400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f007015f150 0x7f007015a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:47988/0 (socket says 172.21.15.196:47988) 2024-04-28T13:31:16.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006f400640 1 -- 172.21.15.196:0/3505240557 learned_addr learned my addr 172.21.15.196:0/3505240557 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:16.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006ea00640 1 --2- 172.21.15.196:0/3505240557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00700b5df0 0x7f0070150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006e000640 1 --2- 172.21.15.196:0/3505240557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00700d17b0 0x7f007015a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006f400640 1 -- 172.21.15.196:0/3505240557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00700d17b0 msgr2=0x7f007015a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:16.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006f400640 1 --2- 172.21.15.196:0/3505240557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00700d17b0 0x7f007015a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006f400640 1 -- 172.21.15.196:0/3505240557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00700b5df0 msgr2=0x7f0070150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:16.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006f400640 1 --2- 172.21.15.196:0/3505240557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00700b5df0 0x7f0070150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:16.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006f400640 1 -- 172.21.15.196:0/3505240557 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0070156c40 con 0x7f007015f150 2024-04-28T13:31:16.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.872+0000 7f006ea00640 1 --2- 172.21.15.196:0/3505240557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00700b5df0 0x7f0070150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:16.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.873+0000 7f006f400640 1 --2- 172.21.15.196:0/3505240557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f007015f150 0x7f007015a880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0068009f40 tx=0x7f0068004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:16.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.873+0000 7f0066a00640 1 -- 172.21.15.196:0/3505240557 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068036050 con 0x7f007015f150 2024-04-28T13:31:16.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.873+0000 7f0066a00640 1 -- 172.21.15.196:0/3505240557 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0068042070 con 0x7f007015f150 2024-04-28T13:31:16.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.873+0000 7f0066a00640 1 -- 172.21.15.196:0/3505240557 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068009510 con 0x7f007015f150 2024-04-28T13:31:16.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.873+0000 7f006fe00640 1 -- 172.21.15.196:0/3505240557 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0070156f20 con 0x7f007015f150 2024-04-28T13:31:16.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.873+0000 7f006fe00640 1 -- 172.21.15.196:0/3505240557 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0070152440 con 0x7f007015f150 2024-04-28T13:31:16.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.874+0000 7f006fe00640 1 -- 172.21.15.196:0/3505240557 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0070050730 con 0x7f007015f150 2024-04-28T13:31:16.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.875+0000 7f0066a00640 1 -- 172.21.15.196:0/3505240557 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0068007bd0 con 0x7f007015f150 2024-04-28T13:31:16.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.876+0000 7f0066a00640 1 --2- 172.21.15.196:0/3505240557 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f005006c8f0 0x7f005006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:16.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.876+0000 7f0066a00640 1 -- 172.21.15.196:0/3505240557 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f00680ac720 con 0x7f007015f150 2024-04-28T13:31:16.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.877+0000 7f006ea00640 1 --2- 172.21.15.196:0/3505240557 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f005006c8f0 0x7f005006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:16.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.878+0000 7f006ea00640 1 --2- 172.21.15.196:0/3505240557 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f005006c8f0 0x7f005006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0070155a20 tx=0x7f0060008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:16.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:16.880+0000 7f0066a00640 1 -- 172.21.15.196:0/3505240557 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0068045360 con 0x7f007015f150 2024-04-28T13:31:17.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.063+0000 7f006fe00640 1 -- 172.21.15.196:0/3505240557 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0070151f30 con 0x7f007015f150 2024-04-28T13:31:17.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.064+0000 7f0066a00640 1 -- 172.21.15.196:0/3505240557 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f006807aea0 con 0x7f007015f150 2024-04-28T13:31:17.064 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:31:17.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.066+0000 7f006fe00640 1 -- 172.21.15.196:0/3505240557 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f005006c8f0 msgr2=0x7f005006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:17.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.066+0000 7f006fe00640 1 --2- 172.21.15.196:0/3505240557 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f005006c8f0 0x7f005006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0070155a20 tx=0x7f0060008040 comp rx=0 tx=0).stop 2024-04-28T13:31:17.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.066+0000 7f006fe00640 1 -- 172.21.15.196:0/3505240557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f007015f150 msgr2=0x7f007015a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:17.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.066+0000 7f006fe00640 1 --2- 172.21.15.196:0/3505240557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f007015f150 0x7f007015a880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0068009f40 tx=0x7f0068004a40 comp rx=0 tx=0).stop 2024-04-28T13:31:17.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.066+0000 7f006fe00640 1 -- 172.21.15.196:0/3505240557 shutdown_connections 2024-04-28T13:31:17.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.066+0000 7f006fe00640 1 --2- 172.21.15.196:0/3505240557 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f005006c8f0 0x7f005006edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.066+0000 7f006fe00640 1 --2- 172.21.15.196:0/3505240557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f007015f150 0x7f007015a880 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.066+0000 7f006fe00640 1 --2- 172.21.15.196:0/3505240557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00700d17b0 0x7f007015a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.066+0000 7f006fe00640 1 --2- 172.21.15.196:0/3505240557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00700b5df0 0x7f0070150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.066+0000 7f006fe00640 1 -- 172.21.15.196:0/3505240557 >> 172.21.15.196:0/3505240557 conn(0x7f007006c6c0 msgr2=0x7f007006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:17.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.066+0000 7f006fe00640 1 -- 172.21.15.196:0/3505240557 shutdown_connections 2024-04-28T13:31:17.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.066+0000 7f006fe00640 1 -- 172.21.15.196:0/3505240557 wait complete. 2024-04-28T13:31:17.085 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:31:17.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.280+0000 7f0b36400640 1 Processor -- start 2024-04-28T13:31:17.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.280+0000 7f0b36400640 1 -- start start 2024-04-28T13:31:17.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.281+0000 7f0b36400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b301469f0 0x7f0b300ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:17.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.281+0000 7f0b36400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0b300d1410 0x7f0b300b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:17.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.281+0000 7f0b36400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b30160f40 con 0x7f0b300cd420 2024-04-28T13:31:17.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.281+0000 7f0b36400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0b300b5960 con 0x7f0b300d1410 2024-04-28T13:31:17.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.281+0000 7f0b36400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0b30161ac0 con 0x7f0b301469f0 2024-04-28T13:31:17.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.282+0000 7f0b2fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0b300d1410 0x7f0b300b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:17.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.282+0000 7f0b2fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0b300d1410 0x7f0b300b5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:47994/0 (socket says 172.21.15.196:47994) 2024-04-28T13:31:17.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.282+0000 7f0b2fe00640 1 -- 172.21.15.196:0/3390189816 learned_addr learned my addr 172.21.15.196:0/3390189816 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:17.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.282+0000 7f0b27e00640 1 --1- 172.21.15.196:0/3390189816 >> v1:172.21.15.196:6789/0 conn(0x7f0b300cd420 0x7f0b300d0d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:57890/0 (socket says 172.21.15.196:57890) 2024-04-28T13:31:17.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.282+0000 7f0b2f400640 1 --2- 172.21.15.196:0/3390189816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b301469f0 0x7f0b300ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:17.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.282+0000 7f0b2fe00640 1 -- 172.21.15.196:0/3390189816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b301469f0 msgr2=0x7f0b300ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:17.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.282+0000 7f0b2fe00640 1 --2- 172.21.15.196:0/3390189816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b301469f0 0x7f0b300ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.282+0000 7f0b2fe00640 1 -- 172.21.15.196:0/3390189816 >> v1:172.21.15.196:6789/0 conn(0x7f0b300cd420 legacy=0x7f0b300d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:17.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.282+0000 7f0b2fe00640 1 -- 172.21.15.196:0/3390189816 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b30161220 con 0x7f0b300d1410 2024-04-28T13:31:17.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.282+0000 7f0b2ea00640 1 -- 172.21.15.196:0/3390189816 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2575049899 0 0) 0x7f0b30160f40 con 0x7f0b300cd420 2024-04-28T13:31:17.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.283+0000 7f0b2fe00640 1 --2- 172.21.15.196:0/3390189816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300d1410 0x7f0b300b5dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009e50 tx=0x7f0b2802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8f7e0f24fe25f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:17.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.283+0000 7f0b2ea00640 1 -- 172.21.15.196:0/3390189816 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28036070 con 0x7f0b300d1410 2024-04-28T13:31:17.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.284+0000 7f0b2ea00640 1 -- 172.21.15.196:0/3390189816 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b28031600 con 0x7f0b300d1410 2024-04-28T13:31:17.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.284+0000 7f0b2ea00640 1 -- 172.21.15.196:0/3390189816 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b280319d0 con 0x7f0b300d1410 2024-04-28T13:31:17.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.284+0000 7f0b36400640 1 -- 172.21.15.196:0/3390189816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300d1410 msgr2=0x7f0b300b5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:17.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.284+0000 7f0b36400640 1 --2- 172.21.15.196:0/3390189816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300d1410 0x7f0b300b5dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009e50 tx=0x7f0b2802a710 comp rx=0 tx=0).stop 2024-04-28T13:31:17.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.284+0000 7f0b36400640 1 -- 172.21.15.196:0/3390189816 shutdown_connections 2024-04-28T13:31:17.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.284+0000 7f0b36400640 1 --2- 172.21.15.196:0/3390189816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300d1410 0x7f0b300b5dd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.284+0000 7f0b36400640 1 --2- 172.21.15.196:0/3390189816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b301469f0 0x7f0b300ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.284+0000 7f0b36400640 1 -- 172.21.15.196:0/3390189816 >> 172.21.15.196:0/3390189816 conn(0x7f0b300cd830 msgr2=0x7f0b300cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:17.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.284+0000 7f0b36400640 1 -- 172.21.15.196:0/3390189816 shutdown_connections 2024-04-28T13:31:17.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.284+0000 7f0b36400640 1 -- 172.21.15.196:0/3390189816 wait complete. 2024-04-28T13:31:17.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.286+0000 7f0b36400640 1 Processor -- start 2024-04-28T13:31:17.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.286+0000 7f0b36400640 1 -- start start 2024-04-28T13:31:17.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.287+0000 7f0b36400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300cd420 0x7f0b3015a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:17.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.287+0000 7f0b36400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b300d1410 0x7f0b300b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:17.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.287+0000 7f0b36400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b301469f0 0x7f0b30158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:17.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.287+0000 7f0b36400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30072020 con 0x7f0b300cd420 2024-04-28T13:31:17.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.287+0000 7f0b36400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301617e0 con 0x7f0b300d1410 2024-04-28T13:31:17.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.287+0000 7f0b36400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0b30160b80 con 0x7f0b301469f0 2024-04-28T13:31:17.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.287+0000 7f0b2f400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300cd420 0x7f0b3015a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:17.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.288+0000 7f0b2f400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300cd420 0x7f0b3015a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:48002/0 (socket says 172.21.15.196:48002) 2024-04-28T13:31:17.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.288+0000 7f0b2f400640 1 -- 172.21.15.196:0/4052131092 learned_addr learned my addr 172.21.15.196:0/4052131092 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:17.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.288+0000 7f0b2f400640 1 -- 172.21.15.196:0/4052131092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b301469f0 msgr2=0x7f0b30158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:17.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.288+0000 7f0b2f400640 1 --2- 172.21.15.196:0/4052131092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b301469f0 0x7f0b30158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.288+0000 7f0b27e00640 1 --2- 172.21.15.196:0/4052131092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b300d1410 0x7f0b300b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:17.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.288+0000 7f0b2f400640 1 -- 172.21.15.196:0/4052131092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b300d1410 msgr2=0x7f0b300b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:17.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.288+0000 7f0b2f400640 1 --2- 172.21.15.196:0/4052131092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b300d1410 0x7f0b300b1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.288+0000 7f0b2f400640 1 -- 172.21.15.196:0/4052131092 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3015faf0 con 0x7f0b300cd420 2024-04-28T13:31:17.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.288+0000 7f0b2f400640 1 --2- 172.21.15.196:0/4052131092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300cd420 0x7f0b3015a3d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000d5f0 tx=0x7f0b2000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:17.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.288+0000 7f0b27400640 1 -- 172.21.15.196:0/4052131092 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b200125f0 con 0x7f0b300cd420 2024-04-28T13:31:17.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.289+0000 7f0b27400640 1 -- 172.21.15.196:0/4052131092 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b20019070 con 0x7f0b300cd420 2024-04-28T13:31:17.290 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.289+0000 7f0b36400640 1 -- 172.21.15.196:0/4052131092 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b30160060 con 0x7f0b300cd420 2024-04-28T13:31:17.290 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.289+0000 7f0b27400640 1 -- 172.21.15.196:0/4052131092 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b200129d0 con 0x7f0b300cd420 2024-04-28T13:31:17.290 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.289+0000 7f0b36400640 1 -- 172.21.15.196:0/4052131092 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b3015a910 con 0x7f0b300cd420 2024-04-28T13:31:17.290 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.290+0000 7f0b36400640 1 -- 172.21.15.196:0/4052131092 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b3006a290 con 0x7f0b300cd420 2024-04-28T13:31:17.290 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.291+0000 7f0b27400640 1 -- 172.21.15.196:0/4052131092 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b20012b70 con 0x7f0b300cd420 2024-04-28T13:31:17.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.291+0000 7f0b27400640 1 --2- 172.21.15.196:0/4052131092 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0b0c06c8f0 0x7f0b0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:17.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.292+0000 7f0b27400640 1 -- 172.21.15.196:0/4052131092 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f0b2008bca0 con 0x7f0b300cd420 2024-04-28T13:31:17.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.292+0000 7f0b27e00640 1 --2- 172.21.15.196:0/4052131092 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0b0c06c8f0 0x7f0b0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:17.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.293+0000 7f0b27e00640 1 --2- 172.21.15.196:0/4052131092 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0b0c06c8f0 0x7f0b0c06edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0b301f63e0 tx=0x7f0b18009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:17.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.296+0000 7f0b27400640 1 -- 172.21.15.196:0/4052131092 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b2005a380 con 0x7f0b300cd420 2024-04-28T13:31:17.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.537+0000 7f0b36400640 1 -- 172.21.15.196:0/4052131092 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b3006fd90 con 0x7f0b300cd420 2024-04-28T13:31:17.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.538+0000 7f0b27400640 1 -- 172.21.15.196:0/4052131092 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3873 (secure 0 0 0) 0x7f0b20059c80 con 0x7f0b300cd420 2024-04-28T13:31:17.538 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:17.539 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/422653807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":422653807},{"type":"v1","addr":"172.21.15.196:6834","nonce":422653807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2793552376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2793552376},{"type":"v1","addr":"172.21.15.196:6835","nonce":2793552376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1810956193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1810956193},{"type":"v1","addr":"172.21.15.203:6837","nonce":1810956193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2114798853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2114798853},{"type":"v1","addr":"172.21.15.203:6836","nonce":2114798853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-28T13:31:17.539 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 3 2024-04-28T13:31:17.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.541+0000 7f0b36400640 1 -- 172.21.15.196:0/4052131092 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0b0c06c8f0 msgr2=0x7f0b0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:17.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.541+0000 7f0b36400640 1 --2- 172.21.15.196:0/4052131092 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0b0c06c8f0 0x7f0b0c06edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0b301f63e0 tx=0x7f0b18009040 comp rx=0 tx=0).stop 2024-04-28T13:31:17.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.541+0000 7f0b36400640 1 -- 172.21.15.196:0/4052131092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300cd420 msgr2=0x7f0b3015a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:17.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.541+0000 7f0b36400640 1 --2- 172.21.15.196:0/4052131092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300cd420 0x7f0b3015a3d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000d5f0 tx=0x7f0b2000dab0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.541+0000 7f0b36400640 1 -- 172.21.15.196:0/4052131092 shutdown_connections 2024-04-28T13:31:17.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.541+0000 7f0b36400640 1 --2- 172.21.15.196:0/4052131092 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0b0c06c8f0 0x7f0b0c06edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.541+0000 7f0b36400640 1 --2- 172.21.15.196:0/4052131092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b301469f0 0x7f0b30158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.541+0000 7f0b36400640 1 --2- 172.21.15.196:0/4052131092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b300d1410 0x7f0b300b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.541+0000 7f0b36400640 1 --2- 172.21.15.196:0/4052131092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b300cd420 0x7f0b3015a3d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.541+0000 7f0b36400640 1 -- 172.21.15.196:0/4052131092 >> 172.21.15.196:0/4052131092 conn(0x7f0b300cd830 msgr2=0x7f0b30143fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:17.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.541+0000 7f0b36400640 1 -- 172.21.15.196:0/4052131092 shutdown_connections 2024-04-28T13:31:17.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.541+0000 7f0b36400640 1 -- 172.21.15.196:0/4052131092 wait complete. 2024-04-28T13:31:17.560 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:17.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.743+0000 7f9abd200640 1 Processor -- start 2024-04-28T13:31:17.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.743+0000 7f9abd200640 1 -- start start 2024-04-28T13:31:17.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.745+0000 7f9abd200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9ab815ef00 0x7f9ab80b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:17.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.745+0000 7f9abd200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9ab815f2e0 0x7f9ab806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:17.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.745+0000 7f9abd200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ab806fce0 con 0x7f9ab80d1050 2024-04-28T13:31:17.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.745+0000 7f9abd200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9ab806dc70 con 0x7f9ab815f2e0 2024-04-28T13:31:17.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.745+0000 7f9abd200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9ab806df50 con 0x7f9ab815ef00 2024-04-28T13:31:17.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.745+0000 7f9ab6a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9ab815f2e0 0x7f9ab806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:17.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.745+0000 7f9ab6a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9ab815f2e0 0x7f9ab806f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:48004/0 (socket says 172.21.15.196:48004) 2024-04-28T13:31:17.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.745+0000 7f9ab6a00640 1 -- 172.21.15.196:0/3321113289 learned_addr learned my addr 172.21.15.196:0/3321113289 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:17.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.745+0000 7f9ab7400640 1 --2- 172.21.15.196:0/3321113289 >> v2:172.21.15.203:3300/0 conn(0x7f9ab815ef00 0x7f9ab80b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:17.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.745+0000 7f9ab6a00640 1 -- 172.21.15.196:0/3321113289 >> v2:172.21.15.203:3300/0 conn(0x7f9ab815ef00 msgr2=0x7f9ab80b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:17.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.745+0000 7f9ab6a00640 1 --2- 172.21.15.196:0/3321113289 >> v2:172.21.15.203:3300/0 conn(0x7f9ab815ef00 0x7f9ab80b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.745+0000 7f9ab6a00640 1 -- 172.21.15.196:0/3321113289 >> v1:172.21.15.196:6789/0 conn(0x7f9ab80d1050 legacy=0x7f9ab8162c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:17.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.745+0000 7f9ab6a00640 1 -- 172.21.15.196:0/3321113289 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab806e230 con 0x7f9ab815f2e0 2024-04-28T13:31:17.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.745+0000 7f9ab7400640 1 --2- 172.21.15.196:0/3321113289 >> v2:172.21.15.203:3300/0 conn(0x7f9ab815ef00 0x7f9ab80b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:17.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.746+0000 7f9ab6a00640 1 --2- 172.21.15.196:0/3321113289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab815f2e0 0x7f9ab806f7a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8068fc0 tx=0x7f9aa402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1d3976e0dde135 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:17.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.747+0000 7f9ab6000640 1 -- 172.21.15.196:0/3321113289 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4037070 con 0x7f9ab815f2e0 2024-04-28T13:31:17.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.747+0000 7f9ab6000640 1 -- 172.21.15.196:0/3321113289 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa4032440 con 0x7f9ab815f2e0 2024-04-28T13:31:17.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.747+0000 7f9ab6000640 1 -- 172.21.15.196:0/3321113289 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4032810 con 0x7f9ab815f2e0 2024-04-28T13:31:17.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.747+0000 7f9abd200640 1 -- 172.21.15.196:0/3321113289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab815f2e0 msgr2=0x7f9ab806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:17.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.747+0000 7f9abd200640 1 --2- 172.21.15.196:0/3321113289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab815f2e0 0x7f9ab806f7a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8068fc0 tx=0x7f9aa402a1f0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.747+0000 7f9abd200640 1 -- 172.21.15.196:0/3321113289 shutdown_connections 2024-04-28T13:31:17.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.747+0000 7f9abd200640 1 --2- 172.21.15.196:0/3321113289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab815f2e0 0x7f9ab806f7a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.747+0000 7f9abd200640 1 --2- 172.21.15.196:0/3321113289 >> v2:172.21.15.203:3300/0 conn(0x7f9ab815ef00 0x7f9ab80b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.747+0000 7f9abd200640 1 -- 172.21.15.196:0/3321113289 >> 172.21.15.196:0/3321113289 conn(0x7f9ab806c6c0 msgr2=0x7f9ab806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:17.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.748+0000 7f9abd200640 1 -- 172.21.15.196:0/3321113289 shutdown_connections 2024-04-28T13:31:17.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.748+0000 7f9abd200640 1 -- 172.21.15.196:0/3321113289 wait complete. 2024-04-28T13:31:17.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.750+0000 7f9abd200640 1 Processor -- start 2024-04-28T13:31:17.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.750+0000 7f9abd200640 1 -- start start 2024-04-28T13:31:17.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.751+0000 7f9abd200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80d1050 0x7f9ab809e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:17.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.751+0000 7f9abd200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab815ef00 0x7f9ab815f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:17.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.751+0000 7f9abd200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab815f2e0 0x7f9ab8155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:17.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.751+0000 7f9abd200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab8073290 con 0x7f9ab815ef00 2024-04-28T13:31:17.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.751+0000 7f9abd200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab8073110 con 0x7f9ab815f2e0 2024-04-28T13:31:17.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.751+0000 7f9abd200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab80d14c0 con 0x7f9ab80d1050 2024-04-28T13:31:17.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.751+0000 7f9ab6a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab815ef00 0x7f9ab815f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:17.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.751+0000 7f9ab6a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab815ef00 0x7f9ab815f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:48012/0 (socket says 172.21.15.196:48012) 2024-04-28T13:31:17.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.751+0000 7f9ab6a00640 1 -- 172.21.15.196:0/1104312403 learned_addr learned my addr 172.21.15.196:0/1104312403 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:17.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.752+0000 7f9ab7400640 1 --2- 172.21.15.196:0/1104312403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80d1050 0x7f9ab809e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:17.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.752+0000 7f9ab6a00640 1 -- 172.21.15.196:0/1104312403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80d1050 msgr2=0x7f9ab809e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:17.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.752+0000 7f9ab6a00640 1 --2- 172.21.15.196:0/1104312403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80d1050 0x7f9ab809e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.752+0000 7f9ab7e00640 1 --2- 172.21.15.196:0/1104312403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab815f2e0 0x7f9ab8155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:17.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.752+0000 7f9ab6a00640 1 -- 172.21.15.196:0/1104312403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab815f2e0 msgr2=0x7f9ab8155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:17.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.752+0000 7f9ab6a00640 1 --2- 172.21.15.196:0/1104312403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab815f2e0 0x7f9ab8155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.752+0000 7f9ab6a00640 1 -- 172.21.15.196:0/1104312403 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab809d610 con 0x7f9ab815ef00 2024-04-28T13:31:17.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.752+0000 7f9ab6a00640 1 --2- 172.21.15.196:0/1104312403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab815ef00 0x7f9ab815f6f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4000c00 tx=0x7f9aa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:17.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.752+0000 7f9aaf400640 1 -- 172.21.15.196:0/1104312403 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4037040 con 0x7f9ab815ef00 2024-04-28T13:31:17.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.752+0000 7f9aaf400640 1 -- 172.21.15.196:0/1104312403 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa4042070 con 0x7f9ab815ef00 2024-04-28T13:31:17.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.752+0000 7f9aaf400640 1 -- 172.21.15.196:0/1104312403 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4030bb0 con 0x7f9ab815ef00 2024-04-28T13:31:17.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.752+0000 7f9abd200640 1 -- 172.21.15.196:0/1104312403 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab809d330 con 0x7f9ab815ef00 2024-04-28T13:31:17.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.753+0000 7f9abd200640 1 -- 172.21.15.196:0/1104312403 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab809dd30 con 0x7f9ab815ef00 2024-04-28T13:31:17.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.754+0000 7f9abd200640 1 -- 172.21.15.196:0/1104312403 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab806a290 con 0x7f9ab815ef00 2024-04-28T13:31:17.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.754+0000 7f9aaf400640 1 -- 172.21.15.196:0/1104312403 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9aa4047460 con 0x7f9ab815ef00 2024-04-28T13:31:17.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.755+0000 7f9aaf400640 1 --2- 172.21.15.196:0/1104312403 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a9806c8f0 0x7f9a9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:17.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.756+0000 7f9aaf400640 1 -- 172.21.15.196:0/1104312403 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f9aa40ac3e0 con 0x7f9ab815ef00 2024-04-28T13:31:17.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.756+0000 7f9ab7400640 1 --2- 172.21.15.196:0/1104312403 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a9806c8f0 0x7f9a9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:17.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.757+0000 7f9ab7400640 1 --2- 172.21.15.196:0/1104312403 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a9806c8f0 0x7f9a9806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8160550 tx=0x7f9aa8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:17.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.760+0000 7f9aaf400640 1 -- 172.21.15.196:0/1104312403 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9aa4045360 con 0x7f9ab815ef00 2024-04-28T13:31:17.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.960+0000 7f9abd200640 1 -- 172.21.15.196:0/1104312403 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ab81629e0 con 0x7f9ab815ef00 2024-04-28T13:31:17.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.962+0000 7f9aaf400640 1 -- 172.21.15.196:0/1104312403 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7f9aa407abf0 con 0x7f9ab815ef00 2024-04-28T13:31:17.962 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:17.963 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":16,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:12.469573+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:17.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.965+0000 7f9abd200640 1 -- 172.21.15.196:0/1104312403 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a9806c8f0 msgr2=0x7f9a9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:17.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.965+0000 7f9abd200640 1 --2- 172.21.15.196:0/1104312403 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a9806c8f0 0x7f9a9806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8160550 tx=0x7f9aa8008040 comp rx=0 tx=0).stop 2024-04-28T13:31:17.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.965+0000 7f9abd200640 1 -- 172.21.15.196:0/1104312403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab815ef00 msgr2=0x7f9ab815f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:17.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.965+0000 7f9abd200640 1 --2- 172.21.15.196:0/1104312403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab815ef00 0x7f9ab815f6f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4000c00 tx=0x7f9aa40046c0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.965+0000 7f9abd200640 1 -- 172.21.15.196:0/1104312403 shutdown_connections 2024-04-28T13:31:17.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.965+0000 7f9abd200640 1 --2- 172.21.15.196:0/1104312403 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a9806c8f0 0x7f9a9806edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.965+0000 7f9abd200640 1 --2- 172.21.15.196:0/1104312403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab815f2e0 0x7f9ab8155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.965+0000 7f9abd200640 1 --2- 172.21.15.196:0/1104312403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab815ef00 0x7f9ab815f6f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.965+0000 7f9abd200640 1 --2- 172.21.15.196:0/1104312403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80d1050 0x7f9ab809e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:17.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.965+0000 7f9abd200640 1 -- 172.21.15.196:0/1104312403 >> 172.21.15.196:0/1104312403 conn(0x7f9ab806c6c0 msgr2=0x7f9ab80b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:17.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.965+0000 7f9abd200640 1 -- 172.21.15.196:0/1104312403 shutdown_connections 2024-04-28T13:31:17.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:17.965+0000 7f9abd200640 1 -- 172.21.15.196:0/1104312403 wait complete. 2024-04-28T13:31:17.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:31:17.984 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:31:18.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.183+0000 7f32b0e00640 1 Processor -- start 2024-04-28T13:31:18.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.183+0000 7f32b0e00640 1 -- start start 2024-04-28T13:31:18.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.184+0000 7f32b0e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f32ac1599f0 0x7f32ac160370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:18.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.184+0000 7f32b0e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f32ac0b0db0 0x7f32ac0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:18.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.184+0000 7f32b0e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32ac159dd0 con 0x7f32ac0b6ed0 2024-04-28T13:31:18.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.184+0000 7f32b0e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f32ac15f8f0 con 0x7f32ac0b0db0 2024-04-28T13:31:18.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.184+0000 7f32b0e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f32ac1608b0 con 0x7f32ac1599f0 2024-04-28T13:31:18.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.185+0000 7f32aa800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f32ac0b0db0 0x7f32ac0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:18.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.185+0000 7f32aa800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f32ac0b0db0 0x7f32ac0b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:48014/0 (socket says 172.21.15.196:48014) 2024-04-28T13:31:18.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.185+0000 7f32aa800640 1 -- 172.21.15.196:0/1988191517 learned_addr learned my addr 172.21.15.196:0/1988191517 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:18.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.185+0000 7f32a9e00640 1 --2- 172.21.15.196:0/1988191517 >> v2:172.21.15.203:3300/0 conn(0x7f32ac1599f0 0x7f32ac160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:18.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.185+0000 7f32aa800640 1 -- 172.21.15.196:0/1988191517 >> v2:172.21.15.203:3300/0 conn(0x7f32ac1599f0 msgr2=0x7f32ac160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:18.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.185+0000 7f32aa800640 1 --2- 172.21.15.196:0/1988191517 >> v2:172.21.15.203:3300/0 conn(0x7f32ac1599f0 0x7f32ac160370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.185+0000 7f32aa800640 1 -- 172.21.15.196:0/1988191517 >> v1:172.21.15.203:6789/0 conn(0x7f32ac0b6ed0 legacy=0x7f32ac06f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:18.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.185+0000 7f32aa800640 1 -- 172.21.15.196:0/1988191517 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac0b1700 con 0x7f32ac0b0db0 2024-04-28T13:31:18.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.186+0000 7f32aa800640 1 --2- 172.21.15.196:0/1988191517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32ac0b0db0 0x7f32ac0b11c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0b49c0 tx=0x7f32a402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb2c0897a3368e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:18.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.187+0000 7f32a8a00640 1 -- 172.21.15.196:0/1988191517 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4036070 con 0x7f32ac0b0db0 2024-04-28T13:31:18.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.187+0000 7f32a8a00640 1 -- 172.21.15.196:0/1988191517 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a4031600 con 0x7f32ac0b0db0 2024-04-28T13:31:18.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.187+0000 7f32a8a00640 1 -- 172.21.15.196:0/1988191517 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a40319d0 con 0x7f32ac0b0db0 2024-04-28T13:31:18.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.187+0000 7f32b0e00640 1 -- 172.21.15.196:0/1988191517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32ac0b0db0 msgr2=0x7f32ac0b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:18.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.187+0000 7f32b0e00640 1 --2- 172.21.15.196:0/1988191517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32ac0b0db0 0x7f32ac0b11c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0b49c0 tx=0x7f32a402a6f0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.187+0000 7f32b0e00640 1 -- 172.21.15.196:0/1988191517 shutdown_connections 2024-04-28T13:31:18.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.187+0000 7f32b0e00640 1 --2- 172.21.15.196:0/1988191517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32ac0b0db0 0x7f32ac0b11c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.187+0000 7f32b0e00640 1 --2- 172.21.15.196:0/1988191517 >> v2:172.21.15.203:3300/0 conn(0x7f32ac1599f0 0x7f32ac160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.187+0000 7f32b0e00640 1 -- 172.21.15.196:0/1988191517 >> 172.21.15.196:0/1988191517 conn(0x7f32ac0b3b60 msgr2=0x7f32ac0b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:18.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.188+0000 7f32b0e00640 1 -- 172.21.15.196:0/1988191517 shutdown_connections 2024-04-28T13:31:18.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.188+0000 7f32b0e00640 1 -- 172.21.15.196:0/1988191517 wait complete. 2024-04-28T13:31:18.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.189+0000 7f32b0e00640 1 Processor -- start 2024-04-28T13:31:18.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.189+0000 7f32b0e00640 1 -- start start 2024-04-28T13:31:18.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.190+0000 7f32b0e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32ac0b0db0 0x7f32ac147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:18.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.190+0000 7f32b0e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ac0b6ed0 0x7f32ac0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:18.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.190+0000 7f32b0e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32ac1599f0 0x7f32ac149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:18.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.190+0000 7f32b0e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac15fb70 con 0x7f32ac0b0db0 2024-04-28T13:31:18.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.190+0000 7f32b0e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac070250 con 0x7f32ac0b6ed0 2024-04-28T13:31:18.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.190+0000 7f32b0e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f32ac15fe50 con 0x7f32ac1599f0 2024-04-28T13:31:18.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.191+0000 7f32a9e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32ac0b0db0 0x7f32ac147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:18.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.191+0000 7f32a9e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32ac0b0db0 0x7f32ac147630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:48028/0 (socket says 172.21.15.196:48028) 2024-04-28T13:31:18.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.191+0000 7f32a9e00640 1 -- 172.21.15.196:0/3783429075 learned_addr learned my addr 172.21.15.196:0/3783429075 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:18.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.191+0000 7f32aa800640 1 --2- 172.21.15.196:0/3783429075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32ac1599f0 0x7f32ac149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:18.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.191+0000 7f32a9400640 1 --2- 172.21.15.196:0/3783429075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ac0b6ed0 0x7f32ac0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:18.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.191+0000 7f32a9e00640 1 -- 172.21.15.196:0/3783429075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32ac1599f0 msgr2=0x7f32ac149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:18.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.191+0000 7f32a9e00640 1 --2- 172.21.15.196:0/3783429075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32ac1599f0 0x7f32ac149df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.191+0000 7f32a9e00640 1 -- 172.21.15.196:0/3783429075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ac0b6ed0 msgr2=0x7f32ac0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:18.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.191+0000 7f32a9e00640 1 --2- 172.21.15.196:0/3783429075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ac0b6ed0 0x7f32ac0c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.191+0000 7f32a9e00640 1 -- 172.21.15.196:0/3783429075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac14b2f0 con 0x7f32ac0b0db0 2024-04-28T13:31:18.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.191+0000 7f32a9e00640 1 --2- 172.21.15.196:0/3783429075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32ac0b0db0 0x7f32ac147630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f329c00e870 tx=0x7f329c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:18.193 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.191+0000 7f32a2000640 1 -- 172.21.15.196:0/3783429075 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c017070 con 0x7f32ac0b0db0 2024-04-28T13:31:18.193 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.192+0000 7f32a2000640 1 -- 172.21.15.196:0/3783429075 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329c00c8e0 con 0x7f32ac0b0db0 2024-04-28T13:31:18.193 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.192+0000 7f32b0e00640 1 -- 172.21.15.196:0/3783429075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac14b5d0 con 0x7f32ac0b0db0 2024-04-28T13:31:18.193 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.192+0000 7f32a2000640 1 -- 172.21.15.196:0/3783429075 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c00ccd0 con 0x7f32ac0b0db0 2024-04-28T13:31:18.193 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.192+0000 7f32b0e00640 1 -- 172.21.15.196:0/3783429075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac14bb90 con 0x7f32ac0b0db0 2024-04-28T13:31:18.193 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.193+0000 7f32b0e00640 1 -- 172.21.15.196:0/3783429075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ac0693d0 con 0x7f32ac0b0db0 2024-04-28T13:31:18.193 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.194+0000 7f32a2000640 1 -- 172.21.15.196:0/3783429075 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f329c01a070 con 0x7f32ac0b0db0 2024-04-28T13:31:18.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.195+0000 7f32a2000640 1 --2- 172.21.15.196:0/3783429075 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f328806c8f0 0x7f328806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:18.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.196+0000 7f32a2000640 1 -- 172.21.15.196:0/3783429075 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f329c08ba90 con 0x7f32ac0b0db0 2024-04-28T13:31:18.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.196+0000 7f32a9400640 1 --2- 172.21.15.196:0/3783429075 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f328806c8f0 0x7f328806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:18.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.197+0000 7f32a9400640 1 --2- 172.21.15.196:0/3783429075 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f328806c8f0 0x7f328806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f32ac1f8440 tx=0x7f3294009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:18.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.200+0000 7f32a2000640 1 -- 172.21.15.196:0/3783429075 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f329c05a170 con 0x7f32ac0b0db0 2024-04-28T13:31:18.400 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.400+0000 7f32b0e00640 1 -- 172.21.15.196:0/3783429075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32ac0b5270 con 0x7f32ac0b0db0 2024-04-28T13:31:18.400 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.400+0000 7f32a2000640 1 -- 172.21.15.196:0/3783429075 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f329c059a70 con 0x7f32ac0b0db0 2024-04-28T13:31:18.400 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:31:18.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.403+0000 7f32b0e00640 1 -- 172.21.15.196:0/3783429075 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f328806c8f0 msgr2=0x7f328806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:18.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.403+0000 7f32b0e00640 1 --2- 172.21.15.196:0/3783429075 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f328806c8f0 0x7f328806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f32ac1f8440 tx=0x7f3294009040 comp rx=0 tx=0).stop 2024-04-28T13:31:18.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.403+0000 7f32b0e00640 1 -- 172.21.15.196:0/3783429075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32ac0b0db0 msgr2=0x7f32ac147630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:18.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.403+0000 7f32b0e00640 1 --2- 172.21.15.196:0/3783429075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32ac0b0db0 0x7f32ac147630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f329c00e870 tx=0x7f329c00ed30 comp rx=0 tx=0).stop 2024-04-28T13:31:18.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.403+0000 7f32b0e00640 1 -- 172.21.15.196:0/3783429075 shutdown_connections 2024-04-28T13:31:18.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.403+0000 7f32b0e00640 1 --2- 172.21.15.196:0/3783429075 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f328806c8f0 0x7f328806edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.403+0000 7f32b0e00640 1 --2- 172.21.15.196:0/3783429075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32ac1599f0 0x7f32ac149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.403+0000 7f32b0e00640 1 --2- 172.21.15.196:0/3783429075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ac0b6ed0 0x7f32ac0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.403+0000 7f32b0e00640 1 --2- 172.21.15.196:0/3783429075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32ac0b0db0 0x7f32ac147630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.403+0000 7f32b0e00640 1 -- 172.21.15.196:0/3783429075 >> 172.21.15.196:0/3783429075 conn(0x7f32ac0b3b60 msgr2=0x7f32ac160bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:18.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.403+0000 7f32b0e00640 1 -- 172.21.15.196:0/3783429075 shutdown_connections 2024-04-28T13:31:18.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.403+0000 7f32b0e00640 1 -- 172.21.15.196:0/3783429075 wait complete. 2024-04-28T13:31:18.423 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-28T13:31:18.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.614+0000 7f9682000640 1 Processor -- start 2024-04-28T13:31:18.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.614+0000 7f9682000640 1 -- start start 2024-04-28T13:31:18.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.616+0000 7f9682000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f967c146790 0x7f967c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:18.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.616+0000 7f9682000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f967c0d05b0 con 0x7f967c140350 2024-04-28T13:31:18.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.616+0000 7f9682000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f967c0d0890 con 0x7f967c13ff70 2024-04-28T13:31:18.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.616+0000 7f9682000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f967c06e450 con 0x7f967c146790 2024-04-28T13:31:18.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.616+0000 7f967be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f967c146790 0x7f967c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:18.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.616+0000 7f9672a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f967c140350 0x7f967c0cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:57904/0 (socket says 172.21.15.196:57904) 2024-04-28T13:31:18.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.616+0000 7f967be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f967c146790 0x7f967c0cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:48044/0 (socket says 172.21.15.196:48044) 2024-04-28T13:31:18.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.616+0000 7f9672a00640 1 -- 172.21.15.196:0/3774784935 learned_addr learned my addr 172.21.15.196:0/3774784935 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:18.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.617+0000 7f967be00640 1 -- 172.21.15.196:0/3774784935 >> v1:172.21.15.203:6789/0 conn(0x7f967c13ff70 legacy=0x7f967c146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:18.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.617+0000 7f967be00640 1 -- 172.21.15.196:0/3774784935 >> v1:172.21.15.196:6789/0 conn(0x7f967c140350 legacy=0x7f967c0cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:18.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.617+0000 7f967be00640 1 -- 172.21.15.196:0/3774784935 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f967c06e730 con 0x7f967c146790 2024-04-28T13:31:18.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.618+0000 7f967be00640 1 --2- 172.21.15.196:0/3774784935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f967c146790 0x7f967c0cff20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f96740097d0 tx=0x7f967402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba62649039e9cfa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:18.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.618+0000 7f967aa00640 1 -- 172.21.15.196:0/3774784935 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674037070 con 0x7f967c146790 2024-04-28T13:31:18.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.618+0000 7f967aa00640 1 -- 172.21.15.196:0/3774784935 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9674032440 con 0x7f967c146790 2024-04-28T13:31:18.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.618+0000 7f967aa00640 1 -- 172.21.15.196:0/3774784935 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674032810 con 0x7f967c146790 2024-04-28T13:31:18.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.619+0000 7f9682000640 1 -- 172.21.15.196:0/3774784935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f967c146790 msgr2=0x7f967c0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:18.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.619+0000 7f9682000640 1 --2- 172.21.15.196:0/3774784935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f967c146790 0x7f967c0cff20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f96740097d0 tx=0x7f967402a180 comp rx=0 tx=0).stop 2024-04-28T13:31:18.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.619+0000 7f9682000640 1 -- 172.21.15.196:0/3774784935 shutdown_connections 2024-04-28T13:31:18.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.619+0000 7f9682000640 1 --2- 172.21.15.196:0/3774784935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f967c146790 0x7f967c0cff20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.619+0000 7f9682000640 1 -- 172.21.15.196:0/3774784935 >> 172.21.15.196:0/3774784935 conn(0x7f967c0d0d30 msgr2=0x7f967c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:18.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.619+0000 7f9682000640 1 -- 172.21.15.196:0/3774784935 shutdown_connections 2024-04-28T13:31:18.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.619+0000 7f9682000640 1 -- 172.21.15.196:0/3774784935 wait complete. 2024-04-28T13:31:18.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.620+0000 7f9682000640 1 Processor -- start 2024-04-28T13:31:18.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.621+0000 7f9682000640 1 -- start start 2024-04-28T13:31:18.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.621+0000 7f9682000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f967c13ff70 0x7f967c15a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:18.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.621+0000 7f9682000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f967c140350 0x7f967c0b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:18.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.621+0000 7f9682000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f967c146790 0x7f967c0b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:18.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.621+0000 7f9682000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f967c1681c0 con 0x7f967c146790 2024-04-28T13:31:18.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.621+0000 7f9682000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f967c168040 con 0x7f967c140350 2024-04-28T13:31:18.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.621+0000 7f9682000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f967c168340 con 0x7f967c13ff70 2024-04-28T13:31:18.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.622+0000 7f967be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f967c146790 0x7f967c0b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:18.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.622+0000 7f967be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f967c146790 0x7f967c0b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:48052/0 (socket says 172.21.15.196:48052) 2024-04-28T13:31:18.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.622+0000 7f967be00640 1 -- 172.21.15.196:0/4161710371 learned_addr learned my addr 172.21.15.196:0/4161710371 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:18.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.622+0000 7f9672a00640 1 --2- 172.21.15.196:0/4161710371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f967c140350 0x7f967c0b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:18.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.622+0000 7f967be00640 1 -- 172.21.15.196:0/4161710371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f967c13ff70 msgr2=0x7f967c15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:18.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.622+0000 7f967be00640 1 --2- 172.21.15.196:0/4161710371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f967c13ff70 0x7f967c15a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.622+0000 7f967be00640 1 -- 172.21.15.196:0/4161710371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f967c140350 msgr2=0x7f967c0b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:18.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.622+0000 7f967be00640 1 --2- 172.21.15.196:0/4161710371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f967c140350 0x7f967c0b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.622+0000 7f967be00640 1 -- 172.21.15.196:0/4161710371 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f967c15be90 con 0x7f967c146790 2024-04-28T13:31:18.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.622+0000 7f9672a00640 1 --2- 172.21.15.196:0/4161710371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f967c140350 0x7f967c0b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:18.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.623+0000 7f967be00640 1 --2- 172.21.15.196:0/4161710371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f967c146790 0x7f967c0b5d90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9674009720 tx=0x7f96740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:18.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.623+0000 7f9673e00640 1 -- 172.21.15.196:0/4161710371 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674037040 con 0x7f967c146790 2024-04-28T13:31:18.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.623+0000 7f9673e00640 1 -- 172.21.15.196:0/4161710371 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9674004230 con 0x7f967c146790 2024-04-28T13:31:18.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.623+0000 7f9673e00640 1 -- 172.21.15.196:0/4161710371 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674032e20 con 0x7f967c146790 2024-04-28T13:31:18.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.623+0000 7f9682000640 1 -- 172.21.15.196:0/4161710371 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f967c15d980 con 0x7f967c146790 2024-04-28T13:31:18.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.623+0000 7f9682000640 1 -- 172.21.15.196:0/4161710371 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f967c15df40 con 0x7f967c146790 2024-04-28T13:31:18.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.624+0000 7f9682000640 1 -- 172.21.15.196:0/4161710371 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f967c06a290 con 0x7f967c146790 2024-04-28T13:31:18.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.625+0000 7f9673e00640 1 -- 172.21.15.196:0/4161710371 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96740306f0 con 0x7f967c146790 2024-04-28T13:31:18.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.626+0000 7f9673e00640 1 --2- 172.21.15.196:0/4161710371 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f965806c8f0 0x7f965806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:18.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.626+0000 7f9673e00640 1 -- 172.21.15.196:0/4161710371 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f96740ac780 con 0x7f967c146790 2024-04-28T13:31:18.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.626+0000 7f967b400640 1 --2- 172.21.15.196:0/4161710371 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f965806c8f0 0x7f965806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:18.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.627+0000 7f967b400640 1 --2- 172.21.15.196:0/4161710371 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f965806c8f0 0x7f965806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f967c0ce5b0 tx=0x7f966c007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:18.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.630+0000 7f9673e00640 1 -- 172.21.15.196:0/4161710371 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f967407ae60 con 0x7f967c146790 2024-04-28T13:31:18.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.828+0000 7f9682000640 1 -- 172.21.15.196:0/4161710371 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f967c0b45e0 con 0x7f967c146790 2024-04-28T13:31:18.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.829+0000 7f9673e00640 1 -- 172.21.15.196:0/4161710371 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f967407a760 con 0x7f967c146790 2024-04-28T13:31:18.829 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:18.829 INFO:teuthology.orchestra.run.smithi196.stdout:[ 2024-04-28T13:31:18.829 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:31:18.829 INFO:teuthology.orchestra.run.smithi196.stdout: "poolnum": 1, 2024-04-28T13:31:18.829 INFO:teuthology.orchestra.run.smithi196.stdout: "poolname": ".mgr" 2024-04-28T13:31:18.829 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:31:18.829 INFO:teuthology.orchestra.run.smithi196.stdout:] 2024-04-28T13:31:18.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.832+0000 7f9682000640 1 -- 172.21.15.196:0/4161710371 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f965806c8f0 msgr2=0x7f965806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:18.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.832+0000 7f9682000640 1 --2- 172.21.15.196:0/4161710371 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f965806c8f0 0x7f965806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f967c0ce5b0 tx=0x7f966c007d30 comp rx=0 tx=0).stop 2024-04-28T13:31:18.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.832+0000 7f9682000640 1 -- 172.21.15.196:0/4161710371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f967c146790 msgr2=0x7f967c0b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:18.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.832+0000 7f9682000640 1 --2- 172.21.15.196:0/4161710371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f967c146790 0x7f967c0b5d90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9674009720 tx=0x7f96740046c0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.832+0000 7f9682000640 1 -- 172.21.15.196:0/4161710371 shutdown_connections 2024-04-28T13:31:18.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.832+0000 7f9682000640 1 --2- 172.21.15.196:0/4161710371 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f965806c8f0 0x7f965806edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.832+0000 7f9682000640 1 --2- 172.21.15.196:0/4161710371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f967c146790 0x7f967c0b5d90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.832+0000 7f9682000640 1 --2- 172.21.15.196:0/4161710371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f967c140350 0x7f967c0b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.832+0000 7f9682000640 1 --2- 172.21.15.196:0/4161710371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f967c13ff70 0x7f967c15a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:18.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.832+0000 7f9682000640 1 -- 172.21.15.196:0/4161710371 >> 172.21.15.196:0/4161710371 conn(0x7f967c0d0d30 msgr2=0x7f967c0b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:18.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.832+0000 7f9682000640 1 -- 172.21.15.196:0/4161710371 shutdown_connections 2024-04-28T13:31:18.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:18.832+0000 7f9682000640 1 -- 172.21.15.196:0/4161710371 wait complete. 2024-04-28T13:31:18.851 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-28T13:31:18.851 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-28T13:31:19.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.044+0000 7f0317200640 1 Processor -- start 2024-04-28T13:31:19.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.044+0000 7f0317200640 1 -- start start 2024-04-28T13:31:19.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.046+0000 7f0317200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03180d1760 0x7f03180b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:19.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.046+0000 7f0317200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f031809ec80 0x7f031806df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:19.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.046+0000 7f0317200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f031815e380 con 0x7f03180b5ee0 2024-04-28T13:31:19.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.047+0000 7f0317200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f03180b4b20 con 0x7f031809ec80 2024-04-28T13:31:19.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.047+0000 7f0317200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f03180b3f30 con 0x7f03180d1760 2024-04-28T13:31:19.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.047+0000 7f0315e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f03180b5ee0 0x7f03180d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:57920/0 (socket says 172.21.15.196:57920) 2024-04-28T13:31:19.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.047+0000 7f0315e00640 1 -- 172.21.15.196:0/79886806 learned_addr learned my addr 172.21.15.196:0/79886806 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:19.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.047+0000 7f0316800640 1 --2- 172.21.15.196:0/79886806 >> v2:172.21.15.196:3300/0 conn(0x7f031809ec80 0x7f031806df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:19.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.047+0000 7f0314a00640 1 -- 172.21.15.196:0/79886806 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2515326095 0 0) 0x7f031815e380 con 0x7f03180b5ee0 2024-04-28T13:31:19.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.047+0000 7f0314a00640 1 -- 172.21.15.196:0/79886806 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02fc003520 con 0x7f03180b5ee0 2024-04-28T13:31:19.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.047+0000 7f0315400640 1 --2- 172.21.15.196:0/79886806 >> v2:172.21.15.203:3300/0 conn(0x7f03180d1760 0x7f03180b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:19.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.047+0000 7f0316800640 1 -- 172.21.15.196:0/79886806 >> v2:172.21.15.203:3300/0 conn(0x7f03180d1760 msgr2=0x7f03180b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:19.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.047+0000 7f0316800640 1 --2- 172.21.15.196:0/79886806 >> v2:172.21.15.203:3300/0 conn(0x7f03180d1760 0x7f03180b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:19.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.047+0000 7f0316800640 1 -- 172.21.15.196:0/79886806 >> v1:172.21.15.196:6789/0 conn(0x7f03180b5ee0 legacy=0x7f03180d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:19.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.047+0000 7f0316800640 1 -- 172.21.15.196:0/79886806 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03180b41f0 con 0x7f031809ec80 2024-04-28T13:31:19.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.048+0000 7f0316800640 1 --2- 172.21.15.196:0/79886806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031809ec80 0x7f031806df50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f03180cd5d0 tx=0x7f031002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98007db6307b21b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:19.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.049+0000 7f0314a00640 1 -- 172.21.15.196:0/79886806 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310007740 con 0x7f031809ec80 2024-04-28T13:31:19.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.049+0000 7f0314a00640 1 -- 172.21.15.196:0/79886806 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03100078e0 con 0x7f031809ec80 2024-04-28T13:31:19.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.049+0000 7f0314a00640 1 -- 172.21.15.196:0/79886806 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310007cb0 con 0x7f031809ec80 2024-04-28T13:31:19.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.049+0000 7f0317200640 1 -- 172.21.15.196:0/79886806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031809ec80 msgr2=0x7f031806df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:19.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.049+0000 7f0317200640 1 --2- 172.21.15.196:0/79886806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031809ec80 0x7f031806df50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f03180cd5d0 tx=0x7f031002a680 comp rx=0 tx=0).stop 2024-04-28T13:31:19.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.049+0000 7f0317200640 1 -- 172.21.15.196:0/79886806 shutdown_connections 2024-04-28T13:31:19.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.049+0000 7f0317200640 1 --2- 172.21.15.196:0/79886806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031809ec80 0x7f031806df50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:19.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.049+0000 7f0317200640 1 --2- 172.21.15.196:0/79886806 >> v2:172.21.15.203:3300/0 conn(0x7f03180d1760 0x7f03180b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:19.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.049+0000 7f0317200640 1 -- 172.21.15.196:0/79886806 >> 172.21.15.196:0/79886806 conn(0x7f031806c6c0 msgr2=0x7f031806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:19.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.049+0000 7f0317200640 1 -- 172.21.15.196:0/79886806 shutdown_connections 2024-04-28T13:31:19.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.049+0000 7f0317200640 1 -- 172.21.15.196:0/79886806 wait complete. 2024-04-28T13:31:19.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.051+0000 7f0317200640 1 Processor -- start 2024-04-28T13:31:19.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.051+0000 7f0317200640 1 -- start start 2024-04-28T13:31:19.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.052+0000 7f0317200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f031809ec80 0x7f031814fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:19.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.052+0000 7f0317200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03180b5ee0 0x7f03180cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:19.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.052+0000 7f0317200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03180d1760 0x7f031815f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:19.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.052+0000 7f0317200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03180b3c20 con 0x7f03180d1760 2024-04-28T13:31:19.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.052+0000 7f0317200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f03180b47b0 con 0x7f031809ec80 2024-04-28T13:31:19.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.052+0000 7f0317200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f03180add70 con 0x7f03180b5ee0 2024-04-28T13:31:19.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.052+0000 7f0316800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03180d1760 0x7f031815f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:19.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.052+0000 7f0316800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03180d1760 0x7f031815f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:48074/0 (socket says 172.21.15.196:48074) 2024-04-28T13:31:19.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.052+0000 7f0316800640 1 -- 172.21.15.196:0/495443581 learned_addr learned my addr 172.21.15.196:0/495443581 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:19.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.053+0000 7f0315e00640 1 --2- 172.21.15.196:0/495443581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f031809ec80 0x7f031814fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:19.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.053+0000 7f0315400640 1 --2- 172.21.15.196:0/495443581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03180b5ee0 0x7f03180cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:19.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.053+0000 7f0316800640 1 -- 172.21.15.196:0/495443581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03180b5ee0 msgr2=0x7f03180cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:19.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.053+0000 7f0316800640 1 --2- 172.21.15.196:0/495443581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03180b5ee0 0x7f03180cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:19.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.053+0000 7f0316800640 1 -- 172.21.15.196:0/495443581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f031809ec80 msgr2=0x7f031814fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:19.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.053+0000 7f0316800640 1 --2- 172.21.15.196:0/495443581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f031809ec80 0x7f031814fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:19.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.053+0000 7f0316800640 1 -- 172.21.15.196:0/495443581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0318150300 con 0x7f03180d1760 2024-04-28T13:31:19.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.053+0000 7f0315e00640 1 --2- 172.21.15.196:0/495443581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f031809ec80 0x7f031814fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:19.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.053+0000 7f0316800640 1 --2- 172.21.15.196:0/495443581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03180d1760 0x7f031815f310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f031002ac70 tx=0x7f0310004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:19.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.053+0000 7f030e000640 1 -- 172.21.15.196:0/495443581 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310040070 con 0x7f03180d1760 2024-04-28T13:31:19.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.053+0000 7f030e000640 1 -- 172.21.15.196:0/495443581 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0310031bc0 con 0x7f03180d1760 2024-04-28T13:31:19.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.053+0000 7f030e000640 1 -- 172.21.15.196:0/495443581 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310009450 con 0x7f03180d1760 2024-04-28T13:31:19.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.053+0000 7f0317200640 1 -- 172.21.15.196:0/495443581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03181505e0 con 0x7f03180d1760 2024-04-28T13:31:19.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.053+0000 7f0317200640 1 -- 172.21.15.196:0/495443581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0318150ba0 con 0x7f03180d1760 2024-04-28T13:31:19.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.055+0000 7f0317200640 1 -- 172.21.15.196:0/495443581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f031815ce00 con 0x7f03180d1760 2024-04-28T13:31:19.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.056+0000 7f030e000640 1 -- 172.21.15.196:0/495443581 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0310039040 con 0x7f03180d1760 2024-04-28T13:31:19.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.057+0000 7f030e000640 1 --2- 172.21.15.196:0/495443581 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f02f806c8f0 0x7f02f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:19.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.057+0000 7f030e000640 1 -- 172.21.15.196:0/495443581 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f03100ad0e0 con 0x7f03180d1760 2024-04-28T13:31:19.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.058+0000 7f0315e00640 1 --2- 172.21.15.196:0/495443581 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f02f806c8f0 0x7f02f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:19.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.059+0000 7f0315e00640 1 --2- 172.21.15.196:0/495443581 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f02f806c8f0 0x7f02f806edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0308005880 tx=0x7f0308009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:19.061 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.062+0000 7f030e000640 1 -- 172.21.15.196:0/495443581 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f031815ce00 con 0x7f03180d1760 2024-04-28T13:31:19.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.265+0000 7f0317200640 1 -- 172.21.15.196:0/495443581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f031806e8d0 con 0x7f03180d1760 2024-04-28T13:31:19.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.578+0000 7f030e000640 1 -- 172.21.15.196:0/495443581 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f031806e8d0 con 0x7f03180d1760 2024-04-28T13:31:19.579 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'cephfs_metadata' created 2024-04-28T13:31:19.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.582+0000 7f0317200640 1 -- 172.21.15.196:0/495443581 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f02f806c8f0 msgr2=0x7f02f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:19.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.582+0000 7f0317200640 1 --2- 172.21.15.196:0/495443581 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f02f806c8f0 0x7f02f806edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0308005880 tx=0x7f0308009040 comp rx=0 tx=0).stop 2024-04-28T13:31:19.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.582+0000 7f0317200640 1 -- 172.21.15.196:0/495443581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03180d1760 msgr2=0x7f031815f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:19.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.582+0000 7f0317200640 1 --2- 172.21.15.196:0/495443581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03180d1760 0x7f031815f310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f031002ac70 tx=0x7f0310004a40 comp rx=0 tx=0).stop 2024-04-28T13:31:19.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.582+0000 7f0317200640 1 -- 172.21.15.196:0/495443581 shutdown_connections 2024-04-28T13:31:19.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.582+0000 7f0317200640 1 --2- 172.21.15.196:0/495443581 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f02f806c8f0 0x7f02f806edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:19.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.582+0000 7f0317200640 1 --2- 172.21.15.196:0/495443581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03180d1760 0x7f031815f310 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:19.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.582+0000 7f0317200640 1 --2- 172.21.15.196:0/495443581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03180b5ee0 0x7f03180cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:19.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.582+0000 7f0317200640 1 --2- 172.21.15.196:0/495443581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f031809ec80 0x7f031814fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:19.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.582+0000 7f0317200640 1 -- 172.21.15.196:0/495443581 >> 172.21.15.196:0/495443581 conn(0x7f031806c6c0 msgr2=0x7f03180d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:19.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.583+0000 7f0317200640 1 -- 172.21.15.196:0/495443581 shutdown_connections 2024-04-28T13:31:19.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.583+0000 7f0317200640 1 -- 172.21.15.196:0/495443581 wait complete. 2024-04-28T13:31:19.610 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-28T13:31:19.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.800+0000 7fe770e00640 1 Processor -- start 2024-04-28T13:31:19.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.800+0000 7fe770e00640 1 -- start start 2024-04-28T13:31:19.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.801+0000 7fe770e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe76c073220 0x7fe76c073640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:19.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.801+0000 7fe770e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe76c071f40 0x7fe76c072390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:19.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.802+0000 7fe770e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe76c073b80 con 0x7fe76c0afe20 2024-04-28T13:31:19.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.802+0000 7fe770e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fe76c0b0200 con 0x7fe76c071f40 2024-04-28T13:31:19.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.802+0000 7fe770e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe76c072f50 con 0x7fe76c073220 2024-04-28T13:31:19.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.802+0000 7fe76a800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe76c071f40 0x7fe76c072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:19.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.802+0000 7fe769e00640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fe76c0afe20 0x7fe76c06fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.196:51718/0 (socket says 172.21.15.196:51718) 2024-04-28T13:31:19.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.802+0000 7fe769e00640 1 -- 172.21.15.196:0/2574578355 learned_addr learned my addr 172.21.15.196:0/2574578355 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:19.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.802+0000 7fe763e00640 1 --2- 172.21.15.196:0/2574578355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe76c073220 0x7fe76c073640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:19.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.803+0000 7fe769400640 1 -- 172.21.15.196:0/2574578355 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1568938660 0 0) 0x7fe76c073b80 con 0x7fe76c0afe20 2024-04-28T13:31:19.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.803+0000 7fe769400640 1 -- 172.21.15.196:0/2574578355 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe750003520 con 0x7fe76c0afe20 2024-04-28T13:31:19.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.803+0000 7fe763e00640 1 -- 172.21.15.196:0/2574578355 >> v2:172.21.15.203:3300/0 conn(0x7fe76c071f40 msgr2=0x7fe76c072390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:19.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.803+0000 7fe763e00640 1 --2- 172.21.15.196:0/2574578355 >> v2:172.21.15.203:3300/0 conn(0x7fe76c071f40 0x7fe76c072390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:19.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.803+0000 7fe763e00640 1 -- 172.21.15.196:0/2574578355 >> v1:172.21.15.203:6789/0 conn(0x7fe76c0afe20 legacy=0x7fe76c06fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:19.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.803+0000 7fe763e00640 1 -- 172.21.15.196:0/2574578355 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76c072c70 con 0x7fe76c073220 2024-04-28T13:31:19.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.803+0000 7fe76a800640 1 --2- 172.21.15.196:0/2574578355 >> v2:172.21.15.203:3300/0 conn(0x7fe76c071f40 0x7fe76c072390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:31:19.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.805+0000 7fe763e00640 1 --2- 172.21.15.196:0/2574578355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe76c073220 0x7fe76c073640 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe7540097a0 tx=0x7fe75402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2148897bbd8abeb server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:19.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.805+0000 7fe769400640 1 -- 172.21.15.196:0/2574578355 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754037070 con 0x7fe76c073220 2024-04-28T13:31:19.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.805+0000 7fe769400640 1 -- 172.21.15.196:0/2574578355 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe754032440 con 0x7fe76c073220 2024-04-28T13:31:19.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.805+0000 7fe769400640 1 -- 172.21.15.196:0/2574578355 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754032810 con 0x7fe76c073220 2024-04-28T13:31:19.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.806+0000 7fe770e00640 1 -- 172.21.15.196:0/2574578355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe76c073220 msgr2=0x7fe76c073640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:19.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.806+0000 7fe770e00640 1 --2- 172.21.15.196:0/2574578355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe76c073220 0x7fe76c073640 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe7540097a0 tx=0x7fe75402a180 comp rx=0 tx=0).stop 2024-04-28T13:31:19.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.806+0000 7fe770e00640 1 -- 172.21.15.196:0/2574578355 shutdown_connections 2024-04-28T13:31:19.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.806+0000 7fe770e00640 1 --2- 172.21.15.196:0/2574578355 >> v2:172.21.15.203:3300/0 conn(0x7fe76c071f40 0x7fe76c072390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:19.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.806+0000 7fe770e00640 1 --2- 172.21.15.196:0/2574578355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe76c073220 0x7fe76c073640 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:19.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.806+0000 7fe770e00640 1 -- 172.21.15.196:0/2574578355 >> 172.21.15.196:0/2574578355 conn(0x7fe76c0d0d30 msgr2=0x7fe76c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:19.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.806+0000 7fe770e00640 1 -- 172.21.15.196:0/2574578355 shutdown_connections 2024-04-28T13:31:19.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.806+0000 7fe770e00640 1 -- 172.21.15.196:0/2574578355 wait complete. 2024-04-28T13:31:19.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.808+0000 7fe770e00640 1 Processor -- start 2024-04-28T13:31:19.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.808+0000 7fe770e00640 1 -- start start 2024-04-28T13:31:19.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.809+0000 7fe770e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76c071f40 0x7fe76c0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:19.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.809+0000 7fe770e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe76c073220 0x7fe76c1f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:19.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.809+0000 7fe770e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe76c0afe20 0x7fe76c1f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:19.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.809+0000 7fe770e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c06f900 con 0x7fe76c071f40 2024-04-28T13:31:19.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.809+0000 7fe770e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c06f620 con 0x7fe76c073220 2024-04-28T13:31:19.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.809+0000 7fe770e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe76c0728d0 con 0x7fe76c0afe20 2024-04-28T13:31:19.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.809+0000 7fe769e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76c071f40 0x7fe76c0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:19.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.809+0000 7fe769e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76c071f40 0x7fe76c0c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:48082/0 (socket says 172.21.15.196:48082) 2024-04-28T13:31:19.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.809+0000 7fe769e00640 1 -- 172.21.15.196:0/385081767 learned_addr learned my addr 172.21.15.196:0/385081767 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:19.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.810+0000 7fe76a800640 1 --2- 172.21.15.196:0/385081767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe76c0afe20 0x7fe76c1f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:19.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.810+0000 7fe763e00640 1 --2- 172.21.15.196:0/385081767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe76c073220 0x7fe76c1f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:19.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.810+0000 7fe769e00640 1 -- 172.21.15.196:0/385081767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe76c0afe20 msgr2=0x7fe76c1f8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:19.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.810+0000 7fe769e00640 1 --2- 172.21.15.196:0/385081767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe76c0afe20 0x7fe76c1f8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:19.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.810+0000 7fe769e00640 1 -- 172.21.15.196:0/385081767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe76c073220 msgr2=0x7fe76c1f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:19.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.810+0000 7fe769e00640 1 --2- 172.21.15.196:0/385081767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe76c073220 0x7fe76c1f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:19.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.810+0000 7fe769e00640 1 -- 172.21.15.196:0/385081767 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76c0ce660 con 0x7fe76c071f40 2024-04-28T13:31:19.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.810+0000 7fe769e00640 1 --2- 172.21.15.196:0/385081767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76c071f40 0x7fe76c0c5d50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c430 tx=0x7fe75c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:19.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.810+0000 7fe762000640 1 -- 172.21.15.196:0/385081767 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c017070 con 0x7fe76c071f40 2024-04-28T13:31:19.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.810+0000 7fe762000640 1 -- 172.21.15.196:0/385081767 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75c005ce0 con 0x7fe76c071f40 2024-04-28T13:31:19.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.810+0000 7fe762000640 1 -- 172.21.15.196:0/385081767 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0072f0 con 0x7fe76c071f40 2024-04-28T13:31:19.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.810+0000 7fe770e00640 1 -- 172.21.15.196:0/385081767 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76c0ce940 con 0x7fe76c071f40 2024-04-28T13:31:19.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.811+0000 7fe770e00640 1 -- 172.21.15.196:0/385081767 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe76c0cf1e0 con 0x7fe76c071f40 2024-04-28T13:31:19.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.812+0000 7fe770e00640 1 -- 172.21.15.196:0/385081767 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe76c0ceb10 con 0x7fe76c071f40 2024-04-28T13:31:19.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.813+0000 7fe762000640 1 -- 172.21.15.196:0/385081767 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe75c01a070 con 0x7fe76c071f40 2024-04-28T13:31:19.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.814+0000 7fe762000640 1 --2- 172.21.15.196:0/385081767 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe74806c8f0 0x7fe74806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:19.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.814+0000 7fe762000640 1 -- 172.21.15.196:0/385081767 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fe75c08ae40 con 0x7fe76c071f40 2024-04-28T13:31:19.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.815+0000 7fe763e00640 1 --2- 172.21.15.196:0/385081767 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe74806c8f0 0x7fe74806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:19.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.816+0000 7fe763e00640 1 --2- 172.21.15.196:0/385081767 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe74806c8f0 0x7fe74806edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe754034000 tx=0x7fe75402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:19.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:19.819+0000 7fe762000640 1 -- 172.21.15.196:0/385081767 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe75c059440 con 0x7fe76c071f40 2024-04-28T13:31:20.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.021+0000 7fe770e00640 1 -- 172.21.15.196:0/385081767 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe76c06a290 con 0x7fe76c071f40 2024-04-28T13:31:20.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.586+0000 7fe762000640 1 -- 172.21.15.196:0/385081767 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7fe75c058d40 con 0x7fe76c071f40 2024-04-28T13:31:20.586 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'cephfs_data' created 2024-04-28T13:31:20.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.590+0000 7fe770e00640 1 -- 172.21.15.196:0/385081767 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe74806c8f0 msgr2=0x7fe74806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:20.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.590+0000 7fe770e00640 1 --2- 172.21.15.196:0/385081767 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe74806c8f0 0x7fe74806edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe754034000 tx=0x7fe75402e000 comp rx=0 tx=0).stop 2024-04-28T13:31:20.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.590+0000 7fe770e00640 1 -- 172.21.15.196:0/385081767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76c071f40 msgr2=0x7fe76c0c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:20.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.590+0000 7fe770e00640 1 --2- 172.21.15.196:0/385081767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76c071f40 0x7fe76c0c5d50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c430 tx=0x7fe75c00c8f0 comp rx=0 tx=0).stop 2024-04-28T13:31:20.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.591+0000 7fe770e00640 1 -- 172.21.15.196:0/385081767 shutdown_connections 2024-04-28T13:31:20.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.591+0000 7fe770e00640 1 --2- 172.21.15.196:0/385081767 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe74806c8f0 0x7fe74806edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:20.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.591+0000 7fe770e00640 1 --2- 172.21.15.196:0/385081767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe76c0afe20 0x7fe76c1f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:20.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.591+0000 7fe770e00640 1 --2- 172.21.15.196:0/385081767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe76c073220 0x7fe76c1f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:20.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.591+0000 7fe770e00640 1 --2- 172.21.15.196:0/385081767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe76c071f40 0x7fe76c0c5d50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:20.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.591+0000 7fe770e00640 1 -- 172.21.15.196:0/385081767 >> 172.21.15.196:0/385081767 conn(0x7fe76c0d0d30 msgr2=0x7fe76c15f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:20.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.591+0000 7fe770e00640 1 -- 172.21.15.196:0/385081767 shutdown_connections 2024-04-28T13:31:20.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.592+0000 7fe770e00640 1 -- 172.21.15.196:0/385081767 wait complete. 2024-04-28T13:31:20.628 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-28T13:31:20.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.804+0000 7f01abe00640 1 Processor -- start 2024-04-28T13:31:20.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.804+0000 7f01abe00640 1 -- start start 2024-04-28T13:31:20.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.805+0000 7f01abe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f01ac06e8f0 0x7f01ac0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:20.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.805+0000 7f01abe00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f01ac0d1480 0x7f01ac160c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:20.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.805+0000 7f01abe00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ac15d700 con 0x7f01ac0d10a0 2024-04-28T13:31:20.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.805+0000 7f01abe00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f01ac15e310 con 0x7f01ac06e8f0 2024-04-28T13:31:20.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.805+0000 7f01abe00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f01ac15d9e0 con 0x7f01ac0d1480 2024-04-28T13:31:20.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.806+0000 7f01aa000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f01ac06e8f0 0x7f01ac0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:20.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.806+0000 7f01aa000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f01ac06e8f0 0x7f01ac0b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:48090/0 (socket says 172.21.15.196:48090) 2024-04-28T13:31:20.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.806+0000 7f01aa000640 1 -- 172.21.15.196:0/2246980028 learned_addr learned my addr 172.21.15.196:0/2246980028 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:20.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.806+0000 7f01aaa00640 1 --1- 172.21.15.196:0/2246980028 >> v1:172.21.15.196:6789/0 conn(0x7f01ac0d10a0 0x7f01ac0b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:57930/0 (socket says 172.21.15.196:57930) 2024-04-28T13:31:20.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.806+0000 7f01ab400640 1 --2- 172.21.15.196:0/2246980028 >> v2:172.21.15.203:3300/0 conn(0x7f01ac0d1480 0x7f01ac160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:20.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.806+0000 7f01aa000640 1 -- 172.21.15.196:0/2246980028 >> v2:172.21.15.203:3300/0 conn(0x7f01ac0d1480 msgr2=0x7f01ac160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:20.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.806+0000 7f01aa000640 1 --2- 172.21.15.196:0/2246980028 >> v2:172.21.15.203:3300/0 conn(0x7f01ac0d1480 0x7f01ac160c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:20.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.806+0000 7f01aa000640 1 -- 172.21.15.196:0/2246980028 >> v1:172.21.15.196:6789/0 conn(0x7f01ac0d10a0 legacy=0x7f01ac0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:20.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.806+0000 7f01a9600640 1 -- 172.21.15.196:0/2246980028 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3117327284 0 0) 0x7f01ac15d700 con 0x7f01ac0d10a0 2024-04-28T13:31:20.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.806+0000 7f01aa000640 1 -- 172.21.15.196:0/2246980028 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac15dcc0 con 0x7f01ac06e8f0 2024-04-28T13:31:20.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.807+0000 7f01aa000640 1 --2- 172.21.15.196:0/2246980028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01ac06e8f0 0x7f01ac0b3c90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f01ac15b030 tx=0x7f019802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d090d77a682bc89 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:20.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.808+0000 7f01a9600640 1 -- 172.21.15.196:0/2246980028 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198037070 con 0x7f01ac06e8f0 2024-04-28T13:31:20.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.808+0000 7f01a9600640 1 -- 172.21.15.196:0/2246980028 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0198032440 con 0x7f01ac06e8f0 2024-04-28T13:31:20.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.808+0000 7f01a9600640 1 -- 172.21.15.196:0/2246980028 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198032810 con 0x7f01ac06e8f0 2024-04-28T13:31:20.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.808+0000 7f01abe00640 1 -- 172.21.15.196:0/2246980028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01ac06e8f0 msgr2=0x7f01ac0b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:20.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.808+0000 7f01abe00640 1 --2- 172.21.15.196:0/2246980028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01ac06e8f0 0x7f01ac0b3c90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f01ac15b030 tx=0x7f019802a1e0 comp rx=0 tx=0).stop 2024-04-28T13:31:20.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.808+0000 7f01abe00640 1 -- 172.21.15.196:0/2246980028 shutdown_connections 2024-04-28T13:31:20.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.808+0000 7f01abe00640 1 --2- 172.21.15.196:0/2246980028 >> v2:172.21.15.203:3300/0 conn(0x7f01ac0d1480 0x7f01ac160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:20.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.808+0000 7f01abe00640 1 --2- 172.21.15.196:0/2246980028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01ac06e8f0 0x7f01ac0b3c90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:20.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.808+0000 7f01abe00640 1 -- 172.21.15.196:0/2246980028 >> 172.21.15.196:0/2246980028 conn(0x7f01ac06c6c0 msgr2=0x7f01ac06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:20.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.808+0000 7f01abe00640 1 -- 172.21.15.196:0/2246980028 shutdown_connections 2024-04-28T13:31:20.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.808+0000 7f01abe00640 1 -- 172.21.15.196:0/2246980028 wait complete. 2024-04-28T13:31:20.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.810+0000 7f01abe00640 1 Processor -- start 2024-04-28T13:31:20.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.810+0000 7f01abe00640 1 -- start start 2024-04-28T13:31:20.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.811+0000 7f01abe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac06e8f0 0x7f01ac156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:20.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.811+0000 7f01abe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01ac0d10a0 0x7f01ac14d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:20.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.811+0000 7f01abe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01ac0d1480 0x7f01ac159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:20.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.811+0000 7f01abe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac0d1a30 con 0x7f01ac0d10a0 2024-04-28T13:31:20.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.811+0000 7f01abe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac0d18b0 con 0x7f01ac06e8f0 2024-04-28T13:31:20.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.811+0000 7f01abe00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f01ac1611d0 con 0x7f01ac0d1480 2024-04-28T13:31:20.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.811+0000 7f01aa000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01ac0d10a0 0x7f01ac14d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:20.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.811+0000 7f01aa000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01ac0d10a0 0x7f01ac14d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:48096/0 (socket says 172.21.15.196:48096) 2024-04-28T13:31:20.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.811+0000 7f01aa000640 1 -- 172.21.15.196:0/3674529719 learned_addr learned my addr 172.21.15.196:0/3674529719 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:20.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.812+0000 7f01aaa00640 1 --2- 172.21.15.196:0/3674529719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac06e8f0 0x7f01ac156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:20.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.812+0000 7f01aa000640 1 -- 172.21.15.196:0/3674529719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01ac0d1480 msgr2=0x7f01ac159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:20.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.812+0000 7f01aa000640 1 --2- 172.21.15.196:0/3674529719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01ac0d1480 0x7f01ac159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:20.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.812+0000 7f01aa000640 1 -- 172.21.15.196:0/3674529719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac06e8f0 msgr2=0x7f01ac156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:20.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.812+0000 7f01aa000640 1 --2- 172.21.15.196:0/3674529719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac06e8f0 0x7f01ac156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:20.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.812+0000 7f01aa000640 1 -- 172.21.15.196:0/3674529719 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac155400 con 0x7f01ac0d10a0 2024-04-28T13:31:20.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.812+0000 7f01aa000640 1 --2- 172.21.15.196:0/3674529719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01ac0d10a0 0x7f01ac14d7f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0198000c00 tx=0x7f01980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:20.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.813+0000 7f01a2a00640 1 -- 172.21.15.196:0/3674529719 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198037040 con 0x7f01ac0d10a0 2024-04-28T13:31:20.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.813+0000 7f01a2a00640 1 -- 172.21.15.196:0/3674529719 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0198004230 con 0x7f01ac0d10a0 2024-04-28T13:31:20.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.813+0000 7f01a2a00640 1 -- 172.21.15.196:0/3674529719 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198003610 con 0x7f01ac0d10a0 2024-04-28T13:31:20.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.813+0000 7f01abe00640 1 -- 172.21.15.196:0/3674529719 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac1556e0 con 0x7f01ac0d10a0 2024-04-28T13:31:20.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.813+0000 7f01abe00640 1 -- 172.21.15.196:0/3674529719 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac155ca0 con 0x7f01ac0d10a0 2024-04-28T13:31:20.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.814+0000 7f01abe00640 1 -- 172.21.15.196:0/3674529719 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ac15ae80 con 0x7f01ac0d10a0 2024-04-28T13:31:20.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.815+0000 7f01a2a00640 1 -- 172.21.15.196:0/3674529719 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01980037b0 con 0x7f01ac0d10a0 2024-04-28T13:31:20.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.816+0000 7f01a2a00640 1 --2- 172.21.15.196:0/3674529719 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f018c06c8f0 0x7f018c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:20.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.816+0000 7f01a2a00640 1 -- 172.21.15.196:0/3674529719 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f01980ac9e0 con 0x7f01ac0d10a0 2024-04-28T13:31:20.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.817+0000 7f01aaa00640 1 --2- 172.21.15.196:0/3674529719 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f018c06c8f0 0x7f018c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:20.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.818+0000 7f01aaa00640 1 --2- 172.21.15.196:0/3674529719 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f018c06c8f0 0x7f018c06edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f019c000f60 tx=0x7f019c007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:20.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:20.821+0000 7f01a2a00640 1 -- 172.21.15.196:0/3674529719 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f01ac15ae80 con 0x7f01ac0d10a0 2024-04-28T13:31:21.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.058+0000 7f01abe00640 1 -- 172.21.15.196:0/3674529719 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f01ac09eea0 con 0x7f01ac0d10a0 2024-04-28T13:31:21.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.608+0000 7f01a2a00640 1 -- 172.21.15.196:0/3674529719 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f019807ade0 con 0x7f01ac0d10a0 2024-04-28T13:31:21.609 INFO:teuthology.orchestra.run.smithi196.stderr:new fs with metadata pool 2 and data pool 3 2024-04-28T13:31:21.612 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.612+0000 7f01abe00640 1 -- 172.21.15.196:0/3674529719 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f018c06c8f0 msgr2=0x7f018c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:21.612 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.612+0000 7f01abe00640 1 --2- 172.21.15.196:0/3674529719 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f018c06c8f0 0x7f018c06edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f019c000f60 tx=0x7f019c007b50 comp rx=0 tx=0).stop 2024-04-28T13:31:21.612 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.612+0000 7f01abe00640 1 -- 172.21.15.196:0/3674529719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01ac0d10a0 msgr2=0x7f01ac14d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:21.612 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.612+0000 7f01abe00640 1 --2- 172.21.15.196:0/3674529719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01ac0d10a0 0x7f01ac14d7f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0198000c00 tx=0x7f01980046c0 comp rx=0 tx=0).stop 2024-04-28T13:31:21.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.612+0000 7f01abe00640 1 -- 172.21.15.196:0/3674529719 shutdown_connections 2024-04-28T13:31:21.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.612+0000 7f01abe00640 1 --2- 172.21.15.196:0/3674529719 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f018c06c8f0 0x7f018c06edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:21.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.612+0000 7f01abe00640 1 --2- 172.21.15.196:0/3674529719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01ac0d1480 0x7f01ac159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:21.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.612+0000 7f01abe00640 1 --2- 172.21.15.196:0/3674529719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01ac0d10a0 0x7f01ac14d7f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:21.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.612+0000 7f01abe00640 1 --2- 172.21.15.196:0/3674529719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac06e8f0 0x7f01ac156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:21.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.612+0000 7f01abe00640 1 -- 172.21.15.196:0/3674529719 >> 172.21.15.196:0/3674529719 conn(0x7f01ac06c6c0 msgr2=0x7f01ac15b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:21.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.612+0000 7f01abe00640 1 -- 172.21.15.196:0/3674529719 shutdown_connections 2024-04-28T13:31:21.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.612+0000 7f01abe00640 1 -- 172.21.15.196:0/3674529719 wait complete. 2024-04-28T13:31:21.657 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:21.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.826+0000 7f0dee000640 1 Processor -- start 2024-04-28T13:31:21.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.826+0000 7f0dee000640 1 -- start start 2024-04-28T13:31:21.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.828+0000 7f0dee000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0de815ef00 0x7f0de80b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:21.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.828+0000 7f0dee000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0de80d1050 0x7f0de80d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:21.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.828+0000 7f0dee000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0de806dc70 con 0x7f0de815f2e0 2024-04-28T13:31:21.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.828+0000 7f0dee000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0de80d1a00 con 0x7f0de80d1050 2024-04-28T13:31:21.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.828+0000 7f0dee000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f0de806df50 con 0x7f0de815ef00 2024-04-28T13:31:21.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.828+0000 7f0ded600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0de80d1050 0x7f0de80d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:21.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.828+0000 7f0de7e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f0de815f2e0 0x7f0de806f7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:57934/0 (socket says 172.21.15.196:57934) 2024-04-28T13:31:21.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.828+0000 7f0de7e00640 1 -- 172.21.15.196:0/303530801 learned_addr learned my addr 172.21.15.196:0/303530801 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:21.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.829+0000 7f0decc00640 1 --2- 172.21.15.196:0/303530801 >> v2:172.21.15.203:3300/0 conn(0x7f0de815ef00 0x7f0de80b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:21.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.829+0000 7f0de7400640 1 -- 172.21.15.196:0/303530801 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 856481343 0 0) 0x7f0de806dc70 con 0x7f0de815f2e0 2024-04-28T13:31:21.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.829+0000 7f0de7400640 1 -- 172.21.15.196:0/303530801 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0dcc003520 con 0x7f0de815f2e0 2024-04-28T13:31:21.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.829+0000 7f0ded600640 1 -- 172.21.15.196:0/303530801 >> v2:172.21.15.203:3300/0 conn(0x7f0de815ef00 msgr2=0x7f0de80b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:21.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.829+0000 7f0ded600640 1 --2- 172.21.15.196:0/303530801 >> v2:172.21.15.203:3300/0 conn(0x7f0de815ef00 0x7f0de80b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:21.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.829+0000 7f0ded600640 1 -- 172.21.15.196:0/303530801 >> v1:172.21.15.196:6789/0 conn(0x7f0de815f2e0 legacy=0x7f0de806f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:21.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.829+0000 7f0ded600640 1 -- 172.21.15.196:0/303530801 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de806e230 con 0x7f0de80d1050 2024-04-28T13:31:21.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.830+0000 7f0ded600640 1 --2- 172.21.15.196:0/303530801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de80d1050 0x7f0de80d14c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0de8068fc0 tx=0x7f0de002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e14fad80e78824 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:21.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.830+0000 7f0de7400640 1 -- 172.21.15.196:0/303530801 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0007740 con 0x7f0de80d1050 2024-04-28T13:31:21.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.830+0000 7f0de7400640 1 -- 172.21.15.196:0/303530801 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de00078e0 con 0x7f0de80d1050 2024-04-28T13:31:21.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.831+0000 7f0de7400640 1 -- 172.21.15.196:0/303530801 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0007cb0 con 0x7f0de80d1050 2024-04-28T13:31:21.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.831+0000 7f0dee000640 1 -- 172.21.15.196:0/303530801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de80d1050 msgr2=0x7f0de80d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:21.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.831+0000 7f0dee000640 1 --2- 172.21.15.196:0/303530801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de80d1050 0x7f0de80d14c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0de8068fc0 tx=0x7f0de002a5f0 comp rx=0 tx=0).stop 2024-04-28T13:31:21.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.831+0000 7f0dee000640 1 -- 172.21.15.196:0/303530801 shutdown_connections 2024-04-28T13:31:21.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.831+0000 7f0dee000640 1 --2- 172.21.15.196:0/303530801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de80d1050 0x7f0de80d14c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:21.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.831+0000 7f0dee000640 1 --2- 172.21.15.196:0/303530801 >> v2:172.21.15.203:3300/0 conn(0x7f0de815ef00 0x7f0de80b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:21.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.831+0000 7f0dee000640 1 -- 172.21.15.196:0/303530801 >> 172.21.15.196:0/303530801 conn(0x7f0de806c6c0 msgr2=0x7f0de806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:21.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.831+0000 7f0dee000640 1 -- 172.21.15.196:0/303530801 shutdown_connections 2024-04-28T13:31:21.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.831+0000 7f0dee000640 1 -- 172.21.15.196:0/303530801 wait complete. 2024-04-28T13:31:21.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.833+0000 7f0dee000640 1 Processor -- start 2024-04-28T13:31:21.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.833+0000 7f0dee000640 1 -- start start 2024-04-28T13:31:21.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.834+0000 7f0dee000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0de80d1050 0x7f0de8158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:21.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.834+0000 7f0dee000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0de815ef00 0x7f0de81f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:21.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.834+0000 7f0dee000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de815f2e0 0x7f0de814f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:21.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.834+0000 7f0dee000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8072280 con 0x7f0de815f2e0 2024-04-28T13:31:21.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.834+0000 7f0dee000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8072100 con 0x7f0de815ef00 2024-04-28T13:31:21.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.834+0000 7f0dee000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0de8072400 con 0x7f0de80d1050 2024-04-28T13:31:21.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.834+0000 7f0ded600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de815f2e0 0x7f0de814f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:21.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.834+0000 7f0ded600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de815f2e0 0x7f0de814f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:48120/0 (socket says 172.21.15.196:48120) 2024-04-28T13:31:21.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.834+0000 7f0ded600640 1 -- 172.21.15.196:0/2690235276 learned_addr learned my addr 172.21.15.196:0/2690235276 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:21.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.835+0000 7f0decc00640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0de80d1050 0x7f0de8158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:21.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.835+0000 7f0de7e00640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0de815ef00 0x7f0de81f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:21.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.835+0000 7f0ded600640 1 -- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0de80d1050 msgr2=0x7f0de8158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:21.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.835+0000 7f0ded600640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0de80d1050 0x7f0de8158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:21.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.835+0000 7f0ded600640 1 -- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0de815ef00 msgr2=0x7f0de81f8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:21.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.835+0000 7f0ded600640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0de815ef00 0x7f0de81f8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:21.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.835+0000 7f0ded600640 1 -- 172.21.15.196:0/2690235276 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de815a190 con 0x7f0de815f2e0 2024-04-28T13:31:21.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.835+0000 7f0de7e00640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0de815ef00 0x7f0de81f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:21.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.835+0000 7f0decc00640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0de80d1050 0x7f0de8158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:21.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.835+0000 7f0ded600640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de815f2e0 0x7f0de814f450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009e50 tx=0x7f0de0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:21.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.835+0000 7f0de4c00640 1 -- 172.21.15.196:0/2690235276 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0040070 con 0x7f0de815f2e0 2024-04-28T13:31:21.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.835+0000 7f0de4c00640 1 -- 172.21.15.196:0/2690235276 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de0031bf0 con 0x7f0de815f2e0 2024-04-28T13:31:21.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.835+0000 7f0de4c00640 1 -- 172.21.15.196:0/2690235276 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0009450 con 0x7f0de815f2e0 2024-04-28T13:31:21.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.835+0000 7f0dee000640 1 -- 172.21.15.196:0/2690235276 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de81598f0 con 0x7f0de815f2e0 2024-04-28T13:31:21.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.835+0000 7f0dee000640 1 -- 172.21.15.196:0/2690235276 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de8159eb0 con 0x7f0de815f2e0 2024-04-28T13:31:21.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.836+0000 7f0dee000640 1 -- 172.21.15.196:0/2690235276 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de806a290 con 0x7f0de815f2e0 2024-04-28T13:31:21.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.837+0000 7f0de4c00640 1 -- 172.21.15.196:0/2690235276 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0de0039040 con 0x7f0de815f2e0 2024-04-28T13:31:21.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.839+0000 7f0de4c00640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0dc806c8f0 0x7f0dc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:21.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.839+0000 7f0de4c00640 1 -- 172.21.15.196:0/2690235276 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f0de00ad570 con 0x7f0de815f2e0 2024-04-28T13:31:21.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.839+0000 7f0decc00640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0dc806c8f0 0x7f0dc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:21.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.840+0000 7f0decc00640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0dc806c8f0 0x7f0dc806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8006fd0 tx=0x7f0dd8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:21.843+0000 7f0de4c00640 1 -- 172.21.15.196:0/2690235276 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0de815a190 con 0x7f0de815f2e0 2024-04-28T13:31:22.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.043+0000 7f0dee000640 1 -- 172.21.15.196:0/2690235276 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0de81629e0 con 0x7f0de815f2e0 2024-04-28T13:31:22.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.047+0000 7f0de4c00640 1 -- 172.21.15.196:0/2690235276 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16654 (secure 0 0 0) 0x7f0de00abe20 con 0x7f0de815f2e0 2024-04-28T13:31:22.047 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:22.047 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":19,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:21.059651+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:22.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.050+0000 7f0dee000640 1 -- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0dc806c8f0 msgr2=0x7f0dc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:22.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.050+0000 7f0dee000640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0dc806c8f0 0x7f0dc806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8006fd0 tx=0x7f0dd8008040 comp rx=0 tx=0).stop 2024-04-28T13:31:22.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.050+0000 7f0dee000640 1 -- 172.21.15.196:0/2690235276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de815f2e0 msgr2=0x7f0de814f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:22.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.050+0000 7f0dee000640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de815f2e0 0x7f0de814f450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009e50 tx=0x7f0de0004a40 comp rx=0 tx=0).stop 2024-04-28T13:31:22.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.050+0000 7f0dee000640 1 -- 172.21.15.196:0/2690235276 shutdown_connections 2024-04-28T13:31:22.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.050+0000 7f0dee000640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0dc806c8f0 0x7f0dc806edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.050+0000 7f0dee000640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0de815f2e0 0x7f0de814f450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.050+0000 7f0dee000640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0de815ef00 0x7f0de81f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.050+0000 7f0dee000640 1 --2- 172.21.15.196:0/2690235276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0de80d1050 0x7f0de8158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.050+0000 7f0dee000640 1 -- 172.21.15.196:0/2690235276 >> 172.21.15.196:0/2690235276 conn(0x7f0de806c6c0 msgr2=0x7f0de80b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:22.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.050+0000 7f0dee000640 1 -- 172.21.15.196:0/2690235276 shutdown_connections 2024-04-28T13:31:22.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.050+0000 7f0dee000640 1 -- 172.21.15.196:0/2690235276 wait complete. 2024-04-28T13:31:22.071 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:22.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.260+0000 7fabb5800640 1 Processor -- start 2024-04-28T13:31:22.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.260+0000 7fabb5800640 1 -- start start 2024-04-28T13:31:22.262 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.262+0000 7fabb5800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fabb00d1190 0x7fabb00b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:22.262 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.262+0000 7fabb5800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabb0160f40 con 0x7fabb01469f0 2024-04-28T13:31:22.262 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.262+0000 7fabb5800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabb0161220 con 0x7fabb00d0d00 2024-04-28T13:31:22.262 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.262+0000 7fabb5800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fabb00b5960 con 0x7fabb00d1190 2024-04-28T13:31:22.262 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.262+0000 7fabaea00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fabb01469f0 0x7fabb00ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:55950/0 (socket says 172.21.15.196:55950) 2024-04-28T13:31:22.262 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.262+0000 7fabaea00640 1 -- 172.21.15.196:0/2362479824 learned_addr learned my addr 172.21.15.196:0/2362479824 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:22.262 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.262+0000 7fabaf400640 1 --2- 172.21.15.196:0/2362479824 >> v2:172.21.15.196:3300/0 conn(0x7fabb00d1190 0x7fabb00b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:22.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.262+0000 7fabad600640 1 -- 172.21.15.196:0/2362479824 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2006951892 0 0) 0x7fabb0160f40 con 0x7fabb01469f0 2024-04-28T13:31:22.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.262+0000 7fabad600640 1 -- 172.21.15.196:0/2362479824 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab94003520 con 0x7fabb01469f0 2024-04-28T13:31:22.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.263+0000 7fabaf400640 1 -- 172.21.15.196:0/2362479824 >> v1:172.21.15.203:6789/0 conn(0x7fabb00d0d00 legacy=0x7fabb00b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:22.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.263+0000 7fabaf400640 1 -- 172.21.15.196:0/2362479824 >> v1:172.21.15.196:6789/0 conn(0x7fabb01469f0 legacy=0x7fabb00ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:22.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.263+0000 7fabaf400640 1 -- 172.21.15.196:0/2362479824 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb0161500 con 0x7fabb00d1190 2024-04-28T13:31:22.264 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.264+0000 7fabaf400640 1 --2- 172.21.15.196:0/2362479824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabb00d1190 0x7fabb00b64e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faba8009d90 tx=0x7faba802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e2030784e5860b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:22.264 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.264+0000 7fabad600640 1 -- 172.21.15.196:0/2362479824 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8007740 con 0x7fabb00d1190 2024-04-28T13:31:22.264 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.264+0000 7fabad600640 1 -- 172.21.15.196:0/2362479824 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba80078e0 con 0x7fabb00d1190 2024-04-28T13:31:22.264 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.264+0000 7fabad600640 1 -- 172.21.15.196:0/2362479824 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8007cb0 con 0x7fabb00d1190 2024-04-28T13:31:22.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.265+0000 7fabb5800640 1 -- 172.21.15.196:0/2362479824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabb00d1190 msgr2=0x7fabb00b64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:22.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.265+0000 7fabb5800640 1 --2- 172.21.15.196:0/2362479824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabb00d1190 0x7fabb00b64e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faba8009d90 tx=0x7faba802a6f0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.265+0000 7fabb5800640 1 -- 172.21.15.196:0/2362479824 shutdown_connections 2024-04-28T13:31:22.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.265+0000 7fabb5800640 1 --2- 172.21.15.196:0/2362479824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabb00d1190 0x7fabb00b64e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.265+0000 7fabb5800640 1 -- 172.21.15.196:0/2362479824 >> 172.21.15.196:0/2362479824 conn(0x7fabb00cd830 msgr2=0x7fabb00cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:22.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.265+0000 7fabb5800640 1 -- 172.21.15.196:0/2362479824 shutdown_connections 2024-04-28T13:31:22.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.265+0000 7fabb5800640 1 -- 172.21.15.196:0/2362479824 wait complete. 2024-04-28T13:31:22.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.267+0000 7fabb5800640 1 Processor -- start 2024-04-28T13:31:22.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.267+0000 7fabb5800640 1 -- start start 2024-04-28T13:31:22.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.268+0000 7fabb5800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabb00d0d00 0x7fabb013f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:22.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.268+0000 7fabb5800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabb00d1190 0x7fabb0141e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:22.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.268+0000 7fabb5800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabb01469f0 0x7fabb01423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:22.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.268+0000 7fabb5800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fabb00cd5f0 con 0x7fabb01469f0 2024-04-28T13:31:22.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.268+0000 7fabb5800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0161ac0 con 0x7fabb00d1190 2024-04-28T13:31:22.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.268+0000 7fabb5800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fabb0072020 con 0x7fabb00d0d00 2024-04-28T13:31:22.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.269+0000 7fabaf400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabb01469f0 0x7fabb01423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:22.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.269+0000 7fabaf400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabb01469f0 0x7fabb01423d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35414/0 (socket says 172.21.15.196:35414) 2024-04-28T13:31:22.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.269+0000 7fabaf400640 1 -- 172.21.15.196:0/2322664086 learned_addr learned my addr 172.21.15.196:0/2322664086 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:22.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.269+0000 7fabae000640 1 --2- 172.21.15.196:0/2322664086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabb00d1190 0x7fabb0141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:22.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.269+0000 7fabaea00640 1 --2- 172.21.15.196:0/2322664086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabb00d0d00 0x7fabb013f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:22.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.269+0000 7fabaf400640 1 -- 172.21.15.196:0/2322664086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabb00d0d00 msgr2=0x7fabb013f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:22.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.269+0000 7fabaf400640 1 --2- 172.21.15.196:0/2322664086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabb00d0d00 0x7fabb013f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.269+0000 7fabaf400640 1 -- 172.21.15.196:0/2322664086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabb00d1190 msgr2=0x7fabb0141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:22.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.269+0000 7fabaf400640 1 --2- 172.21.15.196:0/2322664086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabb00d1190 0x7fabb0141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.269+0000 7fabaf400640 1 -- 172.21.15.196:0/2322664086 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb015fc90 con 0x7fabb01469f0 2024-04-28T13:31:22.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.269+0000 7fabae000640 1 --2- 172.21.15.196:0/2322664086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabb00d1190 0x7fabb0141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:22.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.269+0000 7fabaf400640 1 --2- 172.21.15.196:0/2322664086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabb01469f0 0x7fabb01423d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faba8009eb0 tx=0x7faba8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:22.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.270+0000 7faba6a00640 1 -- 172.21.15.196:0/2322664086 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8040070 con 0x7fabb01469f0 2024-04-28T13:31:22.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.270+0000 7faba6a00640 1 -- 172.21.15.196:0/2322664086 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba8031bc0 con 0x7fabb01469f0 2024-04-28T13:31:22.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.270+0000 7fabb5800640 1 -- 172.21.15.196:0/2322664086 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb015ff70 con 0x7fabb01469f0 2024-04-28T13:31:22.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.270+0000 7faba6a00640 1 -- 172.21.15.196:0/2322664086 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8009450 con 0x7fabb01469f0 2024-04-28T13:31:22.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.270+0000 7fabb5800640 1 -- 172.21.15.196:0/2322664086 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb0160530 con 0x7fabb01469f0 2024-04-28T13:31:22.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.271+0000 7fabb5800640 1 -- 172.21.15.196:0/2322664086 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb006a290 con 0x7fabb01469f0 2024-04-28T13:31:22.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.272+0000 7faba6a00640 1 -- 172.21.15.196:0/2322664086 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faba8039040 con 0x7fabb01469f0 2024-04-28T13:31:22.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.273+0000 7faba6a00640 1 --2- 172.21.15.196:0/2322664086 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fab9006c8f0 0x7fab9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:22.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.273+0000 7faba6a00640 1 -- 172.21.15.196:0/2322664086 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7faba80ad2f0 con 0x7fabb01469f0 2024-04-28T13:31:22.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.273+0000 7fabaea00640 1 --2- 172.21.15.196:0/2322664086 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fab9006c8f0 0x7fab9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:22.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.274+0000 7fabaea00640 1 --2- 172.21.15.196:0/2322664086 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fab9006c8f0 0x7fab9006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faba00011e0 tx=0x7faba0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:22.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.277+0000 7faba6a00640 1 -- 172.21.15.196:0/2322664086 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faba807b830 con 0x7fabb01469f0 2024-04-28T13:31:22.410 INFO:tasks.ceph.mds.b.smithi203.stderr:2024-04-28T13:31:22.409+0000 7f16b1200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-28T13:31:22.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.479+0000 7fabb5800640 1 -- 172.21.15.196:0/2322664086 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabb00b1a00 con 0x7fabb01469f0 2024-04-28T13:31:22.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.482+0000 7faba6a00640 1 -- 172.21.15.196:0/2322664086 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16654 (secure 0 0 0) 0x7faba807b130 con 0x7fabb01469f0 2024-04-28T13:31:22.483 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:22.483 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":19,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:21.059651+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:22.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.485+0000 7fabb5800640 1 -- 172.21.15.196:0/2322664086 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fab9006c8f0 msgr2=0x7fab9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:22.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.485+0000 7fabb5800640 1 --2- 172.21.15.196:0/2322664086 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fab9006c8f0 0x7fab9006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faba00011e0 tx=0x7faba0009040 comp rx=0 tx=0).stop 2024-04-28T13:31:22.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.485+0000 7fabb5800640 1 -- 172.21.15.196:0/2322664086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabb01469f0 msgr2=0x7fabb01423d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:22.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.485+0000 7fabb5800640 1 --2- 172.21.15.196:0/2322664086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabb01469f0 0x7fabb01423d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faba8009eb0 tx=0x7faba8004a40 comp rx=0 tx=0).stop 2024-04-28T13:31:22.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.485+0000 7fabb5800640 1 -- 172.21.15.196:0/2322664086 shutdown_connections 2024-04-28T13:31:22.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.485+0000 7fabb5800640 1 --2- 172.21.15.196:0/2322664086 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fab9006c8f0 0x7fab9006edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.485+0000 7fabb5800640 1 --2- 172.21.15.196:0/2322664086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabb01469f0 0x7fabb01423d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.485+0000 7fabb5800640 1 --2- 172.21.15.196:0/2322664086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabb00d1190 0x7fabb0141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.485+0000 7fabb5800640 1 --2- 172.21.15.196:0/2322664086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabb00d0d00 0x7fabb013f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.485+0000 7fabb5800640 1 -- 172.21.15.196:0/2322664086 >> 172.21.15.196:0/2322664086 conn(0x7fabb00cd830 msgr2=0x7fabb00b6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:22.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.486+0000 7fabb5800640 1 -- 172.21.15.196:0/2322664086 shutdown_connections 2024-04-28T13:31:22.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.486+0000 7fabb5800640 1 -- 172.21.15.196:0/2322664086 wait complete. 2024-04-28T13:31:22.504 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-28T13:31:22.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.713+0000 7f33d2400640 1 Processor -- start 2024-04-28T13:31:22.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.713+0000 7f33d2400640 1 -- start start 2024-04-28T13:31:22.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.715+0000 7f33d2400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f33cc0d10a0 0x7f33cc0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:22.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.715+0000 7f33d2400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33cc06e8f0 0x7f33cc0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:22.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.715+0000 7f33d2400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f33cc0d1480 0x7f33cc160c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:22.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.715+0000 7f33d2400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f33cc15e310 con 0x7f33cc0d10a0 2024-04-28T13:31:22.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.715+0000 7f33d2400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f33cc0b61d0 con 0x7f33cc0d1480 2024-04-28T13:31:22.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.715+0000 7f33d2400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f33cc15d700 con 0x7f33cc06e8f0 2024-04-28T13:31:22.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.715+0000 7f33d1000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f33cc0d10a0 0x7f33cc0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:22.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.715+0000 7f33d1000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f33cc0d10a0 0x7f33cc0b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35420/0 (socket says 172.21.15.196:35420) 2024-04-28T13:31:22.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.715+0000 7f33d1000640 1 -- 172.21.15.196:0/720532898 learned_addr learned my addr 172.21.15.196:0/720532898 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:22.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.716+0000 7f33d1a00640 1 --2- 172.21.15.196:0/720532898 >> v2:172.21.15.203:3300/0 conn(0x7f33cc0d1480 0x7f33cc160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:22.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.716+0000 7f33cbe00640 1 --2- 172.21.15.196:0/720532898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33cc06e8f0 0x7f33cc0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:22.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.716+0000 7f33d1000640 1 -- 172.21.15.196:0/720532898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33cc06e8f0 msgr2=0x7f33cc0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:22.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.716+0000 7f33d1000640 1 --2- 172.21.15.196:0/720532898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33cc06e8f0 0x7f33cc0b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.716+0000 7f33d1000640 1 -- 172.21.15.196:0/720532898 >> v2:172.21.15.203:3300/0 conn(0x7f33cc0d1480 msgr2=0x7f33cc160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:22.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.716+0000 7f33d1000640 1 --2- 172.21.15.196:0/720532898 >> v2:172.21.15.203:3300/0 conn(0x7f33cc0d1480 0x7f33cc160c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.716+0000 7f33d1000640 1 -- 172.21.15.196:0/720532898 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc15d9e0 con 0x7f33cc0d10a0 2024-04-28T13:31:22.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.716+0000 7f33cbe00640 1 --2- 172.21.15.196:0/720532898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33cc06e8f0 0x7f33cc0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:22.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.717+0000 7f33d1000640 1 --2- 172.21.15.196:0/720532898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33cc0d10a0 0x7f33cc0b3c90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f33cc161be0 tx=0x7f33c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce29aff36d19dad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:22.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.717+0000 7f33cb400640 1 -- 172.21.15.196:0/720532898 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0037070 con 0x7f33cc0d10a0 2024-04-28T13:31:22.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.717+0000 7f33cb400640 1 -- 172.21.15.196:0/720532898 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33c0032440 con 0x7f33cc0d10a0 2024-04-28T13:31:22.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.717+0000 7f33cb400640 1 -- 172.21.15.196:0/720532898 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0032810 con 0x7f33cc0d10a0 2024-04-28T13:31:22.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.718+0000 7f33d2400640 1 -- 172.21.15.196:0/720532898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33cc0d10a0 msgr2=0x7f33cc0b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:22.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.718+0000 7f33d2400640 1 --2- 172.21.15.196:0/720532898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33cc0d10a0 0x7f33cc0b3c90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f33cc161be0 tx=0x7f33c002a1e0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.718+0000 7f33d2400640 1 -- 172.21.15.196:0/720532898 shutdown_connections 2024-04-28T13:31:22.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.718+0000 7f33d2400640 1 --2- 172.21.15.196:0/720532898 >> v2:172.21.15.203:3300/0 conn(0x7f33cc0d1480 0x7f33cc160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.718+0000 7f33d2400640 1 --2- 172.21.15.196:0/720532898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33cc06e8f0 0x7f33cc0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.718+0000 7f33d2400640 1 --2- 172.21.15.196:0/720532898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33cc0d10a0 0x7f33cc0b3c90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.718+0000 7f33d2400640 1 -- 172.21.15.196:0/720532898 >> 172.21.15.196:0/720532898 conn(0x7f33cc06c6c0 msgr2=0x7f33cc06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:22.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.718+0000 7f33d2400640 1 -- 172.21.15.196:0/720532898 shutdown_connections 2024-04-28T13:31:22.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.718+0000 7f33d2400640 1 -- 172.21.15.196:0/720532898 wait complete. 2024-04-28T13:31:22.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.720+0000 7f33d2400640 1 Processor -- start 2024-04-28T13:31:22.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.720+0000 7f33d2400640 1 -- start start 2024-04-28T13:31:22.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.721+0000 7f33d2400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33cc06e8f0 0x7f33cc0cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:22.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.721+0000 7f33d2400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33cc0d10a0 0x7f33cc09ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:22.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.721+0000 7f33d2400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33cc0d1480 0x7f33cc1595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:22.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.721+0000 7f33d2400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc0d18f0 con 0x7f33cc06e8f0 2024-04-28T13:31:22.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.721+0000 7f33d2400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc15dfa0 con 0x7f33cc0d1480 2024-04-28T13:31:22.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.721+0000 7f33d2400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f33cc1611d0 con 0x7f33cc0d10a0 2024-04-28T13:31:22.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.721+0000 7f33d1000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33cc06e8f0 0x7f33cc0cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:22.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.721+0000 7f33d1000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33cc06e8f0 0x7f33cc0cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35432/0 (socket says 172.21.15.196:35432) 2024-04-28T13:31:22.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.721+0000 7f33d1000640 1 -- 172.21.15.196:0/1696094815 learned_addr learned my addr 172.21.15.196:0/1696094815 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:22.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.721+0000 7f33cbe00640 1 --2- 172.21.15.196:0/1696094815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33cc0d10a0 0x7f33cc09ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:22.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.721+0000 7f33d1a00640 1 --2- 172.21.15.196:0/1696094815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33cc0d1480 0x7f33cc1595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:22.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.721+0000 7f33d1000640 1 -- 172.21.15.196:0/1696094815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33cc0d10a0 msgr2=0x7f33cc09ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:22.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.721+0000 7f33d1000640 1 --2- 172.21.15.196:0/1696094815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33cc0d10a0 0x7f33cc09ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.722+0000 7f33d1000640 1 -- 172.21.15.196:0/1696094815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33cc0d1480 msgr2=0x7f33cc1595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:22.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.722+0000 7f33d1000640 1 --2- 172.21.15.196:0/1696094815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33cc0d1480 0x7f33cc1595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:22.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.722+0000 7f33d1000640 1 -- 172.21.15.196:0/1696094815 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc09bf80 con 0x7f33cc06e8f0 2024-04-28T13:31:22.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.722+0000 7f33d1000640 1 --2- 172.21.15.196:0/1696094815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33cc06e8f0 0x7f33cc0cb810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f33c0000c00 tx=0x7f33c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:22.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.722+0000 7f33c8c00640 1 -- 172.21.15.196:0/1696094815 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0041020 con 0x7f33cc06e8f0 2024-04-28T13:31:22.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.722+0000 7f33c8c00640 1 -- 172.21.15.196:0/1696094815 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33c00306c0 con 0x7f33cc06e8f0 2024-04-28T13:31:22.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.722+0000 7f33c8c00640 1 -- 172.21.15.196:0/1696094815 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0037040 con 0x7f33cc06e8f0 2024-04-28T13:31:22.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.722+0000 7f33d2400640 1 -- 172.21.15.196:0/1696094815 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33cc1580e0 con 0x7f33cc06e8f0 2024-04-28T13:31:22.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.722+0000 7f33d2400640 1 -- 172.21.15.196:0/1696094815 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33cc1586a0 con 0x7f33cc06e8f0 2024-04-28T13:31:22.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.723+0000 7f33d2400640 1 -- 172.21.15.196:0/1696094815 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33cc050730 con 0x7f33cc06e8f0 2024-04-28T13:31:22.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.724+0000 7f33c8c00640 1 -- 172.21.15.196:0/1696094815 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33c0044020 con 0x7f33cc06e8f0 2024-04-28T13:31:22.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.725+0000 7f33c8c00640 1 --2- 172.21.15.196:0/1696094815 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f33ac06c920 0x7f33ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:22.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.725+0000 7f33c8c00640 1 -- 172.21.15.196:0/1696094815 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f33c00ac460 con 0x7f33cc06e8f0 2024-04-28T13:31:22.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.726+0000 7f33cbe00640 1 --2- 172.21.15.196:0/1696094815 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f33ac06c920 0x7f33ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:22.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.727+0000 7f33cbe00640 1 --2- 172.21.15.196:0/1696094815 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f33ac06c920 0x7f33ac06ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f33b8006fd0 tx=0x7f33b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:22.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.729+0000 7f33c8c00640 1 -- 172.21.15.196:0/1696094815 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f33c004b030 con 0x7f33cc06e8f0 2024-04-28T13:31:22.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:22.969+0000 7f33d2400640 1 -- 172.21.15.196:0/1696094815 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f33cc06f700 con 0x7f33cc06e8f0 2024-04-28T13:31:23.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.638+0000 7f33c8c00640 1 -- 172.21.15.196:0/1696094815 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7f33c007a760 con 0x7f33cc06e8f0 2024-04-28T13:31:23.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.643+0000 7f33d2400640 1 -- 172.21.15.196:0/1696094815 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f33ac06c920 msgr2=0x7f33ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:23.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.643+0000 7f33d2400640 1 --2- 172.21.15.196:0/1696094815 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f33ac06c920 0x7f33ac06ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f33b8006fd0 tx=0x7f33b8008040 comp rx=0 tx=0).stop 2024-04-28T13:31:23.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.643+0000 7f33d2400640 1 -- 172.21.15.196:0/1696094815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33cc06e8f0 msgr2=0x7f33cc0cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:23.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.643+0000 7f33d2400640 1 --2- 172.21.15.196:0/1696094815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33cc06e8f0 0x7f33cc0cb810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f33c0000c00 tx=0x7f33c00046c0 comp rx=0 tx=0).stop 2024-04-28T13:31:23.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.643+0000 7f33d2400640 1 -- 172.21.15.196:0/1696094815 shutdown_connections 2024-04-28T13:31:23.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.643+0000 7f33d2400640 1 --2- 172.21.15.196:0/1696094815 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f33ac06c920 0x7f33ac06ede0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:23.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.643+0000 7f33d2400640 1 --2- 172.21.15.196:0/1696094815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33cc0d1480 0x7f33cc1595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:23.644 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.643+0000 7f33d2400640 1 --2- 172.21.15.196:0/1696094815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33cc0d10a0 0x7f33cc09ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:23.644 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.643+0000 7f33d2400640 1 --2- 172.21.15.196:0/1696094815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33cc06e8f0 0x7f33cc0cb810 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:23.644 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.643+0000 7f33d2400640 1 -- 172.21.15.196:0/1696094815 >> 172.21.15.196:0/1696094815 conn(0x7f33cc06c6c0 msgr2=0x7f33cc15fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:23.644 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.643+0000 7f33d2400640 1 -- 172.21.15.196:0/1696094815 shutdown_connections 2024-04-28T13:31:23.644 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.643+0000 7f33d2400640 1 -- 172.21.15.196:0/1696094815 wait complete. 2024-04-28T13:31:23.666 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-28T13:31:23.666 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-28T13:31:23.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.872+0000 7f3856e00640 1 Processor -- start 2024-04-28T13:31:23.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.872+0000 7f3856e00640 1 -- start start 2024-04-28T13:31:23.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.874+0000 7f3856e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38500b5df0 0x7f385006f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:23.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.874+0000 7f3856e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f385006df10 con 0x7f385015f150 2024-04-28T13:31:23.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.874+0000 7f3856e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f385006e1f0 con 0x7f38500d17b0 2024-04-28T13:31:23.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.874+0000 7f3856e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f385006fda0 con 0x7f38500b5df0 2024-04-28T13:31:23.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.874+0000 7f3855a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f385015f150 0x7f38500d10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:55964/0 (socket says 172.21.15.196:55964) 2024-04-28T13:31:23.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.874+0000 7f3855a00640 1 -- 172.21.15.196:0/1552860261 learned_addr learned my addr 172.21.15.196:0/1552860261 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:23.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.875+0000 7f3856400640 1 --2- 172.21.15.196:0/1552860261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38500b5df0 0x7f385006f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:23.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.875+0000 7f384fe00640 1 -- 172.21.15.196:0/1552860261 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2199674758 0 0) 0x7f385006df10 con 0x7f385015f150 2024-04-28T13:31:23.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.875+0000 7f384fe00640 1 -- 172.21.15.196:0/1552860261 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3838003520 con 0x7f385015f150 2024-04-28T13:31:23.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.875+0000 7f384fe00640 1 -- 172.21.15.196:0/1552860261 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1870904983 0 0) 0x7f3838003520 con 0x7f385015f150 2024-04-28T13:31:23.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.875+0000 7f384fe00640 1 -- 172.21.15.196:0/1552860261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38500b5df0 msgr2=0x7f385006f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:23.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.875+0000 7f384fe00640 1 --2- 172.21.15.196:0/1552860261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38500b5df0 0x7f385006f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:23.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.875+0000 7f384fe00640 1 -- 172.21.15.196:0/1552860261 >> v1:172.21.15.203:6789/0 conn(0x7f38500d17b0 legacy=0x7f3850072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:23.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.875+0000 7f384fe00640 1 -- 172.21.15.196:0/1552860261 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385006e4d0 con 0x7f385015f150 2024-04-28T13:31:23.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.875+0000 7f384fe00640 1 -- 172.21.15.196:0/1552860261 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f3844001670 con 0x7f385015f150 2024-04-28T13:31:23.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.875+0000 7f384fe00640 1 -- 172.21.15.196:0/1552860261 >> v1:172.21.15.196:6789/0 conn(0x7f385015f150 legacy=0x7f38500d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:23.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.875+0000 7f384fe00640 1 --2- 172.21.15.196:0/1552860261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38380039e0 0x7f3838003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:23.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.875+0000 7f384fe00640 1 --2- 172.21.15.196:0/1552860261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3838004800 0x7f3838004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:23.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.876+0000 7f384fe00640 1 --2- 172.21.15.196:0/1552860261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38500b5df0 0x7f38380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:23.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.876+0000 7f384fe00640 1 -- 172.21.15.196:0/1552860261 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f385006fda0 con 0x7f3838004800 2024-04-28T13:31:23.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.876+0000 7f384fe00640 1 -- 172.21.15.196:0/1552860261 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f383800ba70 con 0x7f38380039e0 2024-04-28T13:31:23.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.876+0000 7f384fe00640 1 -- 172.21.15.196:0/1552860261 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f383800bc70 con 0x7f38500b5df0 2024-04-28T13:31:23.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.876+0000 7f3855000640 1 --2- 172.21.15.196:0/1552860261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3838004800 0x7f3838004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:23.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.876+0000 7f3855a00640 1 --2- 172.21.15.196:0/1552860261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38380039e0 0x7f3838003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:23.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.876+0000 7f3856400640 1 --2- 172.21.15.196:0/1552860261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38500b5df0 0x7f38380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:23.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.876+0000 7f3855000640 1 -- 172.21.15.196:0/1552860261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38500b5df0 msgr2=0x7f38380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:23.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.876+0000 7f3855000640 1 --2- 172.21.15.196:0/1552860261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38500b5df0 0x7f38380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:23.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.876+0000 7f3855000640 1 -- 172.21.15.196:0/1552860261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38380039e0 msgr2=0x7f3838003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:23.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.876+0000 7f3855000640 1 --2- 172.21.15.196:0/1552860261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38380039e0 0x7f3838003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:23.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.876+0000 7f3855000640 1 -- 172.21.15.196:0/1552860261 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3838003520 con 0x7f3838004800 2024-04-28T13:31:23.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.876+0000 7f3855a00640 1 --2- 172.21.15.196:0/1552860261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38380039e0 0x7f3838003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:31:23.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.877+0000 7f3855000640 1 --2- 172.21.15.196:0/1552860261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3838004800 0x7f3838004c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f383c001720 tx=0x7f3838005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6a9089dcc83e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:23.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.878+0000 7f384fe00640 1 -- 172.21.15.196:0/1552860261 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c001370 con 0x7f3838004800 2024-04-28T13:31:23.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.878+0000 7f384fe00640 1 -- 172.21.15.196:0/1552860261 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383c0052d0 con 0x7f3838004800 2024-04-28T13:31:23.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.878+0000 7f384fe00640 1 -- 172.21.15.196:0/1552860261 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c02ab90 con 0x7f3838004800 2024-04-28T13:31:23.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.878+0000 7f3856e00640 1 -- 172.21.15.196:0/1552860261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3838004800 msgr2=0x7f3838004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:23.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.878+0000 7f3856e00640 1 --2- 172.21.15.196:0/1552860261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3838004800 0x7f3838004c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f383c001720 tx=0x7f3838005830 comp rx=0 tx=0).stop 2024-04-28T13:31:23.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.878+0000 7f3856400640 1 -- 172.21.15.196:0/1552860261 reap_dead start 2024-04-28T13:31:23.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.878+0000 7f3856e00640 1 -- 172.21.15.196:0/1552860261 shutdown_connections 2024-04-28T13:31:23.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.878+0000 7f3856e00640 1 -- 172.21.15.196:0/1552860261 >> 172.21.15.196:0/1552860261 conn(0x7f385006c6c0 msgr2=0x7f385006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:23.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.878+0000 7f3856e00640 1 -- 172.21.15.196:0/1552860261 shutdown_connections 2024-04-28T13:31:23.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.879+0000 7f3856e00640 1 -- 172.21.15.196:0/1552860261 wait complete. 2024-04-28T13:31:23.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.880+0000 7f3856e00640 1 Processor -- start 2024-04-28T13:31:23.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.880+0000 7f3856e00640 1 -- start start 2024-04-28T13:31:23.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.881+0000 7f3856e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3838004800 0x7f3850152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:23.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.881+0000 7f3856e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3850152730 0x7f38501608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:23.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.881+0000 7f3856e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3850160e30 0x7f38500b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:23.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.881+0000 7f3856e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f385015f530 con 0x7f3838004800 2024-04-28T13:31:23.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.881+0000 7f3856e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f385006ea90 con 0x7f3850152730 2024-04-28T13:31:23.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.881+0000 7f3856e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f385006f860 con 0x7f3850160e30 2024-04-28T13:31:23.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.882+0000 7f3855a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3838004800 0x7f3850152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:23.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.882+0000 7f3855a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3838004800 0x7f3850152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35458/0 (socket says 172.21.15.196:35458) 2024-04-28T13:31:23.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.882+0000 7f3855a00640 1 -- 172.21.15.196:0/2846613596 learned_addr learned my addr 172.21.15.196:0/2846613596 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:23.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.882+0000 7f3855a00640 1 -- 172.21.15.196:0/2846613596 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3850160e30 msgr2=0x7f38500b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:23.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.882+0000 7f3855a00640 1 --2- 172.21.15.196:0/2846613596 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3850160e30 0x7f38500b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:23.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.882+0000 7f3855a00640 1 -- 172.21.15.196:0/2846613596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3850152730 msgr2=0x7f38501608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:23.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.882+0000 7f3855a00640 1 --2- 172.21.15.196:0/2846613596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3850152730 0x7f38501608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:23.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.882+0000 7f3855a00640 1 -- 172.21.15.196:0/2846613596 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385015f850 con 0x7f3838004800 2024-04-28T13:31:23.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.882+0000 7f3855a00640 1 --2- 172.21.15.196:0/2846613596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3838004800 0x7f3850152010 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f384400e780 tx=0x7f384400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:23.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.882+0000 7f384d600640 1 -- 172.21.15.196:0/2846613596 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384400ee30 con 0x7f3838004800 2024-04-28T13:31:23.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.882+0000 7f384d600640 1 -- 172.21.15.196:0/2846613596 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3844018070 con 0x7f3838004800 2024-04-28T13:31:23.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.882+0000 7f384d600640 1 -- 172.21.15.196:0/2846613596 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384400ca70 con 0x7f3838004800 2024-04-28T13:31:23.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.882+0000 7f3856e00640 1 -- 172.21.15.196:0/2846613596 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385015fb30 con 0x7f3838004800 2024-04-28T13:31:23.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.883+0000 7f3856e00640 1 -- 172.21.15.196:0/2846613596 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38501600f0 con 0x7f3838004800 2024-04-28T13:31:23.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.884+0000 7f3856e00640 1 -- 172.21.15.196:0/2846613596 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3850050730 con 0x7f3838004800 2024-04-28T13:31:23.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.885+0000 7f384d600640 1 -- 172.21.15.196:0/2846613596 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f384400cc10 con 0x7f3838004800 2024-04-28T13:31:23.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.885+0000 7f384d600640 1 --2- 172.21.15.196:0/2846613596 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f383006c8f0 0x7f383006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:23.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.886+0000 7f384d600640 1 -- 172.21.15.196:0/2846613596 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f384408ba80 con 0x7f3838004800 2024-04-28T13:31:23.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.886+0000 7f3855000640 1 --2- 172.21.15.196:0/2846613596 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f383006c8f0 0x7f383006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:23.887 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.887+0000 7f3855000640 1 --2- 172.21.15.196:0/2846613596 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f383006c8f0 0x7f383006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f383c032000 tx=0x7f383c02c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:23.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:23.890+0000 7f384d600640 1 -- 172.21.15.196:0/2846613596 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f384408de20 con 0x7f3838004800 2024-04-28T13:31:24.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.130+0000 7f3856e00640 1 -- 172.21.15.196:0/2846613596 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f385006cfa0 con 0x7f3838004800 2024-04-28T13:31:24.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.659+0000 7f384d600640 1 -- 172.21.15.196:0/2846613596 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f3844059d80 con 0x7f3838004800 2024-04-28T13:31:24.663 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.663+0000 7f3856e00640 1 -- 172.21.15.196:0/2846613596 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f383006c8f0 msgr2=0x7f383006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:24.664 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.663+0000 7f3856e00640 1 --2- 172.21.15.196:0/2846613596 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f383006c8f0 0x7f383006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f383c032000 tx=0x7f383c02c000 comp rx=0 tx=0).stop 2024-04-28T13:31:24.664 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.663+0000 7f3856e00640 1 -- 172.21.15.196:0/2846613596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3838004800 msgr2=0x7f3850152010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:24.664 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.663+0000 7f3856e00640 1 --2- 172.21.15.196:0/2846613596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3838004800 0x7f3850152010 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f384400e780 tx=0x7f384400ec40 comp rx=0 tx=0).stop 2024-04-28T13:31:24.664 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.663+0000 7f3856e00640 1 -- 172.21.15.196:0/2846613596 shutdown_connections 2024-04-28T13:31:24.664 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.663+0000 7f3856e00640 1 --2- 172.21.15.196:0/2846613596 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f383006c8f0 0x7f383006edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:24.664 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.663+0000 7f3856e00640 1 --2- 172.21.15.196:0/2846613596 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3850160e30 0x7f38500b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:24.664 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.663+0000 7f3856e00640 1 --2- 172.21.15.196:0/2846613596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3850152730 0x7f38501608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:24.665 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.663+0000 7f3856e00640 1 --2- 172.21.15.196:0/2846613596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3838004800 0x7f3850152010 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:24.665 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.663+0000 7f3856e00640 1 -- 172.21.15.196:0/2846613596 >> 172.21.15.196:0/2846613596 conn(0x7f385006c6c0 msgr2=0x7f385015d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:24.665 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.664+0000 7f3856e00640 1 -- 172.21.15.196:0/2846613596 shutdown_connections 2024-04-28T13:31:24.665 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.664+0000 7f3856e00640 1 -- 172.21.15.196:0/2846613596 wait complete. 2024-04-28T13:31:24.686 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-28T13:31:24.686 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:31:24.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.884+0000 7f4cf5000640 1 Processor -- start 2024-04-28T13:31:24.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.884+0000 7f4cf5000640 1 -- start start 2024-04-28T13:31:24.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.886+0000 7f4cf5000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4cf00cfda0 0x7f4cf00d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:24.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.886+0000 7f4cf5000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf00d0220 con 0x7f4cf00d1b80 2024-04-28T13:31:24.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.886+0000 7f4cf5000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf00d0500 con 0x7f4cf00b60a0 2024-04-28T13:31:24.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.886+0000 7f4cf5000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf00d07e0 con 0x7f4cf00cfda0 2024-04-28T13:31:24.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.886+0000 7f4cede00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f4cf00d1b80 0x7f4cf00d3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:55966/0 (socket says 172.21.15.196:55966) 2024-04-28T13:31:24.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.886+0000 7f4cede00640 1 -- 172.21.15.196:0/2505742262 learned_addr learned my addr 172.21.15.196:0/2505742262 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:24.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ced400640 1 --2- 172.21.15.196:0/2505742262 >> v2:172.21.15.203:3300/0 conn(0x7f4cf00cfda0 0x7f4cf00d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:24.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.886+0000 7f4ceca00640 1 -- 172.21.15.196:0/2505742262 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2710993011 0 0) 0x7f4cf00d0220 con 0x7f4cf00d1b80 2024-04-28T13:31:24.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ceca00640 1 -- 172.21.15.196:0/2505742262 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cd0003520 con 0x7f4cf00d1b80 2024-04-28T13:31:24.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ceca00640 1 -- 172.21.15.196:0/2505742262 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3457578878 0 0) 0x7f4cd0003520 con 0x7f4cf00d1b80 2024-04-28T13:31:24.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ceca00640 1 -- 172.21.15.196:0/2505742262 >> v2:172.21.15.203:3300/0 conn(0x7f4cf00cfda0 msgr2=0x7f4cf00d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:24.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ceca00640 1 --2- 172.21.15.196:0/2505742262 >> v2:172.21.15.203:3300/0 conn(0x7f4cf00cfda0 0x7f4cf00d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:24.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ceca00640 1 -- 172.21.15.196:0/2505742262 >> v1:172.21.15.203:6789/0 conn(0x7f4cf00b60a0 legacy=0x7f4cf00b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:24.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ceca00640 1 -- 172.21.15.196:0/2505742262 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf00d0ac0 con 0x7f4cf00d1b80 2024-04-28T13:31:24.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ceca00640 1 -- 172.21.15.196:0/2505742262 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f4ce0001670 con 0x7f4cf00d1b80 2024-04-28T13:31:24.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ceca00640 1 -- 172.21.15.196:0/2505742262 >> v1:172.21.15.196:6789/0 conn(0x7f4cf00d1b80 legacy=0x7f4cf00d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:24.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ceca00640 1 --2- 172.21.15.196:0/2505742262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cd00039b0 0x7f4cd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:24.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ceca00640 1 --2- 172.21.15.196:0/2505742262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd0004800 0x7f4cd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:24.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ceca00640 1 --2- 172.21.15.196:0/2505742262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cd00051c0 0x7f4cd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:24.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ceca00640 1 -- 172.21.15.196:0/2505742262 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf00d07e0 con 0x7f4cd0004800 2024-04-28T13:31:24.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ceca00640 1 -- 172.21.15.196:0/2505742262 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd000c060 con 0x7f4cd00039b0 2024-04-28T13:31:24.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ceca00640 1 -- 172.21.15.196:0/2505742262 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd000c260 con 0x7f4cd00051c0 2024-04-28T13:31:24.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.887+0000 7f4ced400640 1 --2- 172.21.15.196:0/2505742262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd0004800 0x7f4cd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:24.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.888+0000 7f4cede00640 1 --2- 172.21.15.196:0/2505742262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cd00039b0 0x7f4cd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:24.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.888+0000 7f4ced400640 1 -- 172.21.15.196:0/2505742262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cd00051c0 msgr2=0x7f4cd000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:24.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.888+0000 7f4ced400640 1 --2- 172.21.15.196:0/2505742262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cd00051c0 0x7f4cd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:24.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.888+0000 7f4ced400640 1 -- 172.21.15.196:0/2505742262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cd00039b0 msgr2=0x7f4cd0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:24.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.888+0000 7f4ced400640 1 --2- 172.21.15.196:0/2505742262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cd00039b0 0x7f4cd0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:24.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.888+0000 7f4ced400640 1 -- 172.21.15.196:0/2505742262 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd0003520 con 0x7f4cd0004800 2024-04-28T13:31:24.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.888+0000 7f4cede00640 1 --2- 172.21.15.196:0/2505742262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cd00039b0 0x7f4cd0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:24.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.888+0000 7f4cee800640 1 -- 172.21.15.196:0/2505742262 reap_dead start 2024-04-28T13:31:24.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.889+0000 7f4ced400640 1 --2- 172.21.15.196:0/2505742262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd0004800 0x7f4cd0004c80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8009fc0 tx=0x7f4cd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da30af35fbea51 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:24.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.889+0000 7f4ceca00640 1 -- 172.21.15.196:0/2505742262 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8009c10 con 0x7f4cd0004800 2024-04-28T13:31:24.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.889+0000 7f4ceca00640 1 -- 172.21.15.196:0/2505742262 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd8036070 con 0x7f4cd0004800 2024-04-28T13:31:24.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.889+0000 7f4ceca00640 1 -- 172.21.15.196:0/2505742262 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8031880 con 0x7f4cd0004800 2024-04-28T13:31:24.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.890+0000 7f4cf5000640 1 -- 172.21.15.196:0/2505742262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd0004800 msgr2=0x7f4cd0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:24.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.890+0000 7f4cf5000640 1 --2- 172.21.15.196:0/2505742262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd0004800 0x7f4cd0004c80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8009fc0 tx=0x7f4cd0005830 comp rx=0 tx=0).stop 2024-04-28T13:31:24.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.890+0000 7f4cf5000640 1 -- 172.21.15.196:0/2505742262 shutdown_connections 2024-04-28T13:31:24.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.890+0000 7f4cf5000640 1 --2- 172.21.15.196:0/2505742262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd0004800 0x7f4cd0004c80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:24.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.890+0000 7f4cf5000640 1 -- 172.21.15.196:0/2505742262 >> 172.21.15.196:0/2505742262 conn(0x7f4cf009e540 msgr2=0x7f4cf00d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:24.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.890+0000 7f4cf5000640 1 -- 172.21.15.196:0/2505742262 shutdown_connections 2024-04-28T13:31:24.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.890+0000 7f4cf5000640 1 -- 172.21.15.196:0/2505742262 wait complete. 2024-04-28T13:31:24.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.892+0000 7f4cf5000640 1 Processor -- start 2024-04-28T13:31:24.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.892+0000 7f4cf5000640 1 -- start start 2024-04-28T13:31:24.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.893+0000 7f4cf5000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd0004800 0x7f4cf00c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:24.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.893+0000 7f4cf5000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cf015b1b0 0x7f4cf0158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:24.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.893+0000 7f4cf5000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf00cdc50 0x7f4cf00cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:24.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.893+0000 7f4cf5000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf00d20e0 con 0x7f4cd0004800 2024-04-28T13:31:24.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.893+0000 7f4cf5000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf00d1f60 con 0x7f4cf00cdc50 2024-04-28T13:31:24.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.893+0000 7f4cf5000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf00d2260 con 0x7f4cf015b1b0 2024-04-28T13:31:24.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.893+0000 7f4cede00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd0004800 0x7f4cf00c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:24.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.893+0000 7f4cede00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd0004800 0x7f4cf00c8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35474/0 (socket says 172.21.15.196:35474) 2024-04-28T13:31:24.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.893+0000 7f4cede00640 1 -- 172.21.15.196:0/4186316224 learned_addr learned my addr 172.21.15.196:0/4186316224 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:24.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.893+0000 7f4cee800640 1 --2- 172.21.15.196:0/4186316224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf00cdc50 0x7f4cf00cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:24.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.893+0000 7f4cede00640 1 -- 172.21.15.196:0/4186316224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cf015b1b0 msgr2=0x7f4cf0158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:24.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.893+0000 7f4cede00640 1 --2- 172.21.15.196:0/4186316224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cf015b1b0 0x7f4cf0158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:24.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.893+0000 7f4cede00640 1 -- 172.21.15.196:0/4186316224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf00cdc50 msgr2=0x7f4cf00cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:24.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.893+0000 7f4cede00640 1 --2- 172.21.15.196:0/4186316224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf00cdc50 0x7f4cf00cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:24.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.893+0000 7f4cede00640 1 -- 172.21.15.196:0/4186316224 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf015c7a0 con 0x7f4cd0004800 2024-04-28T13:31:24.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.894+0000 7f4cede00640 1 --2- 172.21.15.196:0/4186316224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd0004800 0x7f4cf00c8d70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000d930 tx=0x7f4ce000ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:24.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.894+0000 7f4ce6000640 1 -- 172.21.15.196:0/4186316224 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0019070 con 0x7f4cd0004800 2024-04-28T13:31:24.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.894+0000 7f4ce6000640 1 -- 172.21.15.196:0/4186316224 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce0007560 con 0x7f4cd0004800 2024-04-28T13:31:24.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.894+0000 7f4ce6000640 1 -- 172.21.15.196:0/4186316224 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00049e0 con 0x7f4cd0004800 2024-04-28T13:31:24.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.894+0000 7f4cf5000640 1 -- 172.21.15.196:0/4186316224 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf015b640 con 0x7f4cd0004800 2024-04-28T13:31:24.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.894+0000 7f4cf5000640 1 -- 172.21.15.196:0/4186316224 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf015bc00 con 0x7f4cd0004800 2024-04-28T13:31:24.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.895+0000 7f4cf5000640 1 -- 172.21.15.196:0/4186316224 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf006a290 con 0x7f4cd0004800 2024-04-28T13:31:24.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.896+0000 7f4ce6000640 1 -- 172.21.15.196:0/4186316224 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ce0004b80 con 0x7f4cd0004800 2024-04-28T13:31:24.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.897+0000 7f4ce6000640 1 --2- 172.21.15.196:0/4186316224 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4ccc06c920 0x7f4ccc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:24.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.897+0000 7f4ce6000640 1 -- 172.21.15.196:0/4186316224 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4ce008b640 con 0x7f4cd0004800 2024-04-28T13:31:24.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.897+0000 7f4ced400640 1 --2- 172.21.15.196:0/4186316224 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4ccc06c920 0x7f4ccc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:24.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.899+0000 7f4ced400640 1 --2- 172.21.15.196:0/4186316224 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4ccc06c920 0x7f4ccc06ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8035000 tx=0x7f4cd802d000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:24.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:24.901+0000 7f4ce6000640 1 -- 172.21.15.196:0/4186316224 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ce004fc00 con 0x7f4cd0004800 2024-04-28T13:31:25.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.140+0000 7f4cf5000640 1 -- 172.21.15.196:0/4186316224 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cf00d27b0 con 0x7f4cd0004800 2024-04-28T13:31:25.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.142+0000 7f4ce6000640 1 -- 172.21.15.196:0/4186316224 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f4ce0059b00 con 0x7f4cd0004800 2024-04-28T13:31:25.145 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:25.145 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/422653807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":422653807},{"type":"v1","addr":"172.21.15.196:6834","nonce":422653807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2793552376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2793552376},{"type":"v1","addr":"172.21.15.196:6835","nonce":2793552376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1810956193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1810956193},{"type":"v1","addr":"172.21.15.203:6837","nonce":1810956193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:31:21.059586+0000","modified":"2024-04-28T13:31:24.132134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/2114798853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2114798853},{"type":"v1","addr":"172.21.15.203:6836","nonce":2114798853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4253,"qdb_cluster":[4253]},"id":1}]} 2024-04-28T13:31:25.150 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 7 2024-04-28T13:31:25.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.144+0000 7f4cf5000640 1 -- 172.21.15.196:0/4186316224 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4ccc06c920 msgr2=0x7f4ccc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:25.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.144+0000 7f4cf5000640 1 --2- 172.21.15.196:0/4186316224 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4ccc06c920 0x7f4ccc06ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8035000 tx=0x7f4cd802d000 comp rx=0 tx=0).stop 2024-04-28T13:31:25.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.144+0000 7f4cf5000640 1 -- 172.21.15.196:0/4186316224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd0004800 msgr2=0x7f4cf00c8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:25.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.144+0000 7f4cf5000640 1 --2- 172.21.15.196:0/4186316224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd0004800 0x7f4cf00c8d70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000d930 tx=0x7f4ce000ddf0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.145+0000 7f4cf5000640 1 -- 172.21.15.196:0/4186316224 shutdown_connections 2024-04-28T13:31:25.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.145+0000 7f4cf5000640 1 --2- 172.21.15.196:0/4186316224 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4ccc06c920 0x7f4ccc06ede0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.145+0000 7f4cf5000640 1 --2- 172.21.15.196:0/4186316224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf00cdc50 0x7f4cf00cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.145+0000 7f4cf5000640 1 --2- 172.21.15.196:0/4186316224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cf015b1b0 0x7f4cf0158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.145+0000 7f4cf5000640 1 --2- 172.21.15.196:0/4186316224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd0004800 0x7f4cf00c8d70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.145+0000 7f4cf5000640 1 -- 172.21.15.196:0/4186316224 >> 172.21.15.196:0/4186316224 conn(0x7f4cf009e540 msgr2=0x7f4cf00ba010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:25.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.145+0000 7f4cf5000640 1 -- 172.21.15.196:0/4186316224 shutdown_connections 2024-04-28T13:31:25.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.145+0000 7f4cf5000640 1 -- 172.21.15.196:0/4186316224 wait complete. 2024-04-28T13:31:25.164 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:25.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.367+0000 7f579fa00640 1 Processor -- start 2024-04-28T13:31:25.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.367+0000 7f579fa00640 1 -- start start 2024-04-28T13:31:25.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.369+0000 7f579fa00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5798145870 0x7f579806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:25.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.369+0000 7f579fa00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5798073420 0x7f579815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:25.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.369+0000 7f579fa00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5798073a40 con 0x7f579806f980 2024-04-28T13:31:25.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.369+0000 7f579fa00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f579806fde0 con 0x7f5798073420 2024-04-28T13:31:25.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.369+0000 7f579fa00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f579815d8d0 con 0x7f5798145870 2024-04-28T13:31:25.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.369+0000 7f579da00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5798073420 0x7f579815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:25.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.369+0000 7f579da00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5798073420 0x7f579815e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35484/0 (socket says 172.21.15.196:35484) 2024-04-28T13:31:25.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.369+0000 7f579da00640 1 -- 172.21.15.196:0/3143353023 learned_addr learned my addr 172.21.15.196:0/3143353023 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:25.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.369+0000 7f5797e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f579806f980 0x7f5798072d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:55968/0 (socket says 172.21.15.196:55968) 2024-04-28T13:31:25.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.369+0000 7f579d000640 1 --2- 172.21.15.196:0/3143353023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5798145870 0x7f579806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:25.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.369+0000 7f579da00640 1 -- 172.21.15.196:0/3143353023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5798145870 msgr2=0x7f579806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:25.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.370+0000 7f579da00640 1 --2- 172.21.15.196:0/3143353023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5798145870 0x7f579806f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.370+0000 7f579da00640 1 -- 172.21.15.196:0/3143353023 >> v1:172.21.15.196:6789/0 conn(0x7f579806f980 legacy=0x7f5798072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:25.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.370+0000 7f579da00640 1 -- 172.21.15.196:0/3143353023 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579815dbb0 con 0x7f5798073420 2024-04-28T13:31:25.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.370+0000 7f5797400640 1 -- 172.21.15.196:0/3143353023 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2055849865 0 0) 0x7f5798073a40 con 0x7f579806f980 2024-04-28T13:31:25.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.371+0000 7f579da00640 1 --2- 172.21.15.196:0/3143353023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5798073420 0x7f579815e3b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5790009e50 tx=0x7f579002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53679af3203e7f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:25.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.371+0000 7f5797400640 1 -- 172.21.15.196:0/3143353023 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790036070 con 0x7f5798073420 2024-04-28T13:31:25.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.371+0000 7f5797400640 1 -- 172.21.15.196:0/3143353023 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5790031600 con 0x7f5798073420 2024-04-28T13:31:25.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.371+0000 7f5797400640 1 -- 172.21.15.196:0/3143353023 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57900319d0 con 0x7f5798073420 2024-04-28T13:31:25.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.371+0000 7f579fa00640 1 -- 172.21.15.196:0/3143353023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5798073420 msgr2=0x7f579815e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:25.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.371+0000 7f579fa00640 1 --2- 172.21.15.196:0/3143353023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5798073420 0x7f579815e3b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5790009e50 tx=0x7f579002a710 comp rx=0 tx=0).stop 2024-04-28T13:31:25.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.372+0000 7f579fa00640 1 -- 172.21.15.196:0/3143353023 shutdown_connections 2024-04-28T13:31:25.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.372+0000 7f579fa00640 1 --2- 172.21.15.196:0/3143353023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5798073420 0x7f579815e3b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.372+0000 7f579fa00640 1 --2- 172.21.15.196:0/3143353023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5798145870 0x7f579806f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.372+0000 7f579fa00640 1 -- 172.21.15.196:0/3143353023 >> 172.21.15.196:0/3143353023 conn(0x7f57980cd830 msgr2=0x7f57980cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:25.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.372+0000 7f579fa00640 1 -- 172.21.15.196:0/3143353023 shutdown_connections 2024-04-28T13:31:25.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.372+0000 7f579fa00640 1 -- 172.21.15.196:0/3143353023 wait complete. 2024-04-28T13:31:25.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.373+0000 7f579fa00640 1 Processor -- start 2024-04-28T13:31:25.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.373+0000 7f579fa00640 1 -- start start 2024-04-28T13:31:25.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.374+0000 7f579fa00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f579806f980 0x7f5798152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:25.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.374+0000 7f579fa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5798073420 0x7f57981591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:25.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.374+0000 7f579fa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5798145870 0x7f57980b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:25.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.374+0000 7f579fa00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5798145c50 con 0x7f5798073420 2024-04-28T13:31:25.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.374+0000 7f579fa00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f579815d5d0 con 0x7f5798145870 2024-04-28T13:31:25.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.374+0000 7f579fa00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f579815e8f0 con 0x7f579806f980 2024-04-28T13:31:25.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.375+0000 7f5797e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5798073420 0x7f57981591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:25.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.375+0000 7f5797e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5798073420 0x7f57981591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35498/0 (socket says 172.21.15.196:35498) 2024-04-28T13:31:25.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.375+0000 7f5797e00640 1 -- 172.21.15.196:0/917854773 learned_addr learned my addr 172.21.15.196:0/917854773 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:25.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.375+0000 7f579da00640 1 --2- 172.21.15.196:0/917854773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5798145870 0x7f57980b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:25.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.375+0000 7f5797e00640 1 -- 172.21.15.196:0/917854773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f579806f980 msgr2=0x7f5798152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:25.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.375+0000 7f5797e00640 1 --2- 172.21.15.196:0/917854773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f579806f980 0x7f5798152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.375+0000 7f5797e00640 1 -- 172.21.15.196:0/917854773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5798145870 msgr2=0x7f57980b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:25.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.375+0000 7f5797e00640 1 --2- 172.21.15.196:0/917854773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5798145870 0x7f57980b22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.375+0000 7f5797e00640 1 -- 172.21.15.196:0/917854773 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57980b5b90 con 0x7f5798073420 2024-04-28T13:31:25.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.375+0000 7f5797e00640 1 --2- 172.21.15.196:0/917854773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5798073420 0x7f57981591d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f578400c4d0 tx=0x7f578400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:25.378 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.376+0000 7f5794c00640 1 -- 172.21.15.196:0/917854773 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5784018070 con 0x7f5798073420 2024-04-28T13:31:25.378 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.376+0000 7f5794c00640 1 -- 172.21.15.196:0/917854773 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5784007500 con 0x7f5798073420 2024-04-28T13:31:25.378 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.376+0000 7f5794c00640 1 -- 172.21.15.196:0/917854773 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5784004a20 con 0x7f5798073420 2024-04-28T13:31:25.378 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.376+0000 7f579fa00640 1 -- 172.21.15.196:0/917854773 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57980b5e70 con 0x7f5798073420 2024-04-28T13:31:25.378 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.376+0000 7f579fa00640 1 -- 172.21.15.196:0/917854773 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5798152b90 con 0x7f5798073420 2024-04-28T13:31:25.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.377+0000 7f579fa00640 1 -- 172.21.15.196:0/917854773 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f579806a290 con 0x7f5798073420 2024-04-28T13:31:25.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.378+0000 7f5794c00640 1 -- 172.21.15.196:0/917854773 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5784004bc0 con 0x7f5798073420 2024-04-28T13:31:25.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.379+0000 7f5794c00640 1 --2- 172.21.15.196:0/917854773 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f577806c920 0x7f577806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:25.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.379+0000 7f5794c00640 1 -- 172.21.15.196:0/917854773 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f578408ae90 con 0x7f5798073420 2024-04-28T13:31:25.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.380+0000 7f579d000640 1 --2- 172.21.15.196:0/917854773 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f577806c920 0x7f577806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:25.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.381+0000 7f579d000640 1 --2- 172.21.15.196:0/917854773 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f577806c920 0x7f577806ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f578c006fd0 tx=0x7f578c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:25.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.383+0000 7f5794c00640 1 -- 172.21.15.196:0/917854773 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5784059220 con 0x7f5798073420 2024-04-28T13:31:25.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.582+0000 7f579fa00640 1 -- 172.21.15.196:0/917854773 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57981e59f0 con 0x7f5798073420 2024-04-28T13:31:25.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.585+0000 7f5794c00640 1 -- 172.21.15.196:0/917854773 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f5784058b20 con 0x7f5798073420 2024-04-28T13:31:25.585 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:25.585 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":20,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:22.624564+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:25.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.587+0000 7f579fa00640 1 -- 172.21.15.196:0/917854773 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f577806c920 msgr2=0x7f577806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:25.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.587+0000 7f579fa00640 1 --2- 172.21.15.196:0/917854773 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f577806c920 0x7f577806ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f578c006fd0 tx=0x7f578c008040 comp rx=0 tx=0).stop 2024-04-28T13:31:25.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.587+0000 7f579fa00640 1 -- 172.21.15.196:0/917854773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5798073420 msgr2=0x7f57981591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:25.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.587+0000 7f579fa00640 1 --2- 172.21.15.196:0/917854773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5798073420 0x7f57981591d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f578400c4d0 tx=0x7f578400c990 comp rx=0 tx=0).stop 2024-04-28T13:31:25.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.588+0000 7f579fa00640 1 -- 172.21.15.196:0/917854773 shutdown_connections 2024-04-28T13:31:25.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.588+0000 7f579fa00640 1 --2- 172.21.15.196:0/917854773 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f577806c920 0x7f577806ede0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.588+0000 7f579fa00640 1 --2- 172.21.15.196:0/917854773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5798145870 0x7f57980b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.588+0000 7f579fa00640 1 --2- 172.21.15.196:0/917854773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5798073420 0x7f57981591d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.588+0000 7f579fa00640 1 --2- 172.21.15.196:0/917854773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f579806f980 0x7f5798152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.588+0000 7f579fa00640 1 -- 172.21.15.196:0/917854773 >> 172.21.15.196:0/917854773 conn(0x7f57980cd830 msgr2=0x7f57980cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:25.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.588+0000 7f579fa00640 1 -- 172.21.15.196:0/917854773 shutdown_connections 2024-04-28T13:31:25.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.588+0000 7f579fa00640 1 -- 172.21.15.196:0/917854773 wait complete. 2024-04-28T13:31:25.606 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-28T13:31:25.606 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T13:31:25.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.805+0000 7fa94ee00640 1 Processor -- start 2024-04-28T13:31:25.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.806+0000 7fa94ee00640 1 -- start start 2024-04-28T13:31:25.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.807+0000 7fa94ee00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa9480d1a50 0x7fa948167e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:25.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.807+0000 7fa94ee00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9480cff40 con 0x7fa94815d210 2024-04-28T13:31:25.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.807+0000 7fa94ee00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9480d0220 con 0x7fa9480d38f0 2024-04-28T13:31:25.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.807+0000 7fa94ee00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa948070440 con 0x7fa9480d1a50 2024-04-28T13:31:25.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.807+0000 7fa94da00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa94815d210 0x7fa9480d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:55984/0 (socket says 172.21.15.196:55984) 2024-04-28T13:31:25.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.807+0000 7fa94da00640 1 -- 172.21.15.196:0/278899346 learned_addr learned my addr 172.21.15.196:0/278899346 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:25.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa947e00640 1 -- 172.21.15.196:0/278899346 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2213719845 0 0) 0x7fa9480cff40 con 0x7fa94815d210 2024-04-28T13:31:25.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa947e00640 1 -- 172.21.15.196:0/278899346 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa92c003520 con 0x7fa94815d210 2024-04-28T13:31:25.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa94e400640 1 --2- 172.21.15.196:0/278899346 >> v2:172.21.15.203:3300/0 conn(0x7fa9480d1a50 0x7fa948167e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:25.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa947e00640 1 -- 172.21.15.196:0/278899346 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1984031822 0 0) 0x7fa92c003520 con 0x7fa94815d210 2024-04-28T13:31:25.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa947e00640 1 -- 172.21.15.196:0/278899346 >> v2:172.21.15.203:3300/0 conn(0x7fa9480d1a50 msgr2=0x7fa948167e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:25.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa947e00640 1 --2- 172.21.15.196:0/278899346 >> v2:172.21.15.203:3300/0 conn(0x7fa9480d1a50 0x7fa948167e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa947e00640 1 -- 172.21.15.196:0/278899346 >> v1:172.21.15.203:6789/0 conn(0x7fa9480d38f0 legacy=0x7fa9480d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:25.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa947e00640 1 -- 172.21.15.196:0/278899346 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9480d0500 con 0x7fa94815d210 2024-04-28T13:31:25.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa947e00640 1 -- 172.21.15.196:0/278899346 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fa938001670 con 0x7fa94815d210 2024-04-28T13:31:25.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa94e400640 1 --2- 172.21.15.196:0/278899346 >> v2:172.21.15.203:3300/0 conn(0x7fa9480d1a50 0x7fa948167e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:31:25.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa947e00640 1 -- 172.21.15.196:0/278899346 >> v1:172.21.15.196:6789/0 conn(0x7fa94815d210 legacy=0x7fa9480d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:25.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa947e00640 1 --2- 172.21.15.196:0/278899346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92c003a00 0x7fa92c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:25.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa947e00640 1 --2- 172.21.15.196:0/278899346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa92c004800 0x7fa92c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:25.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa947e00640 1 --2- 172.21.15.196:0/278899346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa92c005210 0x7fa92c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:25.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa947e00640 1 -- 172.21.15.196:0/278899346 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa948070440 con 0x7fa92c003a00 2024-04-28T13:31:25.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa947e00640 1 -- 172.21.15.196:0/278899346 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c00c160 con 0x7fa92c004800 2024-04-28T13:31:25.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa947e00640 1 -- 172.21.15.196:0/278899346 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa92c00c360 con 0x7fa92c005210 2024-04-28T13:31:25.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.808+0000 7fa94da00640 1 --2- 172.21.15.196:0/278899346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92c003a00 0x7fa92c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:25.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.809+0000 7fa94d000640 1 --2- 172.21.15.196:0/278899346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa92c004800 0x7fa92c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:25.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.809+0000 7fa94da00640 1 -- 172.21.15.196:0/278899346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa92c005210 msgr2=0x7fa92c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:25.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.809+0000 7fa94da00640 1 --2- 172.21.15.196:0/278899346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa92c005210 0x7fa92c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.809+0000 7fa94da00640 1 -- 172.21.15.196:0/278899346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa92c004800 msgr2=0x7fa92c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:25.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.809+0000 7fa94da00640 1 --2- 172.21.15.196:0/278899346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa92c004800 0x7fa92c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.809+0000 7fa94da00640 1 -- 172.21.15.196:0/278899346 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92c003520 con 0x7fa92c003a00 2024-04-28T13:31:25.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.809+0000 7fa94d000640 1 --2- 172.21.15.196:0/278899346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa92c004800 0x7fa92c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:25.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.809+0000 7fa94e400640 1 -- 172.21.15.196:0/278899346 reap_dead start 2024-04-28T13:31:25.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.810+0000 7fa94da00640 1 --2- 172.21.15.196:0/278899346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92c003a00 0x7fa92c003e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa93800c190 tx=0x7fa92c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4691565b4534d649 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:25.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.810+0000 7fa947e00640 1 -- 172.21.15.196:0/278899346 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938007c60 con 0x7fa92c003a00 2024-04-28T13:31:25.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.810+0000 7fa947e00640 1 -- 172.21.15.196:0/278899346 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa938005ad0 con 0x7fa92c003a00 2024-04-28T13:31:25.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.810+0000 7fa947e00640 1 -- 172.21.15.196:0/278899346 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938007070 con 0x7fa92c003a00 2024-04-28T13:31:25.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.810+0000 7fa94ee00640 1 -- 172.21.15.196:0/278899346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92c003a00 msgr2=0x7fa92c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:25.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.810+0000 7fa94ee00640 1 --2- 172.21.15.196:0/278899346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92c003a00 0x7fa92c003e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa93800c190 tx=0x7fa92c005830 comp rx=0 tx=0).stop 2024-04-28T13:31:25.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.810+0000 7fa94ee00640 1 -- 172.21.15.196:0/278899346 shutdown_connections 2024-04-28T13:31:25.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.810+0000 7fa94ee00640 1 --2- 172.21.15.196:0/278899346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa92c003a00 0x7fa92c003e20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.810+0000 7fa94ee00640 1 -- 172.21.15.196:0/278899346 >> 172.21.15.196:0/278899346 conn(0x7fa9480d2d60 msgr2=0x7fa9480d3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:25.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.811+0000 7fa94ee00640 1 -- 172.21.15.196:0/278899346 shutdown_connections 2024-04-28T13:31:25.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.811+0000 7fa94ee00640 1 -- 172.21.15.196:0/278899346 wait complete. 2024-04-28T13:31:25.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.812+0000 7fa94ee00640 1 Processor -- start 2024-04-28T13:31:25.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.812+0000 7fa94ee00640 1 -- start start 2024-04-28T13:31:25.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.813+0000 7fa94ee00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa92c003a00 0x7fa9480b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:25.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.813+0000 7fa94ee00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9481f55a0 0x7fa94806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:25.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.813+0000 7fa94ee00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa948153a00 0x7fa948153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:25.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.813+0000 7fa94ee00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9480d14c0 con 0x7fa948153a00 2024-04-28T13:31:25.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.813+0000 7fa94ee00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa9480d1340 con 0x7fa92c003a00 2024-04-28T13:31:25.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.813+0000 7fa94ee00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa9480d1640 con 0x7fa9481f55a0 2024-04-28T13:31:25.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa94e400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa948153a00 0x7fa948153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:25.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa94e400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa948153a00 0x7fa948153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35524/0 (socket says 172.21.15.196:35524) 2024-04-28T13:31:25.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa94e400640 1 -- 172.21.15.196:0/3469988997 learned_addr learned my addr 172.21.15.196:0/3469988997 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa94da00640 1 --2- 172.21.15.196:0/3469988997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa92c003a00 0x7fa9480b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa94d000640 1 --2- 172.21.15.196:0/3469988997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9481f55a0 0x7fa94806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa94e400640 1 -- 172.21.15.196:0/3469988997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9481f55a0 msgr2=0x7fa94806ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa94e400640 1 --2- 172.21.15.196:0/3469988997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9481f55a0 0x7fa94806ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa94e400640 1 -- 172.21.15.196:0/3469988997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa92c003a00 msgr2=0x7fa9480b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa94e400640 1 --2- 172.21.15.196:0/3469988997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa92c003a00 0x7fa9480b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa94e400640 1 -- 172.21.15.196:0/3469988997 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9480b84e0 con 0x7fa948153a00 2024-04-28T13:31:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa94da00640 1 --2- 172.21.15.196:0/3469988997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa92c003a00 0x7fa9480b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa94e400640 1 --2- 172.21.15.196:0/3469988997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa948153a00 0x7fa948153eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa94806a0b0 tx=0x7fa94000e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa945600640 1 -- 172.21.15.196:0/3469988997 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94001a070 con 0x7fa948153a00 2024-04-28T13:31:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa945600640 1 -- 172.21.15.196:0/3469988997 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa940004230 con 0x7fa948153a00 2024-04-28T13:31:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa945600640 1 -- 172.21.15.196:0/3469988997 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94000f040 con 0x7fa948153a00 2024-04-28T13:31:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.814+0000 7fa94ee00640 1 -- 172.21.15.196:0/3469988997 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9480b6f00 con 0x7fa948153a00 2024-04-28T13:31:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.815+0000 7fa94ee00640 1 -- 172.21.15.196:0/3469988997 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9480b74c0 con 0x7fa948153a00 2024-04-28T13:31:25.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.816+0000 7fa94ee00640 1 -- 172.21.15.196:0/3469988997 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9480693d0 con 0x7fa948153a00 2024-04-28T13:31:25.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.817+0000 7fa945600640 1 -- 172.21.15.196:0/3469988997 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa94001d070 con 0x7fa948153a00 2024-04-28T13:31:25.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.818+0000 7fa945600640 1 --2- 172.21.15.196:0/3469988997 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa92806c8f0 0x7fa92806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:25.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.818+0000 7fa945600640 1 -- 172.21.15.196:0/3469988997 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa94008a860 con 0x7fa948153a00 2024-04-28T13:31:25.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.818+0000 7fa94da00640 1 --2- 172.21.15.196:0/3469988997 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa92806c8f0 0x7fa92806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:25.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.820+0000 7fa94da00640 1 --2- 172.21.15.196:0/3469988997 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa92806c8f0 0x7fa92806edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa93800c160 tx=0x7fa938009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:25.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:25.822+0000 7fa945600640 1 -- 172.21.15.196:0/3469988997 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa940051c00 con 0x7fa948153a00 2024-04-28T13:31:26.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.016+0000 7fa94ee00640 1 -- 172.21.15.196:0/3469988997 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9480b0a50 con 0x7fa92806c8f0 2024-04-28T13:31:26.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.038+0000 7fa945600640 1 -- 172.21.15.196:0/3469988997 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240067 (secure 0 0 0) 0x7fa9480b0a50 con 0x7fa92806c8f0 2024-04-28T13:31:26.041 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:26.042 INFO:teuthology.orchestra.run.smithi196.stderr:dumped all 2024-04-28T13:31:26.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.043+0000 7fa94ee00640 1 -- 172.21.15.196:0/3469988997 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa92806c8f0 msgr2=0x7fa92806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:26.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.043+0000 7fa94ee00640 1 --2- 172.21.15.196:0/3469988997 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa92806c8f0 0x7fa92806edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa93800c160 tx=0x7fa938009030 comp rx=0 tx=0).stop 2024-04-28T13:31:26.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.043+0000 7fa94ee00640 1 -- 172.21.15.196:0/3469988997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa948153a00 msgr2=0x7fa948153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:26.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.043+0000 7fa94ee00640 1 --2- 172.21.15.196:0/3469988997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa948153a00 0x7fa948153eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa94806a0b0 tx=0x7fa94000e830 comp rx=0 tx=0).stop 2024-04-28T13:31:26.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.043+0000 7fa94ee00640 1 -- 172.21.15.196:0/3469988997 shutdown_connections 2024-04-28T13:31:26.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.043+0000 7fa94ee00640 1 --2- 172.21.15.196:0/3469988997 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa92806c8f0 0x7fa92806edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.043+0000 7fa94ee00640 1 --2- 172.21.15.196:0/3469988997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa948153a00 0x7fa948153eb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.043+0000 7fa94ee00640 1 --2- 172.21.15.196:0/3469988997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9481f55a0 0x7fa94806ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.043+0000 7fa94ee00640 1 --2- 172.21.15.196:0/3469988997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa92c003a00 0x7fa9480b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.043+0000 7fa94ee00640 1 -- 172.21.15.196:0/3469988997 >> 172.21.15.196:0/3469988997 conn(0x7fa9480d2d60 msgr2=0x7fa9480b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:26.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.043+0000 7fa94ee00640 1 -- 172.21.15.196:0/3469988997 shutdown_connections 2024-04-28T13:31:26.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.043+0000 7fa94ee00640 1 -- 172.21.15.196:0/3469988997 wait complete. 2024-04-28T13:31:26.063 INFO:teuthology.orchestra.run.smithi196.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-28T13:31:24.180806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218612,"kb_used_data":2448,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756108,"statfs":{"total":773094113280,"available":772870254592,"internally_reserved":0,"allocated":2506752,"data_stored":1581774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.599187"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.586645+0000","last_change":"2024-04-28T13:31:19.586645+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.586645+0000","last_undegraded":"2024-04-28T13:31:19.586645+0000","last_fullsized":"2024-04-28T13:31:19.586645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.595560+0000","last_change":"2024-04-28T13:31:19.595560+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.595560+0000","last_undegraded":"2024-04-28T13:31:19.595560+0000","last_fullsized":"2024-04-28T13:31:19.595560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.587008+0000","last_change":"2024-04-28T13:31:19.587008+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.587008+0000","last_undegraded":"2024-04-28T13:31:19.587008+0000","last_fullsized":"2024-04-28T13:31:19.587008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.600808+0000","last_change":"2024-04-28T13:31:19.600808+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.600808+0000","last_undegraded":"2024-04-28T13:31:19.600808+0000","last_fullsized":"2024-04-28T13:31:19.600808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.597046+0000","last_change":"2024-04-28T13:31:19.597046+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.597046+0000","last_undegraded":"2024-04-28T13:31:19.597046+0000","last_fullsized":"2024-04-28T13:31:19.597046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.602455+0000","last_change":"2024-04-28T13:31:19.602455+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.602455+0000","last_undegraded":"2024-04-28T13:31:19.602455+0000","last_fullsized":"2024-04-28T13:31:19.602455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.595245+0000","last_change":"2024-04-28T13:31:19.595245+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.595245+0000","last_undegraded":"2024-04-28T13:31:19.595245+0000","last_fullsized":"2024-04-28T13:31:19.595245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.595753+0000","last_change":"2024-04-28T13:31:19.595753+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.595753+0000","last_undegraded":"2024-04-28T13:31:19.595753+0000","last_fullsized":"2024-04-28T13:31:19.595753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.591548+0000","last_change":"2024-04-28T13:31:19.591548+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.591548+0000","last_undegraded":"2024-04-28T13:31:19.591548+0000","last_fullsized":"2024-04-28T13:31:19.591548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.601749+0000","last_change":"2024-04-28T13:31:19.601749+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.601749+0000","last_undegraded":"2024-04-28T13:31:19.601749+0000","last_fullsized":"2024-04-28T13:31:19.601749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.594943+0000","last_change":"2024-04-28T13:31:19.594943+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.594943+0000","last_undegraded":"2024-04-28T13:31:19.594943+0000","last_fullsized":"2024-04-28T13:31:19.594943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.607066+0000","last_change":"2024-04-28T13:31:19.607066+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.607066+0000","last_undegraded":"2024-04-28T13:31:19.607066+0000","last_fullsized":"2024-04-28T13:31:19.607066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.588783+0000","last_change":"2024-04-28T13:31:19.588783+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.588783+0000","last_undegraded":"2024-04-28T13:31:19.588783+0000","last_fullsized":"2024-04-28T13:31:19.588783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.606732+0000","last_change":"2024-04-28T13:31:19.606732+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.606732+0000","last_undegraded":"2024-04-28T13:31:19.606732+0000","last_fullsized":"2024-04-28T13:31:19.606732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.594567+0000","last_change":"2024-04-28T13:31:19.594567+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.594567+0000","last_undegraded":"2024-04-28T13:31:19.594567+0000","last_fullsized":"2024-04-28T13:31:19.594567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.607940+0000","last_change":"2024-04-28T13:31:19.607940+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.607940+0000","last_undegraded":"2024-04-28T13:31:19.607940+0000","last_fullsized":"2024-04-28T13:31:19.607940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-28T13:31:19.594871+0000","last_change":"2024-04-28T13:31:11.831538+0000","last_active":"2024-04-28T13:31:19.594871+0000","last_peered":"2024-04-28T13:31:19.594871+0000","last_clean":"2024-04-28T13:31:19.594871+0000","last_became_active":"2024-04-28T13:31:11.828959+0000","last_became_peered":"2024-04-28T13:31:11.828959+0000","last_unstale":"2024-04-28T13:31:19.594871+0000","last_undegraded":"2024-04-28T13:31:19.594871+0000","last_fullsized":"2024-04-28T13:31:19.594871+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_clean_scrub_stamp":"2024-04-28T13:31:10.444388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:25:27.078911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.601135+0000","last_change":"2024-04-28T13:31:19.601135+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.601135+0000","last_undegraded":"2024-04-28T13:31:19.601135+0000","last_fullsized":"2024-04-28T13:31:19.601135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.588693+0000","last_change":"2024-04-28T13:31:19.588693+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.588693+0000","last_undegraded":"2024-04-28T13:31:19.588693+0000","last_fullsized":"2024-04-28T13:31:19.588693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.607039+0000","last_change":"2024-04-28T13:31:19.607039+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.607039+0000","last_undegraded":"2024-04-28T13:31:19.607039+0000","last_fullsized":"2024-04-28T13:31:19.607039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.600778+0000","last_change":"2024-04-28T13:31:19.600778+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.600778+0000","last_undegraded":"2024-04-28T13:31:19.600778+0000","last_fullsized":"2024-04-28T13:31:19.600778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.595271+0000","last_change":"2024-04-28T13:31:19.595271+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.595271+0000","last_undegraded":"2024-04-28T13:31:19.595271+0000","last_fullsized":"2024-04-28T13:31:19.595271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.591366+0000","last_change":"2024-04-28T13:31:19.591366+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.591366+0000","last_undegraded":"2024-04-28T13:31:19.591366+0000","last_fullsized":"2024-04-28T13:31:19.591366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.595981+0000","last_change":"2024-04-28T13:31:19.595981+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.595981+0000","last_undegraded":"2024-04-28T13:31:19.595981+0000","last_fullsized":"2024-04-28T13:31:19.595981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.595276+0000","last_change":"2024-04-28T13:31:19.595276+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.595276+0000","last_undegraded":"2024-04-28T13:31:19.595276+0000","last_fullsized":"2024-04-28T13:31:19.595276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.591483+0000","last_change":"2024-04-28T13:31:19.591483+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.591483+0000","last_undegraded":"2024-04-28T13:31:19.591483+0000","last_fullsized":"2024-04-28T13:31:19.591483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.594011+0000","last_change":"2024-04-28T13:31:19.594011+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.594011+0000","last_undegraded":"2024-04-28T13:31:19.594011+0000","last_fullsized":"2024-04-28T13:31:19.594011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.594637+0000","last_change":"2024-04-28T13:31:19.594637+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.594637+0000","last_undegraded":"2024-04-28T13:31:19.594637+0000","last_fullsized":"2024-04-28T13:31:19.594637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.589192+0000","last_change":"2024-04-28T13:31:19.589192+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.589192+0000","last_undegraded":"2024-04-28T13:31:19.589192+0000","last_fullsized":"2024-04-28T13:31:19.589192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.588893+0000","last_change":"2024-04-28T13:31:19.588893+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.588893+0000","last_undegraded":"2024-04-28T13:31:19.588893+0000","last_fullsized":"2024-04-28T13:31:19.588893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.592875+0000","last_change":"2024-04-28T13:31:19.592875+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.592875+0000","last_undegraded":"2024-04-28T13:31:19.592875+0000","last_fullsized":"2024-04-28T13:31:19.592875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.586968+0000","last_change":"2024-04-28T13:31:19.586968+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.586968+0000","last_undegraded":"2024-04-28T13:31:19.586968+0000","last_fullsized":"2024-04-28T13:31:19.586968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T13:31:20.578890+0000","last_change":"2024-04-28T13:31:20.578890+0000","last_active":"2024-04-28T13:31:20.578890+0000","last_peered":"2024-04-28T13:31:20.578890+0000","last_clean":"2024-04-28T13:31:20.578890+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:20.578890+0000","last_undegraded":"2024-04-28T13:31:20.578890+0000","last_fullsized":"2024-04-28T13:31:20.578890+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-28T13:31:19.591442+0000","last_change":"2024-04-28T13:31:19.591442+0000","last_active":"2024-04-28T13:31:19.564592+0000","last_peered":"2024-04-28T13:31:19.564592+0000","last_clean":"2024-04-28T13:31:19.564592+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T13:31:19.591442+0000","last_undegraded":"2024-04-28T13:31:19.591442+0000","last_fullsized":"2024-04-28T13:31:19.591442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T13:31:26.067 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T13:31:26.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.307+0000 7f5137200640 1 Processor -- start 2024-04-28T13:31:26.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.307+0000 7f5137200640 1 -- start start 2024-04-28T13:31:26.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.309+0000 7f5137200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5138163bf0 0x7f5138163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:26.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.309+0000 7f5137200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5138163570 0x7f51380b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:26.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.309+0000 7f5137200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5138162da0 con 0x7f51381629c0 2024-04-28T13:31:26.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.309+0000 7f5137200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f51380d3e20 con 0x7f5138163570 2024-04-28T13:31:26.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.309+0000 7f5137200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f51380b6190 con 0x7f5138163bf0 2024-04-28T13:31:26.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.309+0000 7f5135e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f51381629c0 0x7f51380d3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:55988/0 (socket says 172.21.15.196:55988) 2024-04-28T13:31:26.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.309+0000 7f5135e00640 1 -- 172.21.15.196:0/2063636283 learned_addr learned my addr 172.21.15.196:0/2063636283 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:26.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.309+0000 7f5136800640 1 --2- 172.21.15.196:0/2063636283 >> v2:172.21.15.196:3300/0 conn(0x7f5138163570 0x7f51380b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:26.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.309+0000 7f5135400640 1 -- 172.21.15.196:0/2063636283 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 855336152 0 0) 0x7f5138162da0 con 0x7f51381629c0 2024-04-28T13:31:26.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.309+0000 7f5135400640 1 -- 172.21.15.196:0/2063636283 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f511c003520 con 0x7f51381629c0 2024-04-28T13:31:26.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.309+0000 7f5136800640 1 -- 172.21.15.196:0/2063636283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5138163bf0 msgr2=0x7f5138163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:26.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.309+0000 7f5136800640 1 --2- 172.21.15.196:0/2063636283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5138163bf0 0x7f5138163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.309+0000 7f5136800640 1 -- 172.21.15.196:0/2063636283 >> v1:172.21.15.196:6789/0 conn(0x7f51381629c0 legacy=0x7f51380d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:26.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.309+0000 7f5136800640 1 -- 172.21.15.196:0/2063636283 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51380b6470 con 0x7f5138163570 2024-04-28T13:31:26.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.310+0000 7f5136800640 1 --2- 172.21.15.196:0/2063636283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138163570 0x7f51380b2380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5130009d90 tx=0x7f513002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5883cd15080cebd server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:26.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.311+0000 7f5135400640 1 -- 172.21.15.196:0/2063636283 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130007740 con 0x7f5138163570 2024-04-28T13:31:26.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.311+0000 7f5135400640 1 -- 172.21.15.196:0/2063636283 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51300078e0 con 0x7f5138163570 2024-04-28T13:31:26.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.311+0000 7f5135400640 1 -- 172.21.15.196:0/2063636283 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130007c90 con 0x7f5138163570 2024-04-28T13:31:26.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.311+0000 7f5137200640 1 -- 172.21.15.196:0/2063636283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138163570 msgr2=0x7f51380b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:26.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.311+0000 7f5137200640 1 --2- 172.21.15.196:0/2063636283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138163570 0x7f51380b2380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5130009d90 tx=0x7f513002a660 comp rx=0 tx=0).stop 2024-04-28T13:31:26.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.311+0000 7f5137200640 1 -- 172.21.15.196:0/2063636283 shutdown_connections 2024-04-28T13:31:26.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.311+0000 7f5137200640 1 --2- 172.21.15.196:0/2063636283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138163570 0x7f51380b2380 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.311+0000 7f5137200640 1 --2- 172.21.15.196:0/2063636283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5138163bf0 0x7f5138163030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.311+0000 7f5137200640 1 -- 172.21.15.196:0/2063636283 >> 172.21.15.196:0/2063636283 conn(0x7f513809e570 msgr2=0x7f51380d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:26.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.311+0000 7f5137200640 1 -- 172.21.15.196:0/2063636283 shutdown_connections 2024-04-28T13:31:26.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.311+0000 7f5137200640 1 -- 172.21.15.196:0/2063636283 wait complete. 2024-04-28T13:31:26.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.313+0000 7f5137200640 1 Processor -- start 2024-04-28T13:31:26.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.313+0000 7f5137200640 1 -- start start 2024-04-28T13:31:26.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.314+0000 7f5137200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51381629c0 0x7f5138158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:26.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.314+0000 7f5137200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5138163570 0x7f51380b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:26.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.314+0000 7f5137200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138163bf0 0x7f51380b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:26.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.314+0000 7f5137200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51380b1e60 con 0x7f5138163bf0 2024-04-28T13:31:26.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.314+0000 7f5137200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f51380b6a30 con 0x7f5138163570 2024-04-28T13:31:26.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.314+0000 7f5137200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f51380b28c0 con 0x7f51381629c0 2024-04-28T13:31:26.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.314+0000 7f5136800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138163bf0 0x7f51380b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:26.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.314+0000 7f5136800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138163bf0 0x7f51380b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35540/0 (socket says 172.21.15.196:35540) 2024-04-28T13:31:26.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.314+0000 7f5136800640 1 -- 172.21.15.196:0/688964305 learned_addr learned my addr 172.21.15.196:0/688964305 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:26.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.315+0000 7f5135e00640 1 --2- 172.21.15.196:0/688964305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51381629c0 0x7f5138158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:26.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.315+0000 7f512d400640 1 --2- 172.21.15.196:0/688964305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5138163570 0x7f51380b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:26.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.315+0000 7f5136800640 1 -- 172.21.15.196:0/688964305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51381629c0 msgr2=0x7f5138158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:26.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.315+0000 7f5136800640 1 --2- 172.21.15.196:0/688964305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51381629c0 0x7f5138158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.315+0000 7f5136800640 1 -- 172.21.15.196:0/688964305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5138163570 msgr2=0x7f51380b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:26.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.315+0000 7f5136800640 1 --2- 172.21.15.196:0/688964305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5138163570 0x7f51380b2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.315+0000 7f5136800640 1 -- 172.21.15.196:0/688964305 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51380b9040 con 0x7f5138163bf0 2024-04-28T13:31:26.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.315+0000 7f5136800640 1 --2- 172.21.15.196:0/688964305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138163bf0 0x7f51380b4d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5130009eb0 tx=0x7f5130004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:26.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.315+0000 7f512ea00640 1 -- 172.21.15.196:0/688964305 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130040070 con 0x7f5138163bf0 2024-04-28T13:31:26.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.315+0000 7f512ea00640 1 -- 172.21.15.196:0/688964305 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5130031bc0 con 0x7f5138163bf0 2024-04-28T13:31:26.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.315+0000 7f512ea00640 1 -- 172.21.15.196:0/688964305 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130009450 con 0x7f5138163bf0 2024-04-28T13:31:26.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.315+0000 7f5137200640 1 -- 172.21.15.196:0/688964305 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51380b92e0 con 0x7f5138163bf0 2024-04-28T13:31:26.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.316+0000 7f5137200640 1 -- 172.21.15.196:0/688964305 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51380b98a0 con 0x7f5138163bf0 2024-04-28T13:31:26.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.317+0000 7f5137200640 1 -- 172.21.15.196:0/688964305 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f513806a290 con 0x7f5138163bf0 2024-04-28T13:31:26.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.317+0000 7f512ea00640 1 -- 172.21.15.196:0/688964305 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5130039040 con 0x7f5138163bf0 2024-04-28T13:31:26.318 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.319+0000 7f512ea00640 1 --2- 172.21.15.196:0/688964305 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f511406c8f0 0x7f511406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:26.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.319+0000 7f512ea00640 1 -- 172.21.15.196:0/688964305 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f51300ad0e0 con 0x7f5138163bf0 2024-04-28T13:31:26.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.319+0000 7f5135e00640 1 --2- 172.21.15.196:0/688964305 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f511406c8f0 0x7f511406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:26.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.320+0000 7f5135e00640 1 --2- 172.21.15.196:0/688964305 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f511406c8f0 0x7f511406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5128005880 tx=0x7f5128009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:26.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.323+0000 7f512ea00640 1 -- 172.21.15.196:0/688964305 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f513007b470 con 0x7f5138163bf0 2024-04-28T13:31:26.525 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.525+0000 7f5137200640 1 -- 172.21.15.196:0/688964305 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f513809ea50 con 0x7f511406c8f0 2024-04-28T13:31:26.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.546+0000 7f512ea00640 1 -- 172.21.15.196:0/688964305 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f513809ea50 con 0x7f511406c8f0 2024-04-28T13:31:26.550 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:26.551 INFO:teuthology.orchestra.run.smithi196.stderr:dumped all 2024-04-28T13:31:26.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.552+0000 7f5137200640 1 -- 172.21.15.196:0/688964305 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f511406c8f0 msgr2=0x7f511406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:26.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.552+0000 7f5137200640 1 --2- 172.21.15.196:0/688964305 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f511406c8f0 0x7f511406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5128005880 tx=0x7f5128009040 comp rx=0 tx=0).stop 2024-04-28T13:31:26.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.552+0000 7f5137200640 1 -- 172.21.15.196:0/688964305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138163bf0 msgr2=0x7f51380b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:26.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.552+0000 7f5137200640 1 --2- 172.21.15.196:0/688964305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138163bf0 0x7f51380b4d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5130009eb0 tx=0x7f5130004a40 comp rx=0 tx=0).stop 2024-04-28T13:31:26.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.552+0000 7f5137200640 1 -- 172.21.15.196:0/688964305 shutdown_connections 2024-04-28T13:31:26.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.552+0000 7f5137200640 1 --2- 172.21.15.196:0/688964305 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f511406c8f0 0x7f511406edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.552+0000 7f5137200640 1 --2- 172.21.15.196:0/688964305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138163bf0 0x7f51380b4d50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.552+0000 7f5137200640 1 --2- 172.21.15.196:0/688964305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5138163570 0x7f51380b2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.552+0000 7f5137200640 1 --2- 172.21.15.196:0/688964305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51381629c0 0x7f5138158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.552+0000 7f5137200640 1 -- 172.21.15.196:0/688964305 >> 172.21.15.196:0/688964305 conn(0x7f513809e570 msgr2=0x7f513815f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:26.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.552+0000 7f5137200640 1 -- 172.21.15.196:0/688964305 shutdown_connections 2024-04-28T13:31:26.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.552+0000 7f5137200640 1 -- 172.21.15.196:0/688964305 wait complete. 2024-04-28T13:31:26.570 INFO:teuthology.orchestra.run.smithi196.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-28T13:31:26.181613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218980,"kb_used_data":2812,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755740,"statfs":{"total":773094113280,"available":772869877760,"internally_reserved":0,"allocated":2879488,"data_stored":1746728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002070"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.232418+0000","last_change":"2024-04-28T13:31:21.653529+0000","last_active":"2024-04-28T13:31:24.232418+0000","last_peered":"2024-04-28T13:31:24.232418+0000","last_clean":"2024-04-28T13:31:24.232418+0000","last_became_active":"2024-04-28T13:31:21.653192+0000","last_became_peered":"2024-04-28T13:31:21.653192+0000","last_unstale":"2024-04-28T13:31:24.232418+0000","last_undegraded":"2024-04-28T13:31:24.232418+0000","last_fullsized":"2024-04-28T13:31:24.232418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:30:36.318507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640382+0000","last_change":"2024-04-28T13:31:21.636383+0000","last_active":"2024-04-28T13:31:22.640382+0000","last_peered":"2024-04-28T13:31:22.640382+0000","last_clean":"2024-04-28T13:31:22.640382+0000","last_became_active":"2024-04-28T13:31:21.635970+0000","last_became_peered":"2024-04-28T13:31:21.635970+0000","last_unstale":"2024-04-28T13:31:22.640382+0000","last_undegraded":"2024-04-28T13:31:22.640382+0000","last_fullsized":"2024-04-28T13:31:22.640382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:33.431077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.729998+0000","last_change":"2024-04-28T13:31:21.642396+0000","last_active":"2024-04-28T13:31:24.729998+0000","last_peered":"2024-04-28T13:31:24.729998+0000","last_clean":"2024-04-28T13:31:24.729998+0000","last_became_active":"2024-04-28T13:31:21.642062+0000","last_became_peered":"2024-04-28T13:31:21.642062+0000","last_unstale":"2024-04-28T13:31:24.729998+0000","last_undegraded":"2024-04-28T13:31:24.729998+0000","last_fullsized":"2024-04-28T13:31:24.729998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:43:30.133775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646626+0000","last_change":"2024-04-28T13:31:21.637265+0000","last_active":"2024-04-28T13:31:22.646626+0000","last_peered":"2024-04-28T13:31:22.646626+0000","last_clean":"2024-04-28T13:31:22.646626+0000","last_became_active":"2024-04-28T13:31:21.636679+0000","last_became_peered":"2024-04-28T13:31:21.636679+0000","last_unstale":"2024-04-28T13:31:22.646626+0000","last_undegraded":"2024-04-28T13:31:22.646626+0000","last_fullsized":"2024-04-28T13:31:22.646626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:25.059779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231103+0000","last_change":"2024-04-28T13:31:21.660088+0000","last_active":"2024-04-28T13:31:24.231103+0000","last_peered":"2024-04-28T13:31:24.231103+0000","last_clean":"2024-04-28T13:31:24.231103+0000","last_became_active":"2024-04-28T13:31:21.652264+0000","last_became_peered":"2024-04-28T13:31:21.652264+0000","last_unstale":"2024-04-28T13:31:24.231103+0000","last_undegraded":"2024-04-28T13:31:24.231103+0000","last_fullsized":"2024-04-28T13:31:24.231103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:28:41.694303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646580+0000","last_change":"2024-04-28T13:31:21.641292+0000","last_active":"2024-04-28T13:31:22.646580+0000","last_peered":"2024-04-28T13:31:22.646580+0000","last_clean":"2024-04-28T13:31:22.646580+0000","last_became_active":"2024-04-28T13:31:21.635397+0000","last_became_peered":"2024-04-28T13:31:21.635397+0000","last_unstale":"2024-04-28T13:31:22.646580+0000","last_undegraded":"2024-04-28T13:31:22.646580+0000","last_fullsized":"2024-04-28T13:31:22.646580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:16:56.574292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231153+0000","last_change":"2024-04-28T13:31:21.654211+0000","last_active":"2024-04-28T13:31:24.231153+0000","last_peered":"2024-04-28T13:31:24.231153+0000","last_clean":"2024-04-28T13:31:24.231153+0000","last_became_active":"2024-04-28T13:31:21.653516+0000","last_became_peered":"2024-04-28T13:31:21.653516+0000","last_unstale":"2024-04-28T13:31:24.231153+0000","last_undegraded":"2024-04-28T13:31:24.231153+0000","last_fullsized":"2024-04-28T13:31:24.231153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:26:26.344814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.729792+0000","last_change":"2024-04-28T13:31:21.641706+0000","last_active":"2024-04-28T13:31:24.729792+0000","last_peered":"2024-04-28T13:31:24.729792+0000","last_clean":"2024-04-28T13:31:24.729792+0000","last_became_active":"2024-04-28T13:31:21.639207+0000","last_became_peered":"2024-04-28T13:31:21.639207+0000","last_unstale":"2024-04-28T13:31:24.729792+0000","last_undegraded":"2024-04-28T13:31:24.729792+0000","last_fullsized":"2024-04-28T13:31:24.729792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:33:40.307112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.232568+0000","last_change":"2024-04-28T13:31:21.657597+0000","last_active":"2024-04-28T13:31:24.232568+0000","last_peered":"2024-04-28T13:31:24.232568+0000","last_clean":"2024-04-28T13:31:24.232568+0000","last_became_active":"2024-04-28T13:31:21.656871+0000","last_became_peered":"2024-04-28T13:31:21.656871+0000","last_unstale":"2024-04-28T13:31:24.232568+0000","last_undegraded":"2024-04-28T13:31:24.232568+0000","last_fullsized":"2024-04-28T13:31:24.232568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:23:11.970214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732465+0000","last_change":"2024-04-28T13:31:21.648874+0000","last_active":"2024-04-28T13:31:24.732465+0000","last_peered":"2024-04-28T13:31:24.732465+0000","last_clean":"2024-04-28T13:31:24.732465+0000","last_became_active":"2024-04-28T13:31:21.646033+0000","last_became_peered":"2024-04-28T13:31:21.646033+0000","last_unstale":"2024-04-28T13:31:24.732465+0000","last_undegraded":"2024-04-28T13:31:24.732465+0000","last_fullsized":"2024-04-28T13:31:24.732465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:33.431077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732151+0000","last_change":"2024-04-28T13:31:21.649391+0000","last_active":"2024-04-28T13:31:24.732151+0000","last_peered":"2024-04-28T13:31:24.732151+0000","last_clean":"2024-04-28T13:31:24.732151+0000","last_became_active":"2024-04-28T13:31:21.649078+0000","last_became_peered":"2024-04-28T13:31:21.649078+0000","last_unstale":"2024-04-28T13:31:24.732151+0000","last_undegraded":"2024-04-28T13:31:24.732151+0000","last_fullsized":"2024-04-28T13:31:24.732151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:32.352858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647834+0000","last_change":"2024-04-28T13:31:21.651903+0000","last_active":"2024-04-28T13:31:22.647834+0000","last_peered":"2024-04-28T13:31:22.647834+0000","last_clean":"2024-04-28T13:31:22.647834+0000","last_became_active":"2024-04-28T13:31:21.650905+0000","last_became_peered":"2024-04-28T13:31:21.650905+0000","last_unstale":"2024-04-28T13:31:22.647834+0000","last_undegraded":"2024-04-28T13:31:22.647834+0000","last_fullsized":"2024-04-28T13:31:22.647834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:01.137759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731025+0000","last_change":"2024-04-28T13:31:21.640551+0000","last_active":"2024-04-28T13:31:24.731025+0000","last_peered":"2024-04-28T13:31:24.731025+0000","last_clean":"2024-04-28T13:31:24.731025+0000","last_became_active":"2024-04-28T13:31:21.640275+0000","last_became_peered":"2024-04-28T13:31:21.640275+0000","last_unstale":"2024-04-28T13:31:24.731025+0000","last_undegraded":"2024-04-28T13:31:24.731025+0000","last_fullsized":"2024-04-28T13:31:24.731025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:43:16.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730356+0000","last_change":"2024-04-28T13:31:21.639830+0000","last_active":"2024-04-28T13:31:24.730356+0000","last_peered":"2024-04-28T13:31:24.730356+0000","last_clean":"2024-04-28T13:31:24.730356+0000","last_became_active":"2024-04-28T13:31:21.639502+0000","last_became_peered":"2024-04-28T13:31:21.639502+0000","last_unstale":"2024-04-28T13:31:24.730356+0000","last_undegraded":"2024-04-28T13:31:24.730356+0000","last_fullsized":"2024-04-28T13:31:24.730356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:30:04.305801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640325+0000","last_change":"2024-04-28T13:31:21.635217+0000","last_active":"2024-04-28T13:31:22.640325+0000","last_peered":"2024-04-28T13:31:22.640325+0000","last_clean":"2024-04-28T13:31:22.640325+0000","last_became_active":"2024-04-28T13:31:21.634165+0000","last_became_peered":"2024-04-28T13:31:21.634165+0000","last_unstale":"2024-04-28T13:31:22.640325+0000","last_undegraded":"2024-04-28T13:31:22.640325+0000","last_fullsized":"2024-04-28T13:31:22.640325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:32.352858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646029+0000","last_change":"2024-04-28T13:31:21.642988+0000","last_active":"2024-04-28T13:31:22.646029+0000","last_peered":"2024-04-28T13:31:22.646029+0000","last_clean":"2024-04-28T13:31:22.646029+0000","last_became_active":"2024-04-28T13:31:21.635586+0000","last_became_peered":"2024-04-28T13:31:21.635586+0000","last_unstale":"2024-04-28T13:31:22.646029+0000","last_undegraded":"2024-04-28T13:31:22.646029+0000","last_fullsized":"2024-04-28T13:31:22.646029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:33.431077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647955+0000","last_change":"2024-04-28T13:31:21.653816+0000","last_active":"2024-04-28T13:31:22.647955+0000","last_peered":"2024-04-28T13:31:22.647955+0000","last_clean":"2024-04-28T13:31:22.647955+0000","last_became_active":"2024-04-28T13:31:21.653606+0000","last_became_peered":"2024-04-28T13:31:21.653606+0000","last_unstale":"2024-04-28T13:31:22.647955+0000","last_undegraded":"2024-04-28T13:31:22.647955+0000","last_fullsized":"2024-04-28T13:31:22.647955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:53:05.667773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731713+0000","last_change":"2024-04-28T13:31:21.643582+0000","last_active":"2024-04-28T13:31:24.731713+0000","last_peered":"2024-04-28T13:31:24.731713+0000","last_clean":"2024-04-28T13:31:24.731713+0000","last_became_active":"2024-04-28T13:31:21.643318+0000","last_became_peered":"2024-04-28T13:31:21.643318+0000","last_unstale":"2024-04-28T13:31:24.731713+0000","last_undegraded":"2024-04-28T13:31:24.731713+0000","last_fullsized":"2024-04-28T13:31:24.731713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:45:35.324680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231673+0000","last_change":"2024-04-28T13:31:21.659912+0000","last_active":"2024-04-28T13:31:24.231673+0000","last_peered":"2024-04-28T13:31:24.231673+0000","last_clean":"2024-04-28T13:31:24.231673+0000","last_became_active":"2024-04-28T13:31:21.659577+0000","last_became_peered":"2024-04-28T13:31:21.659577+0000","last_unstale":"2024-04-28T13:31:24.231673+0000","last_undegraded":"2024-04-28T13:31:24.231673+0000","last_fullsized":"2024-04-28T13:31:24.231673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:04:16.728668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731562+0000","last_change":"2024-04-28T13:31:21.654570+0000","last_active":"2024-04-28T13:31:24.731562+0000","last_peered":"2024-04-28T13:31:24.731562+0000","last_clean":"2024-04-28T13:31:24.731562+0000","last_became_active":"2024-04-28T13:31:21.654238+0000","last_became_peered":"2024-04-28T13:31:21.654238+0000","last_unstale":"2024-04-28T13:31:24.731562+0000","last_undegraded":"2024-04-28T13:31:24.731562+0000","last_fullsized":"2024-04-28T13:31:24.731562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:33:40.307112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730818+0000","last_change":"2024-04-28T13:31:21.653783+0000","last_active":"2024-04-28T13:31:24.730818+0000","last_peered":"2024-04-28T13:31:24.730818+0000","last_clean":"2024-04-28T13:31:24.730818+0000","last_became_active":"2024-04-28T13:31:21.653493+0000","last_became_peered":"2024-04-28T13:31:21.653493+0000","last_unstale":"2024-04-28T13:31:24.730818+0000","last_undegraded":"2024-04-28T13:31:24.730818+0000","last_fullsized":"2024-04-28T13:31:24.730818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:52:32.051589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647048+0000","last_change":"2024-04-28T13:31:21.660686+0000","last_active":"2024-04-28T13:31:22.647048+0000","last_peered":"2024-04-28T13:31:22.647048+0000","last_clean":"2024-04-28T13:31:22.647048+0000","last_became_active":"2024-04-28T13:31:21.660320+0000","last_became_peered":"2024-04-28T13:31:21.660320+0000","last_unstale":"2024-04-28T13:31:22.647048+0000","last_undegraded":"2024-04-28T13:31:22.647048+0000","last_fullsized":"2024-04-28T13:31:22.647048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:09:03.049024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730352+0000","last_change":"2024-04-28T13:31:21.640131+0000","last_active":"2024-04-28T13:31:24.730352+0000","last_peered":"2024-04-28T13:31:24.730352+0000","last_clean":"2024-04-28T13:31:24.730352+0000","last_became_active":"2024-04-28T13:31:21.639821+0000","last_became_peered":"2024-04-28T13:31:21.639821+0000","last_unstale":"2024-04-28T13:31:24.730352+0000","last_undegraded":"2024-04-28T13:31:24.730352+0000","last_fullsized":"2024-04-28T13:31:24.730352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:39:23.345225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646449+0000","last_change":"2024-04-28T13:31:21.648922+0000","last_active":"2024-04-28T13:31:22.646449+0000","last_peered":"2024-04-28T13:31:22.646449+0000","last_clean":"2024-04-28T13:31:22.646449+0000","last_became_active":"2024-04-28T13:31:21.639876+0000","last_became_peered":"2024-04-28T13:31:21.639876+0000","last_unstale":"2024-04-28T13:31:22.646449+0000","last_undegraded":"2024-04-28T13:31:22.646449+0000","last_fullsized":"2024-04-28T13:31:22.646449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:43:16.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732198+0000","last_change":"2024-04-28T13:31:21.644835+0000","last_active":"2024-04-28T13:31:24.732198+0000","last_peered":"2024-04-28T13:31:24.732198+0000","last_clean":"2024-04-28T13:31:24.732198+0000","last_became_active":"2024-04-28T13:31:21.644564+0000","last_became_peered":"2024-04-28T13:31:21.644564+0000","last_unstale":"2024-04-28T13:31:24.732198+0000","last_undegraded":"2024-04-28T13:31:24.732198+0000","last_fullsized":"2024-04-28T13:31:24.732198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:09:03.049024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732493+0000","last_change":"2024-04-28T13:31:21.653502+0000","last_active":"2024-04-28T13:31:24.732493+0000","last_peered":"2024-04-28T13:31:24.732493+0000","last_clean":"2024-04-28T13:31:24.732493+0000","last_became_active":"2024-04-28T13:31:21.653235+0000","last_became_peered":"2024-04-28T13:31:21.653235+0000","last_unstale":"2024-04-28T13:31:24.732493+0000","last_undegraded":"2024-04-28T13:31:24.732493+0000","last_fullsized":"2024-04-28T13:31:24.732493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:43:16.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640377+0000","last_change":"2024-04-28T13:31:21.649404+0000","last_active":"2024-04-28T13:31:22.640377+0000","last_peered":"2024-04-28T13:31:22.640377+0000","last_clean":"2024-04-28T13:31:22.640377+0000","last_became_active":"2024-04-28T13:31:21.649097+0000","last_became_peered":"2024-04-28T13:31:21.649097+0000","last_unstale":"2024-04-28T13:31:22.640377+0000","last_undegraded":"2024-04-28T13:31:22.640377+0000","last_fullsized":"2024-04-28T13:31:22.640377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:43:16.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731939+0000","last_change":"2024-04-28T13:31:21.649378+0000","last_active":"2024-04-28T13:31:24.731939+0000","last_peered":"2024-04-28T13:31:24.731939+0000","last_clean":"2024-04-28T13:31:24.731939+0000","last_became_active":"2024-04-28T13:31:21.647397+0000","last_became_peered":"2024-04-28T13:31:21.647397+0000","last_unstale":"2024-04-28T13:31:24.731939+0000","last_undegraded":"2024-04-28T13:31:24.731939+0000","last_fullsized":"2024-04-28T13:31:24.731939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:25.059779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730846+0000","last_change":"2024-04-28T13:31:21.642005+0000","last_active":"2024-04-28T13:31:24.730846+0000","last_peered":"2024-04-28T13:31:24.730846+0000","last_clean":"2024-04-28T13:31:24.730846+0000","last_became_active":"2024-04-28T13:31:21.641662+0000","last_became_peered":"2024-04-28T13:31:21.641662+0000","last_unstale":"2024-04-28T13:31:24.730846+0000","last_undegraded":"2024-04-28T13:31:24.730846+0000","last_fullsized":"2024-04-28T13:31:24.730846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:32.352858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647442+0000","last_change":"2024-04-28T13:31:21.642673+0000","last_active":"2024-04-28T13:31:22.647442+0000","last_peered":"2024-04-28T13:31:22.647442+0000","last_clean":"2024-04-28T13:31:22.647442+0000","last_became_active":"2024-04-28T13:31:21.640799+0000","last_became_peered":"2024-04-28T13:31:21.640799+0000","last_unstale":"2024-04-28T13:31:22.647442+0000","last_undegraded":"2024-04-28T13:31:22.647442+0000","last_fullsized":"2024-04-28T13:31:22.647442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:04:16.728668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730785+0000","last_change":"2024-04-28T13:31:21.657008+0000","last_active":"2024-04-28T13:31:24.730785+0000","last_peered":"2024-04-28T13:31:24.730785+0000","last_clean":"2024-04-28T13:31:24.730785+0000","last_became_active":"2024-04-28T13:31:21.656390+0000","last_became_peered":"2024-04-28T13:31:21.656390+0000","last_unstale":"2024-04-28T13:31:24.730785+0000","last_undegraded":"2024-04-28T13:31:24.730785+0000","last_fullsized":"2024-04-28T13:31:24.730785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:01.137759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731902+0000","last_change":"2024-04-28T13:31:21.643888+0000","last_active":"2024-04-28T13:31:24.731902+0000","last_peered":"2024-04-28T13:31:24.731902+0000","last_clean":"2024-04-28T13:31:24.731902+0000","last_became_active":"2024-04-28T13:31:21.643153+0000","last_became_peered":"2024-04-28T13:31:21.643153+0000","last_unstale":"2024-04-28T13:31:24.731902+0000","last_undegraded":"2024-04-28T13:31:24.731902+0000","last_fullsized":"2024-04-28T13:31:24.731902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:52:32.051589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.645551+0000","last_change":"2024-04-28T13:31:21.640935+0000","last_active":"2024-04-28T13:31:22.645551+0000","last_peered":"2024-04-28T13:31:22.645551+0000","last_clean":"2024-04-28T13:31:22.645551+0000","last_became_active":"2024-04-28T13:31:21.640633+0000","last_became_peered":"2024-04-28T13:31:21.640633+0000","last_unstale":"2024-04-28T13:31:22.645551+0000","last_undegraded":"2024-04-28T13:31:22.645551+0000","last_fullsized":"2024-04-28T13:31:22.645551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:23.207789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731261+0000","last_change":"2024-04-28T13:31:20.624370+0000","last_active":"2024-04-28T13:31:24.731261+0000","last_peered":"2024-04-28T13:31:24.731261+0000","last_clean":"2024-04-28T13:31:24.731261+0000","last_became_active":"2024-04-28T13:31:20.624091+0000","last_became_peered":"2024-04-28T13:31:20.624091+0000","last_unstale":"2024-04-28T13:31:24.731261+0000","last_undegraded":"2024-04-28T13:31:24.731261+0000","last_fullsized":"2024-04-28T13:31:24.731261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:35.060920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731062+0000","last_change":"2024-04-28T13:31:21.626821+0000","last_active":"2024-04-28T13:31:24.731062+0000","last_peered":"2024-04-28T13:31:24.731062+0000","last_clean":"2024-04-28T13:31:24.731062+0000","last_became_active":"2024-04-28T13:31:21.625590+0000","last_became_peered":"2024-04-28T13:31:21.625590+0000","last_unstale":"2024-04-28T13:31:24.731062+0000","last_undegraded":"2024-04-28T13:31:24.731062+0000","last_fullsized":"2024-04-28T13:31:24.731062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:33.431077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731325+0000","last_change":"2024-04-28T13:31:20.629009+0000","last_active":"2024-04-28T13:31:24.731325+0000","last_peered":"2024-04-28T13:31:24.731325+0000","last_clean":"2024-04-28T13:31:24.731325+0000","last_became_active":"2024-04-28T13:31:20.627394+0000","last_became_peered":"2024-04-28T13:31:20.627394+0000","last_unstale":"2024-04-28T13:31:24.731325+0000","last_undegraded":"2024-04-28T13:31:24.731325+0000","last_fullsized":"2024-04-28T13:31:24.731325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:01.229713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730417+0000","last_change":"2024-04-28T13:31:20.624345+0000","last_active":"2024-04-28T13:31:24.730417+0000","last_peered":"2024-04-28T13:31:24.730417+0000","last_clean":"2024-04-28T13:31:24.730417+0000","last_became_active":"2024-04-28T13:31:20.624052+0000","last_became_peered":"2024-04-28T13:31:20.624052+0000","last_unstale":"2024-04-28T13:31:24.730417+0000","last_undegraded":"2024-04-28T13:31:24.730417+0000","last_fullsized":"2024-04-28T13:31:24.730417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:46.616532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.729933+0000","last_change":"2024-04-28T13:31:21.637599+0000","last_active":"2024-04-28T13:31:24.729933+0000","last_peered":"2024-04-28T13:31:24.729933+0000","last_clean":"2024-04-28T13:31:24.729933+0000","last_became_active":"2024-04-28T13:31:21.637366+0000","last_became_peered":"2024-04-28T13:31:21.637366+0000","last_unstale":"2024-04-28T13:31:24.729933+0000","last_undegraded":"2024-04-28T13:31:24.729933+0000","last_fullsized":"2024-04-28T13:31:24.729933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:14:19.057730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646642+0000","last_change":"2024-04-28T13:31:20.625321+0000","last_active":"2024-04-28T13:31:22.646642+0000","last_peered":"2024-04-28T13:31:22.646642+0000","last_clean":"2024-04-28T13:31:22.646642+0000","last_became_active":"2024-04-28T13:31:20.624034+0000","last_became_peered":"2024-04-28T13:31:20.624034+0000","last_unstale":"2024-04-28T13:31:22.646642+0000","last_undegraded":"2024-04-28T13:31:22.646642+0000","last_fullsized":"2024-04-28T13:31:22.646642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:22.193491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.639404+0000","last_change":"2024-04-28T13:31:21.650660+0000","last_active":"2024-04-28T13:31:22.639404+0000","last_peered":"2024-04-28T13:31:22.639404+0000","last_clean":"2024-04-28T13:31:22.639404+0000","last_became_active":"2024-04-28T13:31:21.650349+0000","last_became_peered":"2024-04-28T13:31:21.650349+0000","last_unstale":"2024-04-28T13:31:22.639404+0000","last_undegraded":"2024-04-28T13:31:22.639404+0000","last_fullsized":"2024-04-28T13:31:22.639404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:14:19.057730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732233+0000","last_change":"2024-04-28T13:31:20.632738+0000","last_active":"2024-04-28T13:31:24.732233+0000","last_peered":"2024-04-28T13:31:24.732233+0000","last_clean":"2024-04-28T13:31:24.732233+0000","last_became_active":"2024-04-28T13:31:20.631174+0000","last_became_peered":"2024-04-28T13:31:20.631174+0000","last_unstale":"2024-04-28T13:31:24.732233+0000","last_undegraded":"2024-04-28T13:31:24.732233+0000","last_fullsized":"2024-04-28T13:31:24.732233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:50.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647343+0000","last_change":"2024-04-28T13:31:21.639067+0000","last_active":"2024-04-28T13:31:22.647343+0000","last_peered":"2024-04-28T13:31:22.647343+0000","last_clean":"2024-04-28T13:31:22.647343+0000","last_became_active":"2024-04-28T13:31:21.638664+0000","last_became_peered":"2024-04-28T13:31:21.638664+0000","last_unstale":"2024-04-28T13:31:22.647343+0000","last_undegraded":"2024-04-28T13:31:22.647343+0000","last_fullsized":"2024-04-28T13:31:22.647343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:44:09.266587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647483+0000","last_change":"2024-04-28T13:31:20.631748+0000","last_active":"2024-04-28T13:31:22.647483+0000","last_peered":"2024-04-28T13:31:22.647483+0000","last_clean":"2024-04-28T13:31:22.647483+0000","last_became_active":"2024-04-28T13:31:20.631345+0000","last_became_peered":"2024-04-28T13:31:20.631345+0000","last_unstale":"2024-04-28T13:31:22.647483+0000","last_undegraded":"2024-04-28T13:31:22.647483+0000","last_fullsized":"2024-04-28T13:31:22.647483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:09:16.028560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730844+0000","last_change":"2024-04-28T13:31:21.636554+0000","last_active":"2024-04-28T13:31:24.730844+0000","last_peered":"2024-04-28T13:31:24.730844+0000","last_clean":"2024-04-28T13:31:24.730844+0000","last_became_active":"2024-04-28T13:31:21.636062+0000","last_became_peered":"2024-04-28T13:31:21.636062+0000","last_unstale":"2024-04-28T13:31:24.730844+0000","last_undegraded":"2024-04-28T13:31:24.730844+0000","last_fullsized":"2024-04-28T13:31:24.730844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:39:23.345225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640458+0000","last_change":"2024-04-28T13:31:20.627856+0000","last_active":"2024-04-28T13:31:22.640458+0000","last_peered":"2024-04-28T13:31:22.640458+0000","last_clean":"2024-04-28T13:31:22.640458+0000","last_became_active":"2024-04-28T13:31:20.627632+0000","last_became_peered":"2024-04-28T13:31:20.627632+0000","last_unstale":"2024-04-28T13:31:22.640458+0000","last_undegraded":"2024-04-28T13:31:22.640458+0000","last_fullsized":"2024-04-28T13:31:22.640458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:09:16.028560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.648191+0000","last_change":"2024-04-28T13:31:21.657348+0000","last_active":"2024-04-28T13:31:22.648191+0000","last_peered":"2024-04-28T13:31:22.648191+0000","last_clean":"2024-04-28T13:31:22.648191+0000","last_became_active":"2024-04-28T13:31:21.657055+0000","last_became_peered":"2024-04-28T13:31:21.657055+0000","last_unstale":"2024-04-28T13:31:22.648191+0000","last_undegraded":"2024-04-28T13:31:22.648191+0000","last_fullsized":"2024-04-28T13:31:22.648191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:45:35.324680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731568+0000","last_change":"2024-04-28T13:31:20.629687+0000","last_active":"2024-04-28T13:31:24.731568+0000","last_peered":"2024-04-28T13:31:24.731568+0000","last_clean":"2024-04-28T13:31:24.731568+0000","last_became_active":"2024-04-28T13:31:20.628070+0000","last_became_peered":"2024-04-28T13:31:20.628070+0000","last_unstale":"2024-04-28T13:31:24.731568+0000","last_undegraded":"2024-04-28T13:31:24.731568+0000","last_fullsized":"2024-04-28T13:31:24.731568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:09:16.028560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231080+0000","last_change":"2024-04-28T13:31:21.659218+0000","last_active":"2024-04-28T13:31:24.231080+0000","last_peered":"2024-04-28T13:31:24.231080+0000","last_clean":"2024-04-28T13:31:24.231080+0000","last_became_active":"2024-04-28T13:31:21.658927+0000","last_became_peered":"2024-04-28T13:31:21.658927+0000","last_unstale":"2024-04-28T13:31:24.231080+0000","last_undegraded":"2024-04-28T13:31:24.231080+0000","last_fullsized":"2024-04-28T13:31:24.231080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:09:03.049024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730927+0000","last_change":"2024-04-28T13:31:20.624425+0000","last_active":"2024-04-28T13:31:24.730927+0000","last_peered":"2024-04-28T13:31:24.730927+0000","last_clean":"2024-04-28T13:31:24.730927+0000","last_became_active":"2024-04-28T13:31:20.623737+0000","last_became_peered":"2024-04-28T13:31:20.623737+0000","last_unstale":"2024-04-28T13:31:24.730927+0000","last_undegraded":"2024-04-28T13:31:24.730927+0000","last_fullsized":"2024-04-28T13:31:24.730927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:09:09.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731087+0000","last_change":"2024-04-28T13:31:21.658397+0000","last_active":"2024-04-28T13:31:24.731087+0000","last_peered":"2024-04-28T13:31:24.731087+0000","last_clean":"2024-04-28T13:31:24.731087+0000","last_became_active":"2024-04-28T13:31:21.658082+0000","last_became_peered":"2024-04-28T13:31:21.658082+0000","last_unstale":"2024-04-28T13:31:24.731087+0000","last_undegraded":"2024-04-28T13:31:24.731087+0000","last_fullsized":"2024-04-28T13:31:24.731087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:43:16.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647574+0000","last_change":"2024-04-28T13:31:20.625371+0000","last_active":"2024-04-28T13:31:22.647574+0000","last_peered":"2024-04-28T13:31:22.647574+0000","last_clean":"2024-04-28T13:31:22.647574+0000","last_became_active":"2024-04-28T13:31:20.625028+0000","last_became_peered":"2024-04-28T13:31:20.625028+0000","last_unstale":"2024-04-28T13:31:22.647574+0000","last_undegraded":"2024-04-28T13:31:22.647574+0000","last_fullsized":"2024-04-28T13:31:22.647574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:01.229713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231541+0000","last_change":"2024-04-28T13:31:21.660871+0000","last_active":"2024-04-28T13:31:24.231541+0000","last_peered":"2024-04-28T13:31:24.231541+0000","last_clean":"2024-04-28T13:31:24.231541+0000","last_became_active":"2024-04-28T13:31:21.660649+0000","last_became_peered":"2024-04-28T13:31:21.660649+0000","last_unstale":"2024-04-28T13:31:24.231541+0000","last_undegraded":"2024-04-28T13:31:24.231541+0000","last_fullsized":"2024-04-28T13:31:24.231541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:44:09.266587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731265+0000","last_change":"2024-04-28T13:31:20.625903+0000","last_active":"2024-04-28T13:31:24.731265+0000","last_peered":"2024-04-28T13:31:24.731265+0000","last_clean":"2024-04-28T13:31:24.731265+0000","last_became_active":"2024-04-28T13:31:20.625350+0000","last_became_peered":"2024-04-28T13:31:20.625350+0000","last_unstale":"2024-04-28T13:31:24.731265+0000","last_undegraded":"2024-04-28T13:31:24.731265+0000","last_fullsized":"2024-04-28T13:31:24.731265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:35.060920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731578+0000","last_change":"2024-04-28T13:31:21.643517+0000","last_active":"2024-04-28T13:31:24.731578+0000","last_peered":"2024-04-28T13:31:24.731578+0000","last_clean":"2024-04-28T13:31:24.731578+0000","last_became_active":"2024-04-28T13:31:21.643128+0000","last_became_peered":"2024-04-28T13:31:21.643128+0000","last_unstale":"2024-04-28T13:31:24.731578+0000","last_undegraded":"2024-04-28T13:31:24.731578+0000","last_fullsized":"2024-04-28T13:31:24.731578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:14:19.057730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.729987+0000","last_change":"2024-04-28T13:31:20.609440+0000","last_active":"2024-04-28T13:31:24.729987+0000","last_peered":"2024-04-28T13:31:24.729987+0000","last_clean":"2024-04-28T13:31:24.729987+0000","last_became_active":"2024-04-28T13:31:20.609120+0000","last_became_peered":"2024-04-28T13:31:20.609120+0000","last_unstale":"2024-04-28T13:31:24.729987+0000","last_undegraded":"2024-04-28T13:31:24.729987+0000","last_fullsized":"2024-04-28T13:31:24.729987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:01.229713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.230626+0000","last_change":"2024-04-28T13:31:21.654431+0000","last_active":"2024-04-28T13:31:24.230626+0000","last_peered":"2024-04-28T13:31:24.230626+0000","last_clean":"2024-04-28T13:31:24.230626+0000","last_became_active":"2024-04-28T13:31:21.653884+0000","last_became_peered":"2024-04-28T13:31:21.653884+0000","last_unstale":"2024-04-28T13:31:24.230626+0000","last_undegraded":"2024-04-28T13:31:24.230626+0000","last_fullsized":"2024-04-28T13:31:24.230626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:41:07.707990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646224+0000","last_change":"2024-04-28T13:31:20.619411+0000","last_active":"2024-04-28T13:31:22.646224+0000","last_peered":"2024-04-28T13:31:22.646224+0000","last_clean":"2024-04-28T13:31:22.646224+0000","last_became_active":"2024-04-28T13:31:20.618632+0000","last_became_peered":"2024-04-28T13:31:20.618632+0000","last_unstale":"2024-04-28T13:31:22.646224+0000","last_undegraded":"2024-04-28T13:31:22.646224+0000","last_fullsized":"2024-04-28T13:31:22.646224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:09:09.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.232213+0000","last_change":"2024-04-28T13:31:21.656026+0000","last_active":"2024-04-28T13:31:24.232213+0000","last_peered":"2024-04-28T13:31:24.232213+0000","last_clean":"2024-04-28T13:31:24.232213+0000","last_became_active":"2024-04-28T13:31:21.655737+0000","last_became_peered":"2024-04-28T13:31:21.655737+0000","last_unstale":"2024-04-28T13:31:24.232213+0000","last_undegraded":"2024-04-28T13:31:24.232213+0000","last_fullsized":"2024-04-28T13:31:24.232213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:53:05.667773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731528+0000","last_change":"2024-04-28T13:31:20.616200+0000","last_active":"2024-04-28T13:31:24.731528+0000","last_peered":"2024-04-28T13:31:24.731528+0000","last_clean":"2024-04-28T13:31:24.731528+0000","last_became_active":"2024-04-28T13:31:20.615813+0000","last_became_peered":"2024-04-28T13:31:20.615813+0000","last_unstale":"2024-04-28T13:31:24.731528+0000","last_undegraded":"2024-04-28T13:31:24.731528+0000","last_fullsized":"2024-04-28T13:31:24.731528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:35.060920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.232625+0000","last_change":"2024-04-28T13:31:21.657900+0000","last_active":"2024-04-28T13:31:24.232625+0000","last_peered":"2024-04-28T13:31:24.232625+0000","last_clean":"2024-04-28T13:31:24.232625+0000","last_became_active":"2024-04-28T13:31:21.657276+0000","last_became_peered":"2024-04-28T13:31:21.657276+0000","last_unstale":"2024-04-28T13:31:24.232625+0000","last_undegraded":"2024-04-28T13:31:24.232625+0000","last_fullsized":"2024-04-28T13:31:24.232625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:15:21.164101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640518+0000","last_change":"2024-04-28T13:31:20.620725+0000","last_active":"2024-04-28T13:31:22.640518+0000","last_peered":"2024-04-28T13:31:22.640518+0000","last_clean":"2024-04-28T13:31:22.640518+0000","last_became_active":"2024-04-28T13:31:20.620392+0000","last_became_peered":"2024-04-28T13:31:20.620392+0000","last_unstale":"2024-04-28T13:31:22.640518+0000","last_undegraded":"2024-04-28T13:31:22.640518+0000","last_fullsized":"2024-04-28T13:31:22.640518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:50.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647204+0000","last_change":"2024-04-28T13:31:21.640722+0000","last_active":"2024-04-28T13:31:22.647204+0000","last_peered":"2024-04-28T13:31:22.647204+0000","last_clean":"2024-04-28T13:31:22.647204+0000","last_became_active":"2024-04-28T13:31:21.637069+0000","last_became_peered":"2024-04-28T13:31:21.637069+0000","last_unstale":"2024-04-28T13:31:22.647204+0000","last_undegraded":"2024-04-28T13:31:22.647204+0000","last_fullsized":"2024-04-28T13:31:22.647204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:26:26.344814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731464+0000","last_change":"2024-04-28T13:31:20.623333+0000","last_active":"2024-04-28T13:31:24.731464+0000","last_peered":"2024-04-28T13:31:24.731464+0000","last_clean":"2024-04-28T13:31:24.731464+0000","last_became_active":"2024-04-28T13:31:20.623063+0000","last_became_peered":"2024-04-28T13:31:20.623063+0000","last_unstale":"2024-04-28T13:31:24.731464+0000","last_undegraded":"2024-04-28T13:31:24.731464+0000","last_fullsized":"2024-04-28T13:31:24.731464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:50.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231274+0000","last_change":"2024-04-28T13:31:21.663106+0000","last_active":"2024-04-28T13:31:24.231274+0000","last_peered":"2024-04-28T13:31:24.231274+0000","last_clean":"2024-04-28T13:31:24.231274+0000","last_became_active":"2024-04-28T13:31:21.662627+0000","last_became_peered":"2024-04-28T13:31:21.662627+0000","last_unstale":"2024-04-28T13:31:24.231274+0000","last_undegraded":"2024-04-28T13:31:24.231274+0000","last_fullsized":"2024-04-28T13:31:24.231274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:04:59.462014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.230624+0000","last_change":"2024-04-28T13:31:11.831538+0000","last_active":"2024-04-28T13:31:24.230624+0000","last_peered":"2024-04-28T13:31:24.230624+0000","last_clean":"2024-04-28T13:31:24.230624+0000","last_became_active":"2024-04-28T13:31:11.828959+0000","last_became_peered":"2024-04-28T13:31:11.828959+0000","last_unstale":"2024-04-28T13:31:24.230624+0000","last_undegraded":"2024-04-28T13:31:24.230624+0000","last_fullsized":"2024-04-28T13:31:24.230624+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_clean_scrub_stamp":"2024-04-28T13:31:10.444388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:25:27.078911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.645900+0000","last_change":"2024-04-28T13:31:20.619170+0000","last_active":"2024-04-28T13:31:22.645900+0000","last_peered":"2024-04-28T13:31:22.645900+0000","last_clean":"2024-04-28T13:31:22.645900+0000","last_became_active":"2024-04-28T13:31:20.618861+0000","last_became_peered":"2024-04-28T13:31:20.618861+0000","last_unstale":"2024-04-28T13:31:22.645900+0000","last_undegraded":"2024-04-28T13:31:22.645900+0000","last_fullsized":"2024-04-28T13:31:22.645900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:35.060920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730744+0000","last_change":"2024-04-28T13:31:21.649271+0000","last_active":"2024-04-28T13:31:24.730744+0000","last_peered":"2024-04-28T13:31:24.730744+0000","last_clean":"2024-04-28T13:31:24.730744+0000","last_became_active":"2024-04-28T13:31:21.649009+0000","last_became_peered":"2024-04-28T13:31:21.649009+0000","last_unstale":"2024-04-28T13:31:24.730744+0000","last_undegraded":"2024-04-28T13:31:24.730744+0000","last_fullsized":"2024-04-28T13:31:24.730744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:32.352858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.230284+0000","last_change":"2024-04-28T13:31:20.628745+0000","last_active":"2024-04-28T13:31:24.230284+0000","last_peered":"2024-04-28T13:31:24.230284+0000","last_clean":"2024-04-28T13:31:24.230284+0000","last_became_active":"2024-04-28T13:31:20.628430+0000","last_became_peered":"2024-04-28T13:31:20.628430+0000","last_unstale":"2024-04-28T13:31:24.230284+0000","last_undegraded":"2024-04-28T13:31:24.230284+0000","last_fullsized":"2024-04-28T13:31:24.230284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:09:16.028560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.729791+0000","last_change":"2024-04-28T13:31:21.628901+0000","last_active":"2024-04-28T13:31:24.729791+0000","last_peered":"2024-04-28T13:31:24.729791+0000","last_clean":"2024-04-28T13:31:24.729791+0000","last_became_active":"2024-04-28T13:31:21.628483+0000","last_became_peered":"2024-04-28T13:31:21.628483+0000","last_unstale":"2024-04-28T13:31:24.729791+0000","last_undegraded":"2024-04-28T13:31:24.729791+0000","last_fullsized":"2024-04-28T13:31:24.729791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:06:20.265833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731104+0000","last_change":"2024-04-28T13:31:20.615910+0000","last_active":"2024-04-28T13:31:24.731104+0000","last_peered":"2024-04-28T13:31:24.731104+0000","last_clean":"2024-04-28T13:31:24.731104+0000","last_became_active":"2024-04-28T13:31:20.615610+0000","last_became_peered":"2024-04-28T13:31:20.615610+0000","last_unstale":"2024-04-28T13:31:24.731104+0000","last_undegraded":"2024-04-28T13:31:24.731104+0000","last_fullsized":"2024-04-28T13:31:24.731104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:50:14.806138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.639480+0000","last_change":"2024-04-28T13:31:21.648562+0000","last_active":"2024-04-28T13:31:22.639480+0000","last_peered":"2024-04-28T13:31:22.639480+0000","last_clean":"2024-04-28T13:31:22.639480+0000","last_became_active":"2024-04-28T13:31:21.648282+0000","last_became_peered":"2024-04-28T13:31:21.648282+0000","last_unstale":"2024-04-28T13:31:22.639480+0000","last_undegraded":"2024-04-28T13:31:22.639480+0000","last_fullsized":"2024-04-28T13:31:22.639480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:52:32.051589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647763+0000","last_change":"2024-04-28T13:31:20.623861+0000","last_active":"2024-04-28T13:31:22.647763+0000","last_peered":"2024-04-28T13:31:22.647763+0000","last_clean":"2024-04-28T13:31:22.647763+0000","last_became_active":"2024-04-28T13:31:20.623459+0000","last_became_peered":"2024-04-28T13:31:20.623459+0000","last_unstale":"2024-04-28T13:31:22.647763+0000","last_undegraded":"2024-04-28T13:31:22.647763+0000","last_fullsized":"2024-04-28T13:31:22.647763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:50.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.645431+0000","last_change":"2024-04-28T13:31:21.634320+0000","last_active":"2024-04-28T13:31:22.645431+0000","last_peered":"2024-04-28T13:31:22.645431+0000","last_clean":"2024-04-28T13:31:22.645431+0000","last_became_active":"2024-04-28T13:31:21.634045+0000","last_became_peered":"2024-04-28T13:31:21.634045+0000","last_unstale":"2024-04-28T13:31:22.645431+0000","last_undegraded":"2024-04-28T13:31:22.645431+0000","last_fullsized":"2024-04-28T13:31:22.645431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:51.731818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.639846+0000","last_change":"2024-04-28T13:31:20.612653+0000","last_active":"2024-04-28T13:31:22.639846+0000","last_peered":"2024-04-28T13:31:22.639846+0000","last_clean":"2024-04-28T13:31:22.639846+0000","last_became_active":"2024-04-28T13:31:20.612341+0000","last_became_peered":"2024-04-28T13:31:20.612341+0000","last_unstale":"2024-04-28T13:31:22.639846+0000","last_undegraded":"2024-04-28T13:31:22.639846+0000","last_fullsized":"2024-04-28T13:31:22.639846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:45:34.310382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647331+0000","last_change":"2024-04-28T13:31:21.642073+0000","last_active":"2024-04-28T13:31:22.647331+0000","last_peered":"2024-04-28T13:31:22.647331+0000","last_clean":"2024-04-28T13:31:22.647331+0000","last_became_active":"2024-04-28T13:31:21.641771+0000","last_became_peered":"2024-04-28T13:31:21.641771+0000","last_unstale":"2024-04-28T13:31:22.647331+0000","last_undegraded":"2024-04-28T13:31:22.647331+0000","last_fullsized":"2024-04-28T13:31:22.647331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:04:59.462014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231572+0000","last_change":"2024-04-28T13:31:20.634223+0000","last_active":"2024-04-28T13:31:24.231572+0000","last_peered":"2024-04-28T13:31:24.231572+0000","last_clean":"2024-04-28T13:31:24.231572+0000","last_became_active":"2024-04-28T13:31:20.633834+0000","last_became_peered":"2024-04-28T13:31:20.633834+0000","last_unstale":"2024-04-28T13:31:24.231572+0000","last_undegraded":"2024-04-28T13:31:24.231572+0000","last_fullsized":"2024-04-28T13:31:24.231572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:37:34.377933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.639712+0000","last_change":"2024-04-28T13:31:21.643483+0000","last_active":"2024-04-28T13:31:22.639712+0000","last_peered":"2024-04-28T13:31:22.639712+0000","last_clean":"2024-04-28T13:31:22.639712+0000","last_became_active":"2024-04-28T13:31:21.643157+0000","last_became_peered":"2024-04-28T13:31:21.643157+0000","last_unstale":"2024-04-28T13:31:22.639712+0000","last_undegraded":"2024-04-28T13:31:22.639712+0000","last_fullsized":"2024-04-28T13:31:22.639712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:39:23.345225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640516+0000","last_change":"2024-04-28T13:31:20.630471+0000","last_active":"2024-04-28T13:31:22.640516+0000","last_peered":"2024-04-28T13:31:22.640516+0000","last_clean":"2024-04-28T13:31:22.640516+0000","last_became_active":"2024-04-28T13:31:20.630161+0000","last_became_peered":"2024-04-28T13:31:20.630161+0000","last_unstale":"2024-04-28T13:31:22.640516+0000","last_undegraded":"2024-04-28T13:31:22.640516+0000","last_fullsized":"2024-04-28T13:31:22.640516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:22.193491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732154+0000","last_change":"2024-04-28T13:31:21.648853+0000","last_active":"2024-04-28T13:31:24.732154+0000","last_peered":"2024-04-28T13:31:24.732154+0000","last_clean":"2024-04-28T13:31:24.732154+0000","last_became_active":"2024-04-28T13:31:21.647783+0000","last_became_peered":"2024-04-28T13:31:21.647783+0000","last_unstale":"2024-04-28T13:31:24.732154+0000","last_undegraded":"2024-04-28T13:31:24.732154+0000","last_fullsized":"2024-04-28T13:31:24.732154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:41:07.707990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732342+0000","last_change":"2024-04-28T13:31:20.629409+0000","last_active":"2024-04-28T13:31:24.732342+0000","last_peered":"2024-04-28T13:31:24.732342+0000","last_clean":"2024-04-28T13:31:24.732342+0000","last_became_active":"2024-04-28T13:31:20.628182+0000","last_became_peered":"2024-04-28T13:31:20.628182+0000","last_unstale":"2024-04-28T13:31:24.732342+0000","last_undegraded":"2024-04-28T13:31:24.732342+0000","last_fullsized":"2024-04-28T13:31:24.732342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:22.193491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730384+0000","last_change":"2024-04-28T13:31:21.640225+0000","last_active":"2024-04-28T13:31:24.730384+0000","last_peered":"2024-04-28T13:31:24.730384+0000","last_clean":"2024-04-28T13:31:24.730384+0000","last_became_active":"2024-04-28T13:31:21.639921+0000","last_became_peered":"2024-04-28T13:31:21.639921+0000","last_unstale":"2024-04-28T13:31:24.730384+0000","last_undegraded":"2024-04-28T13:31:24.730384+0000","last_fullsized":"2024-04-28T13:31:24.730384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:33:40.307112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.230944+0000","last_change":"2024-04-28T13:31:20.633572+0000","last_active":"2024-04-28T13:31:24.230944+0000","last_peered":"2024-04-28T13:31:24.230944+0000","last_clean":"2024-04-28T13:31:24.230944+0000","last_became_active":"2024-04-28T13:31:20.630758+0000","last_became_peered":"2024-04-28T13:31:20.630758+0000","last_unstale":"2024-04-28T13:31:24.230944+0000","last_undegraded":"2024-04-28T13:31:24.230944+0000","last_fullsized":"2024-04-28T13:31:24.230944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:50.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.645842+0000","last_change":"2024-04-28T13:31:21.637178+0000","last_active":"2024-04-28T13:31:22.645842+0000","last_peered":"2024-04-28T13:31:22.645842+0000","last_clean":"2024-04-28T13:31:22.645842+0000","last_became_active":"2024-04-28T13:31:21.636898+0000","last_became_peered":"2024-04-28T13:31:21.636898+0000","last_unstale":"2024-04-28T13:31:22.645842+0000","last_undegraded":"2024-04-28T13:31:22.645842+0000","last_fullsized":"2024-04-28T13:31:22.645842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:09:17.042858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.639609+0000","last_change":"2024-04-28T13:31:20.624716+0000","last_active":"2024-04-28T13:31:22.639609+0000","last_peered":"2024-04-28T13:31:22.639609+0000","last_clean":"2024-04-28T13:31:22.639609+0000","last_became_active":"2024-04-28T13:31:20.623554+0000","last_became_peered":"2024-04-28T13:31:20.623554+0000","last_unstale":"2024-04-28T13:31:22.639609+0000","last_undegraded":"2024-04-28T13:31:22.639609+0000","last_fullsized":"2024-04-28T13:31:22.639609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:01.229713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730144+0000","last_change":"2024-04-28T13:31:21.646597+0000","last_active":"2024-04-28T13:31:24.730144+0000","last_peered":"2024-04-28T13:31:24.730144+0000","last_clean":"2024-04-28T13:31:24.730144+0000","last_became_active":"2024-04-28T13:31:21.646307+0000","last_became_peered":"2024-04-28T13:31:21.646307+0000","last_unstale":"2024-04-28T13:31:24.730144+0000","last_undegraded":"2024-04-28T13:31:24.730144+0000","last_fullsized":"2024-04-28T13:31:24.730144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:52:32.051589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640004+0000","last_change":"2024-04-28T13:31:20.617782+0000","last_active":"2024-04-28T13:31:22.640004+0000","last_peered":"2024-04-28T13:31:22.640004+0000","last_clean":"2024-04-28T13:31:22.640004+0000","last_became_active":"2024-04-28T13:31:20.617213+0000","last_became_peered":"2024-04-28T13:31:20.617213+0000","last_unstale":"2024-04-28T13:31:22.640004+0000","last_undegraded":"2024-04-28T13:31:22.640004+0000","last_fullsized":"2024-04-28T13:31:22.640004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:00.123461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731198+0000","last_change":"2024-04-28T13:31:21.637666+0000","last_active":"2024-04-28T13:31:24.731198+0000","last_peered":"2024-04-28T13:31:24.731198+0000","last_clean":"2024-04-28T13:31:24.731198+0000","last_became_active":"2024-04-28T13:31:21.637359+0000","last_became_peered":"2024-04-28T13:31:21.637359+0000","last_unstale":"2024-04-28T13:31:24.731198+0000","last_undegraded":"2024-04-28T13:31:24.731198+0000","last_fullsized":"2024-04-28T13:31:24.731198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:16:56.574292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231671+0000","last_change":"2024-04-28T13:31:20.632480+0000","last_active":"2024-04-28T13:31:24.231671+0000","last_peered":"2024-04-28T13:31:24.231671+0000","last_clean":"2024-04-28T13:31:24.231671+0000","last_became_active":"2024-04-28T13:31:20.632177+0000","last_became_peered":"2024-04-28T13:31:20.632177+0000","last_unstale":"2024-04-28T13:31:24.231671+0000","last_undegraded":"2024-04-28T13:31:24.231671+0000","last_fullsized":"2024-04-28T13:31:24.231671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:22.193491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732048+0000","last_change":"2024-04-28T13:31:21.645354+0000","last_active":"2024-04-28T13:31:24.732048+0000","last_peered":"2024-04-28T13:31:24.732048+0000","last_clean":"2024-04-28T13:31:24.732048+0000","last_became_active":"2024-04-28T13:31:21.644879+0000","last_became_peered":"2024-04-28T13:31:21.644879+0000","last_unstale":"2024-04-28T13:31:24.732048+0000","last_undegraded":"2024-04-28T13:31:24.732048+0000","last_fullsized":"2024-04-28T13:31:24.732048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:39:23.345225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647195+0000","last_change":"2024-04-28T13:31:21.636961+0000","last_active":"2024-04-28T13:31:22.647195+0000","last_peered":"2024-04-28T13:31:22.647195+0000","last_clean":"2024-04-28T13:31:22.647195+0000","last_became_active":"2024-04-28T13:31:21.636609+0000","last_became_peered":"2024-04-28T13:31:21.636609+0000","last_unstale":"2024-04-28T13:31:22.647195+0000","last_undegraded":"2024-04-28T13:31:22.647195+0000","last_fullsized":"2024-04-28T13:31:22.647195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:28:41.694303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.230631+0000","last_change":"2024-04-28T13:31:20.632395+0000","last_active":"2024-04-28T13:31:24.230631+0000","last_peered":"2024-04-28T13:31:24.230631+0000","last_clean":"2024-04-28T13:31:24.230631+0000","last_became_active":"2024-04-28T13:31:20.632083+0000","last_became_peered":"2024-04-28T13:31:20.632083+0000","last_unstale":"2024-04-28T13:31:24.230631+0000","last_undegraded":"2024-04-28T13:31:24.230631+0000","last_fullsized":"2024-04-28T13:31:24.230631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:00.123461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646732+0000","last_change":"2024-04-28T13:31:21.637466+0000","last_active":"2024-04-28T13:31:22.646732+0000","last_peered":"2024-04-28T13:31:22.646732+0000","last_clean":"2024-04-28T13:31:22.646732+0000","last_became_active":"2024-04-28T13:31:21.636945+0000","last_became_peered":"2024-04-28T13:31:21.636945+0000","last_unstale":"2024-04-28T13:31:22.646732+0000","last_undegraded":"2024-04-28T13:31:22.646732+0000","last_fullsized":"2024-04-28T13:31:22.646732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:41:07.707990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231240+0000","last_change":"2024-04-28T13:31:20.636621+0000","last_active":"2024-04-28T13:31:24.231240+0000","last_peered":"2024-04-28T13:31:24.231240+0000","last_clean":"2024-04-28T13:31:24.231240+0000","last_became_active":"2024-04-28T13:31:20.636363+0000","last_became_peered":"2024-04-28T13:31:20.636363+0000","last_unstale":"2024-04-28T13:31:24.231240+0000","last_undegraded":"2024-04-28T13:31:24.231240+0000","last_fullsized":"2024-04-28T13:31:24.231240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:45:34.310382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730662+0000","last_change":"2024-04-28T13:31:21.636329+0000","last_active":"2024-04-28T13:31:24.730662+0000","last_peered":"2024-04-28T13:31:24.730662+0000","last_clean":"2024-04-28T13:31:24.730662+0000","last_became_active":"2024-04-28T13:31:21.635635+0000","last_became_peered":"2024-04-28T13:31:21.635635+0000","last_unstale":"2024-04-28T13:31:24.730662+0000","last_undegraded":"2024-04-28T13:31:24.730662+0000","last_fullsized":"2024-04-28T13:31:24.730662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:14:19.057730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732045+0000","last_change":"2024-04-28T13:31:20.622197+0000","last_active":"2024-04-28T13:31:24.732045+0000","last_peered":"2024-04-28T13:31:24.732045+0000","last_clean":"2024-04-28T13:31:24.732045+0000","last_became_active":"2024-04-28T13:31:20.621973+0000","last_became_peered":"2024-04-28T13:31:20.621973+0000","last_unstale":"2024-04-28T13:31:24.732045+0000","last_undegraded":"2024-04-28T13:31:24.732045+0000","last_fullsized":"2024-04-28T13:31:24.732045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:50:14.806138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730760+0000","last_change":"2024-04-28T13:31:21.642200+0000","last_active":"2024-04-28T13:31:24.730760+0000","last_peered":"2024-04-28T13:31:24.730760+0000","last_clean":"2024-04-28T13:31:24.730760+0000","last_became_active":"2024-04-28T13:31:21.641925+0000","last_became_peered":"2024-04-28T13:31:21.641925+0000","last_unstale":"2024-04-28T13:31:24.730760+0000","last_undegraded":"2024-04-28T13:31:24.730760+0000","last_fullsized":"2024-04-28T13:31:24.730760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:33.431077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730908+0000","last_change":"2024-04-28T13:31:20.628724+0000","last_active":"2024-04-28T13:31:24.730908+0000","last_peered":"2024-04-28T13:31:24.730908+0000","last_clean":"2024-04-28T13:31:24.730908+0000","last_became_active":"2024-04-28T13:31:20.628363+0000","last_became_peered":"2024-04-28T13:31:20.628363+0000","last_unstale":"2024-04-28T13:31:24.730908+0000","last_undegraded":"2024-04-28T13:31:24.730908+0000","last_fullsized":"2024-04-28T13:31:24.730908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:19:25.770351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T13:31:26.574 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-28T13:31:26.574 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-28T13:31:26.575 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-28T13:31:26.575 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-28T13:31:26.575 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-28T13:31:26.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.819+0000 7f3faa400640 1 Processor -- start 2024-04-28T13:31:26.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.819+0000 7f3faa400640 1 -- start start 2024-04-28T13:31:26.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.821+0000 7f3faa400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3fa40cfda0 0x7f3fa40d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:26.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.821+0000 7f3faa400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3fa40b60a0 0x7f3fa40b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:26.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.821+0000 7f3faa400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fa40d0220 con 0x7f3fa40d1b80 2024-04-28T13:31:26.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.821+0000 7f3faa400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3fa40d0500 con 0x7f3fa40cfda0 2024-04-28T13:31:26.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.821+0000 7f3faa400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3fa40d07e0 con 0x7f3fa40b60a0 2024-04-28T13:31:26.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.821+0000 7f3fa2a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3fa40cfda0 0x7f3fa40d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:26.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.821+0000 7f3fa3400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f3fa40d1b80 0x7f3fa40d3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:55998/0 (socket says 172.21.15.196:55998) 2024-04-28T13:31:26.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.821+0000 7f3fa2a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3fa40cfda0 0x7f3fa40d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35546/0 (socket says 172.21.15.196:35546) 2024-04-28T13:31:26.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.821+0000 7f3fa3400640 1 -- 172.21.15.196:0/2706706023 learned_addr learned my addr 172.21.15.196:0/2706706023 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:26.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.821+0000 7f3fa2a00640 1 -- 172.21.15.196:0/2706706023 >> v2:172.21.15.203:3300/0 conn(0x7f3fa40b60a0 msgr2=0x7f3fa40b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:26.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.821+0000 7f3fa2a00640 1 --2- 172.21.15.196:0/2706706023 >> v2:172.21.15.203:3300/0 conn(0x7f3fa40b60a0 0x7f3fa40b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.821+0000 7f3fa2a00640 1 -- 172.21.15.196:0/2706706023 >> v1:172.21.15.196:6789/0 conn(0x7f3fa40d1b80 legacy=0x7f3fa40d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:26.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.821+0000 7f3fa2a00640 1 -- 172.21.15.196:0/2706706023 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa40d0ac0 con 0x7f3fa40cfda0 2024-04-28T13:31:26.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.821+0000 7f3fa2000640 1 -- 172.21.15.196:0/2706706023 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2793546148 0 0) 0x7f3fa40d0220 con 0x7f3fa40d1b80 2024-04-28T13:31:26.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.822+0000 7f3fa2a00640 1 --2- 172.21.15.196:0/2706706023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40cfda0 0x7f3fa40d1f60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3fa409e9e0 tx=0x7f3f9002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9eb743c7aa2c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:26.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.823+0000 7f3fa2000640 1 -- 172.21.15.196:0/2706706023 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90037070 con 0x7f3fa40cfda0 2024-04-28T13:31:26.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.823+0000 7f3fa2000640 1 -- 172.21.15.196:0/2706706023 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f90032440 con 0x7f3fa40cfda0 2024-04-28T13:31:26.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.823+0000 7f3fa2000640 1 -- 172.21.15.196:0/2706706023 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90032810 con 0x7f3fa40cfda0 2024-04-28T13:31:26.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.823+0000 7f3faa400640 1 -- 172.21.15.196:0/2706706023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40cfda0 msgr2=0x7f3fa40d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:26.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.823+0000 7f3faa400640 1 --2- 172.21.15.196:0/2706706023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40cfda0 0x7f3fa40d1f60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3fa409e9e0 tx=0x7f3f9002a1a0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.823+0000 7f3faa400640 1 -- 172.21.15.196:0/2706706023 shutdown_connections 2024-04-28T13:31:26.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.823+0000 7f3faa400640 1 --2- 172.21.15.196:0/2706706023 >> v2:172.21.15.203:3300/0 conn(0x7f3fa40b60a0 0x7f3fa40b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.823+0000 7f3faa400640 1 --2- 172.21.15.196:0/2706706023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40cfda0 0x7f3fa40d1f60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.823+0000 7f3faa400640 1 -- 172.21.15.196:0/2706706023 >> 172.21.15.196:0/2706706023 conn(0x7f3fa409e540 msgr2=0x7f3fa40d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:26.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.823+0000 7f3faa400640 1 -- 172.21.15.196:0/2706706023 shutdown_connections 2024-04-28T13:31:26.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.823+0000 7f3faa400640 1 -- 172.21.15.196:0/2706706023 wait complete. 2024-04-28T13:31:26.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.825+0000 7f3faa400640 1 Processor -- start 2024-04-28T13:31:26.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.825+0000 7f3faa400640 1 -- start start 2024-04-28T13:31:26.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.826+0000 7f3faa400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fa40b60a0 0x7f3fa414e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:26.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.826+0000 7f3faa400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fa40cfda0 0x7f3fa41522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:26.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.826+0000 7f3faa400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40d1b80 0x7f3fa40a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:26.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.826+0000 7f3faa400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa40d4170 con 0x7f3fa40d1b80 2024-04-28T13:31:26.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.826+0000 7f3faa400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa40b6a70 con 0x7f3fa40b60a0 2024-04-28T13:31:26.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.826+0000 7f3faa400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa415ee50 con 0x7f3fa40cfda0 2024-04-28T13:31:26.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.826+0000 7f3fa3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40d1b80 0x7f3fa40a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:26.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.826+0000 7f3fa3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40d1b80 0x7f3fa40a1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35556/0 (socket says 172.21.15.196:35556) 2024-04-28T13:31:26.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.826+0000 7f3fa3e00640 1 -- 172.21.15.196:0/562795743 learned_addr learned my addr 172.21.15.196:0/562795743 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:26.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.827+0000 7f3fa3400640 1 --2- 172.21.15.196:0/562795743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fa40b60a0 0x7f3fa414e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:26.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.827+0000 7f3fa3e00640 1 -- 172.21.15.196:0/562795743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fa40cfda0 msgr2=0x7f3fa41522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:26.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.827+0000 7f3fa3e00640 1 --2- 172.21.15.196:0/562795743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fa40cfda0 0x7f3fa41522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.827+0000 7f3fa3e00640 1 -- 172.21.15.196:0/562795743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fa40b60a0 msgr2=0x7f3fa414e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:26.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.827+0000 7f3fa3e00640 1 --2- 172.21.15.196:0/562795743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fa40b60a0 0x7f3fa414e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:26.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.827+0000 7f3fa3e00640 1 -- 172.21.15.196:0/562795743 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa409fef0 con 0x7f3fa40d1b80 2024-04-28T13:31:26.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.827+0000 7f3fa3e00640 1 --2- 172.21.15.196:0/562795743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40d1b80 0x7f3fa40a1bd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00b970 tx=0x7f3f9c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:26.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.827+0000 7f3f9b400640 1 -- 172.21.15.196:0/562795743 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c019070 con 0x7f3fa40d1b80 2024-04-28T13:31:26.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.827+0000 7f3f9b400640 1 -- 172.21.15.196:0/562795743 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f9c00cb60 con 0x7f3fa40d1b80 2024-04-28T13:31:26.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.827+0000 7f3f9b400640 1 -- 172.21.15.196:0/562795743 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c014440 con 0x7f3fa40d1b80 2024-04-28T13:31:26.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.827+0000 7f3faa400640 1 -- 172.21.15.196:0/562795743 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa40a01d0 con 0x7f3fa40d1b80 2024-04-28T13:31:26.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.827+0000 7f3faa400640 1 -- 172.21.15.196:0/562795743 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa40a0790 con 0x7f3fa40d1b80 2024-04-28T13:31:26.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.828+0000 7f3faa400640 1 -- 172.21.15.196:0/562795743 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa406a290 con 0x7f3fa40d1b80 2024-04-28T13:31:26.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.829+0000 7f3f9b400640 1 -- 172.21.15.196:0/562795743 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f9c01c070 con 0x7f3fa40d1b80 2024-04-28T13:31:26.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.830+0000 7f3f9b400640 1 --2- 172.21.15.196:0/562795743 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3f8406c8f0 0x7f3f8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:26.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.830+0000 7f3f9b400640 1 -- 172.21.15.196:0/562795743 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3f9c08ba20 con 0x7f3fa40d1b80 2024-04-28T13:31:26.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.831+0000 7f3fa3400640 1 --2- 172.21.15.196:0/562795743 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3f8406c8f0 0x7f3f8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:26.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.832+0000 7f3fa3400640 1 --2- 172.21.15.196:0/562795743 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3f8406c8f0 0x7f3f8406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f94000f60 tx=0x7f3f94007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:26.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:26.835+0000 7f3f9b400640 1 -- 172.21.15.196:0/562795743 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f9c059d20 con 0x7f3fa40d1b80 2024-04-28T13:31:27.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.087+0000 7f3faa400640 1 -- 172.21.15.196:0/562795743 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f3fa40d3170 con 0x7f3fa40d1b80 2024-04-28T13:31:27.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.097+0000 7f3f9b400640 1 -- 172.21.15.196:0/562795743 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174143 (secure 0 0 0) 0x7f3f9c059620 con 0x7f3fa40d1b80 2024-04-28T13:31:27.099 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:27.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.102+0000 7f3faa400640 1 -- 172.21.15.196:0/562795743 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3f8406c8f0 msgr2=0x7f3f8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:27.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.102+0000 7f3faa400640 1 --2- 172.21.15.196:0/562795743 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3f8406c8f0 0x7f3f8406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f94000f60 tx=0x7f3f94007c00 comp rx=0 tx=0).stop 2024-04-28T13:31:27.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.102+0000 7f3faa400640 1 -- 172.21.15.196:0/562795743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40d1b80 msgr2=0x7f3fa40a1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:27.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.102+0000 7f3faa400640 1 --2- 172.21.15.196:0/562795743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40d1b80 0x7f3fa40a1bd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00b970 tx=0x7f3f9c00be30 comp rx=0 tx=0).stop 2024-04-28T13:31:27.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.102+0000 7f3faa400640 1 -- 172.21.15.196:0/562795743 shutdown_connections 2024-04-28T13:31:27.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.102+0000 7f3faa400640 1 --2- 172.21.15.196:0/562795743 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3f8406c8f0 0x7f3f8406edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.102+0000 7f3faa400640 1 --2- 172.21.15.196:0/562795743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40d1b80 0x7f3fa40a1bd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.102+0000 7f3faa400640 1 --2- 172.21.15.196:0/562795743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fa40cfda0 0x7f3fa41522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.102+0000 7f3faa400640 1 --2- 172.21.15.196:0/562795743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fa40b60a0 0x7f3fa414e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.102+0000 7f3faa400640 1 -- 172.21.15.196:0/562795743 >> 172.21.15.196:0/562795743 conn(0x7f3fa409e540 msgr2=0x7f3fa40cbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:27.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.102+0000 7f3faa400640 1 -- 172.21.15.196:0/562795743 shutdown_connections 2024-04-28T13:31:27.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.102+0000 7f3faa400640 1 -- 172.21.15.196:0/562795743 wait complete. 2024-04-28T13:31:27.121 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3172262857},{"type":"v1","addr":"172.21.15.203:6833","nonce":3172262857}]},"active_addr":"172.21.15.203:6833/3172262857","active_change":"2024-04-28T13:31:02.157575+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":145642823}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":2525443502}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":1040786544}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":3224897856}]}]} 2024-04-28T13:31:27.124 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-28T13:31:27.125 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-28T13:31:27.125 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:27.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.319+0000 7f1d47400640 1 Processor -- start 2024-04-28T13:31:27.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.320+0000 7f1d47400640 1 -- start start 2024-04-28T13:31:27.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.321+0000 7f1d47400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1d40145870 0x7f1d4006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:27.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.321+0000 7f1d47400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d4006f980 0x7f1d40072d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:27.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.321+0000 7f1d47400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d40073a40 con 0x7f1d40073250 2024-04-28T13:31:27.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.321+0000 7f1d47400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1d4006fde0 con 0x7f1d40145870 2024-04-28T13:31:27.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.321+0000 7f1d47400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4015d8d0 con 0x7f1d4006f980 2024-04-28T13:31:27.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.321+0000 7f1d44a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1d40145870 0x7f1d4006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:27.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.321+0000 7f1d44a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1d40145870 0x7f1d4006f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35560/0 (socket says 172.21.15.196:35560) 2024-04-28T13:31:27.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.321+0000 7f1d44a00640 1 -- 172.21.15.196:0/298817543 learned_addr learned my addr 172.21.15.196:0/298817543 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:27.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.322+0000 7f1d37e00640 1 --2- 172.21.15.196:0/298817543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d4006f980 0x7f1d40072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:27.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.322+0000 7f1d44a00640 1 -- 172.21.15.196:0/298817543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d4006f980 msgr2=0x7f1d40072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:27.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.322+0000 7f1d44a00640 1 --2- 172.21.15.196:0/298817543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d4006f980 0x7f1d40072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.322+0000 7f1d44a00640 1 -- 172.21.15.196:0/298817543 >> v1:172.21.15.203:6789/0 conn(0x7f1d40073250 legacy=0x7f1d4015fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:27.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.322+0000 7f1d44a00640 1 -- 172.21.15.196:0/298817543 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4015dbb0 con 0x7f1d40145870 2024-04-28T13:31:27.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.323+0000 7f1d44a00640 1 --2- 172.21.15.196:0/298817543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40145870 0x7f1d4006f440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009870 tx=0x7f1d3002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8e596b02e84bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:27.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.323+0000 7f1d3fe00640 1 -- 172.21.15.196:0/298817543 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30037070 con 0x7f1d40145870 2024-04-28T13:31:27.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.323+0000 7f1d3fe00640 1 -- 172.21.15.196:0/298817543 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d30032440 con 0x7f1d40145870 2024-04-28T13:31:27.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.323+0000 7f1d3fe00640 1 -- 172.21.15.196:0/298817543 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30032810 con 0x7f1d40145870 2024-04-28T13:31:27.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.324+0000 7f1d47400640 1 -- 172.21.15.196:0/298817543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40145870 msgr2=0x7f1d4006f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:27.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.324+0000 7f1d47400640 1 --2- 172.21.15.196:0/298817543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40145870 0x7f1d4006f440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009870 tx=0x7f1d3002a210 comp rx=0 tx=0).stop 2024-04-28T13:31:27.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.324+0000 7f1d47400640 1 -- 172.21.15.196:0/298817543 shutdown_connections 2024-04-28T13:31:27.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.324+0000 7f1d47400640 1 --2- 172.21.15.196:0/298817543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d4006f980 0x7f1d40072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.324+0000 7f1d47400640 1 --2- 172.21.15.196:0/298817543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40145870 0x7f1d4006f440 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.324+0000 7f1d47400640 1 -- 172.21.15.196:0/298817543 >> 172.21.15.196:0/298817543 conn(0x7f1d400cd830 msgr2=0x7f1d400cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:27.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.324+0000 7f1d47400640 1 -- 172.21.15.196:0/298817543 shutdown_connections 2024-04-28T13:31:27.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.324+0000 7f1d47400640 1 -- 172.21.15.196:0/298817543 wait complete. 2024-04-28T13:31:27.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.326+0000 7f1d47400640 1 Processor -- start 2024-04-28T13:31:27.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.326+0000 7f1d47400640 1 -- start start 2024-04-28T13:31:27.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.327+0000 7f1d47400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d4006f980 0x7f1d400afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:27.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.327+0000 7f1d47400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d40073250 0x7f1d401535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:27.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.327+0000 7f1d47400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40145870 0x7f1d40158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:27.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.327+0000 7f1d47400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d40145c50 con 0x7f1d40145870 2024-04-28T13:31:27.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.327+0000 7f1d47400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4015d5d0 con 0x7f1d4006f980 2024-04-28T13:31:27.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.327+0000 7f1d47400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1d400736e0 con 0x7f1d40073250 2024-04-28T13:31:27.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.327+0000 7f1d45400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40145870 0x7f1d40158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:27.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.327+0000 7f1d45400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40145870 0x7f1d40158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35564/0 (socket says 172.21.15.196:35564) 2024-04-28T13:31:27.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.327+0000 7f1d45400640 1 -- 172.21.15.196:0/215688693 learned_addr learned my addr 172.21.15.196:0/215688693 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:27.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.328+0000 7f1d44a00640 1 --2- 172.21.15.196:0/215688693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d4006f980 0x7f1d400afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:27.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.328+0000 7f1d37e00640 1 --2- 172.21.15.196:0/215688693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d40073250 0x7f1d401535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:27.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.328+0000 7f1d45400640 1 -- 172.21.15.196:0/215688693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d40073250 msgr2=0x7f1d401535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:27.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.328+0000 7f1d45400640 1 --2- 172.21.15.196:0/215688693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d40073250 0x7f1d401535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.328+0000 7f1d45400640 1 -- 172.21.15.196:0/215688693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d4006f980 msgr2=0x7f1d400afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:27.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.328+0000 7f1d45400640 1 --2- 172.21.15.196:0/215688693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d4006f980 0x7f1d400afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.328+0000 7f1d45400640 1 -- 172.21.15.196:0/215688693 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d40155f50 con 0x7f1d40145870 2024-04-28T13:31:27.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.328+0000 7f1d45400640 1 --2- 172.21.15.196:0/215688693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40145870 0x7f1d40158dd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800c780 tx=0x7f1d3800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:27.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.328+0000 7f1d3d600640 1 -- 172.21.15.196:0/215688693 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38004b10 con 0x7f1d40145870 2024-04-28T13:31:27.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.328+0000 7f1d3d600640 1 -- 172.21.15.196:0/215688693 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d38004cb0 con 0x7f1d40145870 2024-04-28T13:31:27.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.328+0000 7f1d3d600640 1 -- 172.21.15.196:0/215688693 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800aa90 con 0x7f1d40145870 2024-04-28T13:31:27.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.328+0000 7f1d47400640 1 -- 172.21.15.196:0/215688693 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d40156230 con 0x7f1d40145870 2024-04-28T13:31:27.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.328+0000 7f1d47400640 1 -- 172.21.15.196:0/215688693 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d400b01a0 con 0x7f1d40145870 2024-04-28T13:31:27.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.329+0000 7f1d47400640 1 -- 172.21.15.196:0/215688693 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d4006a290 con 0x7f1d40145870 2024-04-28T13:31:27.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.330+0000 7f1d3d600640 1 -- 172.21.15.196:0/215688693 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d3800ac30 con 0x7f1d40145870 2024-04-28T13:31:27.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.331+0000 7f1d3d600640 1 --2- 172.21.15.196:0/215688693 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1d1c06c8f0 0x7f1d1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:27.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.331+0000 7f1d3d600640 1 -- 172.21.15.196:0/215688693 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1d3808bd80 con 0x7f1d40145870 2024-04-28T13:31:27.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.332+0000 7f1d44a00640 1 --2- 172.21.15.196:0/215688693 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1d1c06c8f0 0x7f1d1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:27.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.333+0000 7f1d44a00640 1 --2- 172.21.15.196:0/215688693 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1d1c06c8f0 0x7f1d1c06edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1d30034000 tx=0x7f1d3002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:27.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.336+0000 7f1d3d600640 1 -- 172.21.15.196:0/215688693 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d38020070 con 0x7f1d40145870 2024-04-28T13:31:27.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.534+0000 7f1d47400640 1 -- 172.21.15.196:0/215688693 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d400b6a10 con 0x7f1d40145870 2024-04-28T13:31:27.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.536+0000 7f1d3d600640 1 -- 172.21.15.196:0/215688693 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f1d3805a1a0 con 0x7f1d40145870 2024-04-28T13:31:27.536 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:27.537 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":20,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:22.624564+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:27.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.539+0000 7f1d47400640 1 -- 172.21.15.196:0/215688693 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1d1c06c8f0 msgr2=0x7f1d1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:27.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.539+0000 7f1d47400640 1 --2- 172.21.15.196:0/215688693 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1d1c06c8f0 0x7f1d1c06edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1d30034000 tx=0x7f1d3002e000 comp rx=0 tx=0).stop 2024-04-28T13:31:27.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.539+0000 7f1d47400640 1 -- 172.21.15.196:0/215688693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40145870 msgr2=0x7f1d40158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:27.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.539+0000 7f1d47400640 1 --2- 172.21.15.196:0/215688693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40145870 0x7f1d40158dd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800c780 tx=0x7f1d3800cc40 comp rx=0 tx=0).stop 2024-04-28T13:31:27.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.539+0000 7f1d47400640 1 -- 172.21.15.196:0/215688693 shutdown_connections 2024-04-28T13:31:27.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.539+0000 7f1d47400640 1 --2- 172.21.15.196:0/215688693 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1d1c06c8f0 0x7f1d1c06edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.539+0000 7f1d47400640 1 --2- 172.21.15.196:0/215688693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40145870 0x7f1d40158dd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.539+0000 7f1d47400640 1 --2- 172.21.15.196:0/215688693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d40073250 0x7f1d401535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.539+0000 7f1d47400640 1 --2- 172.21.15.196:0/215688693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d4006f980 0x7f1d400afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.539+0000 7f1d47400640 1 -- 172.21.15.196:0/215688693 >> 172.21.15.196:0/215688693 conn(0x7f1d400cd830 msgr2=0x7f1d400b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:27.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.539+0000 7f1d47400640 1 -- 172.21.15.196:0/215688693 shutdown_connections 2024-04-28T13:31:27.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.539+0000 7f1d47400640 1 -- 172.21.15.196:0/215688693 wait complete. 2024-04-28T13:31:27.557 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-28T13:31:27.558 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:27.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.759+0000 7fef6c400640 1 Processor -- start 2024-04-28T13:31:27.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.759+0000 7fef6c400640 1 -- start start 2024-04-28T13:31:27.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.760+0000 7fef6c400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fef641469f0 0x7fef640ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:27.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.760+0000 7fef6c400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef64160f40 con 0x7fef640cd420 2024-04-28T13:31:27.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.761+0000 7fef6c400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef64161220 con 0x7fef640d1410 2024-04-28T13:31:27.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.761+0000 7fef6c400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fef640b5960 con 0x7fef641469f0 2024-04-28T13:31:27.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.761+0000 7fef69a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fef641469f0 0x7fef640ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:27.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.761+0000 7fef69a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fef641469f0 0x7fef640ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35574/0 (socket says 172.21.15.196:35574) 2024-04-28T13:31:27.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.761+0000 7fef69a00640 1 -- 172.21.15.196:0/4056692808 learned_addr learned my addr 172.21.15.196:0/4056692808 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:27.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.761+0000 7fef69000640 1 --1- 172.21.15.196:0/4056692808 >> v1:172.21.15.196:6789/0 conn(0x7fef640cd420 0x7fef640d0d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56008/0 (socket says 172.21.15.196:56008) 2024-04-28T13:31:27.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.761+0000 7fef69a00640 1 -- 172.21.15.196:0/4056692808 >> v1:172.21.15.203:6789/0 conn(0x7fef640d1410 legacy=0x7fef640b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:27.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.761+0000 7fef69a00640 1 -- 172.21.15.196:0/4056692808 >> v1:172.21.15.196:6789/0 conn(0x7fef640cd420 legacy=0x7fef640d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:27.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.761+0000 7fef69a00640 1 -- 172.21.15.196:0/4056692808 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef64161500 con 0x7fef641469f0 2024-04-28T13:31:27.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.762+0000 7fef69a00640 1 --2- 172.21.15.196:0/4056692808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef641469f0 0x7fef640ccee0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fef58009870 tx=0x7fef5802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2613833b7dcc0c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:27.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.763+0000 7fef63e00640 1 -- 172.21.15.196:0/4056692808 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef58037070 con 0x7fef641469f0 2024-04-28T13:31:27.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.763+0000 7fef63e00640 1 -- 172.21.15.196:0/4056692808 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef58032440 con 0x7fef641469f0 2024-04-28T13:31:27.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.763+0000 7fef63e00640 1 -- 172.21.15.196:0/4056692808 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef58032810 con 0x7fef641469f0 2024-04-28T13:31:27.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.763+0000 7fef6c400640 1 -- 172.21.15.196:0/4056692808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef641469f0 msgr2=0x7fef640ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:27.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.763+0000 7fef6c400640 1 --2- 172.21.15.196:0/4056692808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef641469f0 0x7fef640ccee0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fef58009870 tx=0x7fef5802a2a0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.763+0000 7fef6c400640 1 -- 172.21.15.196:0/4056692808 shutdown_connections 2024-04-28T13:31:27.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.763+0000 7fef6c400640 1 --2- 172.21.15.196:0/4056692808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef641469f0 0x7fef640ccee0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.763+0000 7fef6c400640 1 -- 172.21.15.196:0/4056692808 >> 172.21.15.196:0/4056692808 conn(0x7fef640cd830 msgr2=0x7fef640cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:27.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.763+0000 7fef6c400640 1 -- 172.21.15.196:0/4056692808 shutdown_connections 2024-04-28T13:31:27.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.763+0000 7fef6c400640 1 -- 172.21.15.196:0/4056692808 wait complete. 2024-04-28T13:31:27.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.765+0000 7fef6c400640 1 Processor -- start 2024-04-28T13:31:27.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.765+0000 7fef6c400640 1 -- start start 2024-04-28T13:31:27.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.766+0000 7fef6c400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef640cd420 0x7fef6413cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:27.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.766+0000 7fef6c400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef640d1410 0x7fef6413eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:27.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.766+0000 7fef6c400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef641469f0 0x7fef64144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:27.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.766+0000 7fef6c400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef64072020 con 0x7fef641469f0 2024-04-28T13:31:27.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.766+0000 7fef6c400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef64161ac0 con 0x7fef640cd420 2024-04-28T13:31:27.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.766+0000 7fef6c400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fef64160b80 con 0x7fef640d1410 2024-04-28T13:31:27.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.766+0000 7fef6a400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef641469f0 0x7fef64144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:27.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.766+0000 7fef6a400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef641469f0 0x7fef64144ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35584/0 (socket says 172.21.15.196:35584) 2024-04-28T13:31:27.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.766+0000 7fef6a400640 1 -- 172.21.15.196:0/4082439936 learned_addr learned my addr 172.21.15.196:0/4082439936 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:27.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.767+0000 7fef69a00640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef640cd420 0x7fef6413cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:27.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.767+0000 7fef69000640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef640d1410 0x7fef6413eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:27.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.767+0000 7fef6a400640 1 -- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef640d1410 msgr2=0x7fef6413eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:27.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.767+0000 7fef6a400640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef640d1410 0x7fef6413eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.767+0000 7fef6a400640 1 -- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef640cd420 msgr2=0x7fef6413cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:27.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.767+0000 7fef6a400640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef640cd420 0x7fef6413cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.767+0000 7fef6a400640 1 -- 172.21.15.196:0/4082439936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef64140980 con 0x7fef641469f0 2024-04-28T13:31:27.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.767+0000 7fef69000640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef640d1410 0x7fef6413eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:27.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.767+0000 7fef69a00640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef640cd420 0x7fef6413cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:31:27.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.767+0000 7fef6a400640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef641469f0 0x7fef64144ff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00c780 tx=0x7fef5c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:27.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.767+0000 7fef61600640 1 -- 172.21.15.196:0/4082439936 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c004b10 con 0x7fef641469f0 2024-04-28T13:31:27.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.768+0000 7fef61600640 1 -- 172.21.15.196:0/4082439936 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5c004cb0 con 0x7fef641469f0 2024-04-28T13:31:27.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.768+0000 7fef61600640 1 -- 172.21.15.196:0/4082439936 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00aa90 con 0x7fef641469f0 2024-04-28T13:31:27.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.768+0000 7fef6c400640 1 -- 172.21.15.196:0/4082439936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef64140c60 con 0x7fef641469f0 2024-04-28T13:31:27.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.768+0000 7fef6c400640 1 -- 172.21.15.196:0/4082439936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef64141220 con 0x7fef641469f0 2024-04-28T13:31:27.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.769+0000 7fef6c400640 1 -- 172.21.15.196:0/4082439936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef6406a290 con 0x7fef641469f0 2024-04-28T13:31:27.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.770+0000 7fef61600640 1 -- 172.21.15.196:0/4082439936 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef5c00ac30 con 0x7fef641469f0 2024-04-28T13:31:27.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.771+0000 7fef61600640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fef4406c8f0 0x7fef4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:27.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.771+0000 7fef61600640 1 -- 172.21.15.196:0/4082439936 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fef5c08bd30 con 0x7fef641469f0 2024-04-28T13:31:27.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.772+0000 7fef69a00640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fef4406c8f0 0x7fef4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:27.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.773+0000 7fef69a00640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fef4406c8f0 0x7fef4406edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fef58034000 tx=0x7fef5802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:27.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.776+0000 7fef61600640 1 -- 172.21.15.196:0/4082439936 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef5c020070 con 0x7fef641469f0 2024-04-28T13:31:27.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.961+0000 7fef6c400640 1 -- 172.21.15.196:0/4082439936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef641608b0 con 0x7fef641469f0 2024-04-28T13:31:27.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.962+0000 7fef61600640 1 -- 172.21.15.196:0/4082439936 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7fef5c05a160 con 0x7fef641469f0 2024-04-28T13:31:27.962 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:27.962 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":20,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:22.624564+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:27.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.964+0000 7fef6c400640 1 -- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fef4406c8f0 msgr2=0x7fef4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:27.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.964+0000 7fef6c400640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fef4406c8f0 0x7fef4406edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fef58034000 tx=0x7fef5802e000 comp rx=0 tx=0).stop 2024-04-28T13:31:27.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.964+0000 7fef6c400640 1 -- 172.21.15.196:0/4082439936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef641469f0 msgr2=0x7fef64144ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:27.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.964+0000 7fef6c400640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef641469f0 0x7fef64144ff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00c780 tx=0x7fef5c00cc40 comp rx=0 tx=0).stop 2024-04-28T13:31:27.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.964+0000 7fef6c400640 1 -- 172.21.15.196:0/4082439936 shutdown_connections 2024-04-28T13:31:27.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.964+0000 7fef6c400640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fef4406c8f0 0x7fef4406edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.964+0000 7fef6c400640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef641469f0 0x7fef64144ff0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.964+0000 7fef6c400640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef640d1410 0x7fef6413eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.964+0000 7fef6c400640 1 --2- 172.21.15.196:0/4082439936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef640cd420 0x7fef6413cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:27.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.964+0000 7fef6c400640 1 -- 172.21.15.196:0/4082439936 >> 172.21.15.196:0/4082439936 conn(0x7fef640cd830 msgr2=0x7fef640b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:27.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.965+0000 7fef6c400640 1 -- 172.21.15.196:0/4082439936 shutdown_connections 2024-04-28T13:31:27.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:27.965+0000 7fef6c400640 1 -- 172.21.15.196:0/4082439936 wait complete. 2024-04-28T13:31:27.983 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-28T13:31:27.984 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-28T13:31:27.984 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-28T13:31:27.985 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-28T13:31:27.985 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-28T13:31:27.985 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-28T13:31:27.986 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-28T13:31:27.986 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-28T13:31:28.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.272+0000 7fad2d800640 1 Processor -- start 2024-04-28T13:31:28.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.272+0000 7fad2d800640 1 -- start start 2024-04-28T13:31:28.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.274+0000 7fad2d800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fad281635d0 0x7fad280731f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.275+0000 7fad2d800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad280d17f0 0x7fad280add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.275+0000 7fad2d800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad281639b0 con 0x7fad28073730 2024-04-28T13:31:28.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.275+0000 7fad2d800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fad280724c0 con 0x7fad281635d0 2024-04-28T13:31:28.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.275+0000 7fad2d800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fad280b5d70 con 0x7fad280d17f0 2024-04-28T13:31:28.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.275+0000 7fad26000640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fad28073730 0x7fad280d10e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.196:36394/0 (socket says 172.21.15.196:36394) 2024-04-28T13:31:28.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.276+0000 7fad26000640 1 -- 172.21.15.196:0/4152373550 learned_addr learned my addr 172.21.15.196:0/4152373550 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:28.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.276+0000 7fad26a00640 1 --2- 172.21.15.196:0/4152373550 >> v2:172.21.15.196:3300/0 conn(0x7fad281635d0 0x7fad280731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.277+0000 7fad26a00640 1 -- 172.21.15.196:0/4152373550 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad280d17f0 msgr2=0x7fad280add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.277+0000 7fad26a00640 1 --2- 172.21.15.196:0/4152373550 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad280d17f0 0x7fad280add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.277+0000 7fad26a00640 1 -- 172.21.15.196:0/4152373550 >> v1:172.21.15.203:6789/0 conn(0x7fad28073730 legacy=0x7fad280d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.277+0000 7fad26a00640 1 -- 172.21.15.196:0/4152373550 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad28072710 con 0x7fad281635d0 2024-04-28T13:31:28.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.278+0000 7fad26a00640 1 --2- 172.21.15.196:0/4152373550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad281635d0 0x7fad280731f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fad140097a0 tx=0x7fad1402a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf331653893493a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.279+0000 7fad25600640 1 -- 172.21.15.196:0/4152373550 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14037070 con 0x7fad281635d0 2024-04-28T13:31:28.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.279+0000 7fad25600640 1 -- 172.21.15.196:0/4152373550 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad1402adf0 con 0x7fad281635d0 2024-04-28T13:31:28.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.279+0000 7fad25600640 1 -- 172.21.15.196:0/4152373550 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14032640 con 0x7fad281635d0 2024-04-28T13:31:28.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.279+0000 7f849a200640 1 Processor -- start 2024-04-28T13:31:28.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.280+0000 7fad2d800640 1 -- 172.21.15.196:0/4152373550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad281635d0 msgr2=0x7fad280731f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.280+0000 7fad2d800640 1 --2- 172.21.15.196:0/4152373550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad281635d0 0x7fad280731f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fad140097a0 tx=0x7fad1402a0a0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.280+0000 7f849a200640 1 -- start start 2024-04-28T13:31:28.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.281+0000 7fad2d800640 1 -- 172.21.15.196:0/4152373550 shutdown_connections 2024-04-28T13:31:28.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.281+0000 7fad2d800640 1 --2- 172.21.15.196:0/4152373550 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad280d17f0 0x7fad280add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.281+0000 7fad2d800640 1 --2- 172.21.15.196:0/4152373550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad281635d0 0x7fad280731f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.281+0000 7fad2d800640 1 -- 172.21.15.196:0/4152373550 >> 172.21.15.196:0/4152373550 conn(0x7fad2806c6f0 msgr2=0x7fad2806cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.281+0000 7f6ad0a00640 1 Processor -- start 2024-04-28T13:31:28.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.281+0000 7fad2d800640 1 -- 172.21.15.196:0/4152373550 shutdown_connections 2024-04-28T13:31:28.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.281+0000 7fad2d800640 1 -- 172.21.15.196:0/4152373550 wait complete. 2024-04-28T13:31:28.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.284+0000 7fad2d800640 1 Processor -- start 2024-04-28T13:31:28.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.285+0000 7fad2d800640 1 -- start start 2024-04-28T13:31:28.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.282+0000 7f849a200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84940aed70 0x7f84940731f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.282+0000 7f849a200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84940b61e0 con 0x7f84940b5c90 2024-04-28T13:31:28.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.282+0000 7f849a200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8494072430 con 0x7f849415b0e0 2024-04-28T13:31:28.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.282+0000 7f849a200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f849406e5d0 con 0x7f84940aed70 2024-04-28T13:31:28.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.282+0000 7f8492a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f84940b5c90 0x7f84940ae660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56022/0 (socket says 172.21.15.196:56022) 2024-04-28T13:31:28.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.282+0000 7f8492a00640 1 -- 172.21.15.196:0/4166529893 learned_addr learned my addr 172.21.15.196:0/4166529893 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:28.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.283+0000 7f8493e00640 1 --2- 172.21.15.196:0/4166529893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84940aed70 0x7f84940731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.283+0000 7f8492000640 1 -- 172.21.15.196:0/4166529893 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2771753404 0 0) 0x7f84940b61e0 con 0x7f84940b5c90 2024-04-28T13:31:28.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.283+0000 7f8492000640 1 -- 172.21.15.196:0/4166529893 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8478007930 con 0x7f84940b5c90 2024-04-28T13:31:28.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.284+0000 7f8493e00640 1 -- 172.21.15.196:0/4166529893 >> v1:172.21.15.203:6789/0 conn(0x7f849415b0e0 legacy=0x7f849406dec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-28T13:31:28.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.284+0000 7f8493e00640 1 -- 172.21.15.196:0/4166529893 >> v1:172.21.15.196:6789/0 conn(0x7f84940b5c90 legacy=0x7f84940ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.284+0000 7f8493e00640 1 -- 172.21.15.196:0/4166529893 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8494072710 con 0x7f84940aed70 2024-04-28T13:31:28.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.281+0000 7f6ad0a00640 1 -- start start 2024-04-28T13:31:28.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.284+0000 7f6ad0a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6ac403c0a0 0x7f6ac40e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.284+0000 7f6ad0a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ac4053cb0 0x7f6ac4054140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.284+0000 7f6ad0a00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ac4054680 con 0x7f6ac40e3160 2024-04-28T13:31:28.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.284+0000 7f6ad0a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f6ac4007b70 con 0x7f6ac403c0a0 2024-04-28T13:31:28.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.284+0000 7f6ad0a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6ac4058810 con 0x7f6ac4053cb0 2024-04-28T13:31:28.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.285+0000 7f6acb400640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f6ac40e3160 0x7f6ac4058050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.196:36404/0 (socket says 172.21.15.196:36404) 2024-04-28T13:31:28.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.285+0000 7f6acbe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ac4053cb0 0x7f6ac4054140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.285+0000 7f6acbe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ac4053cb0 0x7f6ac4054140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:44666/0 (socket says 172.21.15.196:44666) 2024-04-28T13:31:28.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.285+0000 7f6acbe00640 1 -- 172.21.15.196:0/3108271108 learned_addr learned my addr 172.21.15.196:0/3108271108 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.285+0000 7f6acaa00640 1 --2- 172.21.15.196:0/3108271108 >> v2:172.21.15.203:3300/0 conn(0x7f6ac403c0a0 0x7f6ac40e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.285+0000 7fe45c200640 1 Processor -- start 2024-04-28T13:31:28.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.285+0000 7fe45c200640 1 -- start start 2024-04-28T13:31:28.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.286+0000 7f8493e00640 1 --2- 172.21.15.196:0/4166529893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84940aed70 0x7f84940731f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f848c00d210 tx=0x7f848c033000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6a15dfa5dc02907 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.286+0000 7f6aca000640 1 -- 172.21.15.196:0/3108271108 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3508103566 0 0) 0x7f6ac4054680 con 0x7f6ac40e3160 2024-04-28T13:31:28.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.286+0000 7f6aca000640 1 -- 172.21.15.196:0/3108271108 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ab4003520 con 0x7f6ac40e3160 2024-04-28T13:31:28.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.286+0000 7f8492000640 1 -- 172.21.15.196:0/4166529893 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c039070 con 0x7f84940aed70 2024-04-28T13:31:28.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.287+0000 7fad2d800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad280d17f0 0x7fad2809e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.287+0000 7f8492000640 1 -- 172.21.15.196:0/4166529893 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f848c033d10 con 0x7f84940aed70 2024-04-28T13:31:28.290 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.287+0000 7fad2d800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad281635d0 0x7fad2809ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.290 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.287+0000 7fad2d800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad28201ce0 0x7fad28202170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.290 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.287+0000 7fe45c200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe454073790 0x7fe454073c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.290 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.287+0000 7fe45c200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe45406e570 con 0x7fe45414ce30 2024-04-28T13:31:28.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.287+0000 7fe45c200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe45406e850 con 0x7fe4540b4b90 2024-04-28T13:31:28.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.287+0000 7fe45c200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fe45406e320 con 0x7fe454073790 2024-04-28T13:31:28.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.287+0000 7f849a200640 1 -- 172.21.15.196:0/4166529893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84940aed70 msgr2=0x7f84940731f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7f8492000640 1 -- 172.21.15.196:0/4166529893 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c039070 con 0x7f84940aed70 2024-04-28T13:31:28.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7f849a200640 1 --2- 172.21.15.196:0/4166529893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84940aed70 0x7f84940731f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f848c00d210 tx=0x7f848c033000 comp rx=0 tx=0).stop 2024-04-28T13:31:28.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7fe459800640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fe45414ce30 0x7fe454163c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56038/0 (socket says 172.21.15.196:56038) 2024-04-28T13:31:28.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7fe459800640 1 -- 172.21.15.196:0/809114213 learned_addr learned my addr 172.21.15.196:0/809114213 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:28.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7fad2d800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad280b59b0 con 0x7fad280d17f0 2024-04-28T13:31:28.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7f849a200640 1 -- 172.21.15.196:0/4166529893 shutdown_connections 2024-04-28T13:31:28.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7f849a200640 1 --2- 172.21.15.196:0/4166529893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84940aed70 0x7f84940731f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7f849a200640 1 -- 172.21.15.196:0/4166529893 >> 172.21.15.196:0/4166529893 conn(0x7f84940b2990 msgr2=0x7f84940b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.286+0000 7f6acaa00640 1 -- 172.21.15.196:0/3108271108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ac4053cb0 msgr2=0x7f6ac4054140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.286+0000 7f6acaa00640 1 --2- 172.21.15.196:0/3108271108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ac4053cb0 0x7f6ac4054140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.286+0000 7f6acaa00640 1 -- 172.21.15.196:0/3108271108 >> v1:172.21.15.203:6789/0 conn(0x7f6ac40e3160 legacy=0x7f6ac4058050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.286+0000 7f6acaa00640 1 -- 172.21.15.196:0/3108271108 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac4058af0 con 0x7f6ac403c0a0 2024-04-28T13:31:28.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7fad26000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad281635d0 0x7fad2809ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7f849a200640 1 -- 172.21.15.196:0/4166529893 shutdown_connections 2024-04-28T13:31:28.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7fad26000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad281635d0 0x7fad2809ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:39528/0 (socket says 172.21.15.196:39528) 2024-04-28T13:31:28.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7fad26000640 1 -- 172.21.15.196:0/3671383154 learned_addr learned my addr 172.21.15.196:0/3671383154 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7fad26a00640 1 --2- 172.21.15.196:0/3671383154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad280d17f0 0x7fad2809e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7fad2d800640 1 -- 172.21.15.196:0/3671383154 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fad28072cd0 con 0x7fad281635d0 2024-04-28T13:31:28.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7fad2d800640 1 -- 172.21.15.196:0/3671383154 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fad280b5b30 con 0x7fad28201ce0 2024-04-28T13:31:28.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7fe458e00640 1 --2- 172.21.15.196:0/809114213 >> v2:172.21.15.203:3300/0 conn(0x7fe454073790 0x7fe454073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7fad27400640 1 --2- 172.21.15.196:0/3671383154 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad28201ce0 0x7fad28202170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7fe453e00640 1 -- 172.21.15.196:0/809114213 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 79940642 0 0) 0x7fe45406e570 con 0x7fe45414ce30 2024-04-28T13:31:28.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fe453e00640 1 -- 172.21.15.196:0/809114213 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe43c003520 con 0x7fe45414ce30 2024-04-28T13:31:28.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.288+0000 7f849a200640 1 -- 172.21.15.196:0/4166529893 wait complete. 2024-04-28T13:31:28.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7f6acaa00640 1 --2- 172.21.15.196:0/3108271108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ac403c0a0 0x7f6ac40e7780 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80097a0 tx=0x7f6ab802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fd5873857ed5eea server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fad26a00640 1 -- 172.21.15.196:0/3671383154 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad28201ce0 msgr2=0x7fad28202170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fad26a00640 1 --2- 172.21.15.196:0/3671383154 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad28201ce0 0x7fad28202170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fad26a00640 1 -- 172.21.15.196:0/3671383154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad281635d0 msgr2=0x7fad2809ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fad26a00640 1 --2- 172.21.15.196:0/3671383154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad281635d0 0x7fad2809ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fad26a00640 1 -- 172.21.15.196:0/3671383154 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2809c390 con 0x7fad280d17f0 2024-04-28T13:31:28.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fe453e00640 1 -- 172.21.15.196:0/809114213 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 440484263 0 0) 0x7fe43c003520 con 0x7fe45414ce30 2024-04-28T13:31:28.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fe453e00640 1 -- 172.21.15.196:0/809114213 >> v2:172.21.15.203:3300/0 conn(0x7fe454073790 msgr2=0x7fe454073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fe453e00640 1 --2- 172.21.15.196:0/809114213 >> v2:172.21.15.203:3300/0 conn(0x7fe454073790 0x7fe454073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fe453e00640 1 -- 172.21.15.196:0/809114213 >> v1:172.21.15.203:6789/0 conn(0x7fe4540b4b90 legacy=0x7fe45406f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fe453e00640 1 -- 172.21.15.196:0/809114213 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45406eb30 con 0x7fe45414ce30 2024-04-28T13:31:28.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fe453e00640 1 -- 172.21.15.196:0/809114213 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fe44c001670 con 0x7fe45414ce30 2024-04-28T13:31:28.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fe453e00640 1 -- 172.21.15.196:0/809114213 >> v1:172.21.15.196:6789/0 conn(0x7fe45414ce30 legacy=0x7fe454163c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fe453e00640 1 --2- 172.21.15.196:0/809114213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe43c0039b0 0x7fe43c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fe453e00640 1 --2- 172.21.15.196:0/809114213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe43c004800 0x7fe43c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fe453e00640 1 --2- 172.21.15.196:0/809114213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe43c0051c0 0x7fe43c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fe453e00640 1 -- 172.21.15.196:0/809114213 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe45406e320 con 0x7fe43c004800 2024-04-28T13:31:28.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fe453e00640 1 -- 172.21.15.196:0/809114213 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c00c060 con 0x7fe43c0039b0 2024-04-28T13:31:28.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fe453e00640 1 -- 172.21.15.196:0/809114213 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe43c00c260 con 0x7fe43c0051c0 2024-04-28T13:31:28.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.290+0000 7f41dec00640 1 Processor -- start 2024-04-28T13:31:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.289+0000 7fad26a00640 1 --2- 172.21.15.196:0/3671383154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad280d17f0 0x7fad2809e140 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fad14031000 tx=0x7fad140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.290+0000 7fe458e00640 1 --2- 172.21.15.196:0/809114213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe43c004800 0x7fe43c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.290+0000 7fe459800640 1 --2- 172.21.15.196:0/809114213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe43c0039b0 0x7fe43c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.291+0000 7fe458e00640 1 -- 172.21.15.196:0/809114213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe43c0051c0 msgr2=0x7fe43c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.291+0000 7fe458e00640 1 --2- 172.21.15.196:0/809114213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe43c0051c0 0x7fe43c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.291+0000 7fe458e00640 1 -- 172.21.15.196:0/809114213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe43c0039b0 msgr2=0x7fe43c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.291+0000 7fe458e00640 1 --2- 172.21.15.196:0/809114213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe43c0039b0 0x7fe43c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.291+0000 7fe458e00640 1 -- 172.21.15.196:0/809114213 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45406eb30 con 0x7fe43c004800 2024-04-28T13:31:28.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.291+0000 7fe45a200640 1 -- 172.21.15.196:0/809114213 reap_dead start 2024-04-28T13:31:28.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.290+0000 7f41dec00640 1 -- start start 2024-04-28T13:31:28.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.291+0000 7f849a200640 1 Processor -- start 2024-04-28T13:31:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.291+0000 7fad1ea00640 1 -- 172.21.15.196:0/3671383154 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14037040 con 0x7fad280d17f0 2024-04-28T13:31:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.291+0000 7fad1ea00640 1 -- 172.21.15.196:0/3671383154 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad1403d470 con 0x7fad280d17f0 2024-04-28T13:31:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.291+0000 7fad1ea00640 1 -- 172.21.15.196:0/3671383154 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1403d840 con 0x7fad280d17f0 2024-04-28T13:31:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.291+0000 7f41dec00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f41d8162e10 0x7f41d80c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.291+0000 7f41dec00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d80d3e00 0x7f41d80d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.292+0000 7fad2d800640 1 -- 172.21.15.196:0/3671383154 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2809c670 con 0x7fad280d17f0 2024-04-28T13:31:28.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.292+0000 7f86fe000640 1 Processor -- start 2024-04-28T13:31:28.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.292+0000 7f86fe000640 1 -- start start 2024-04-28T13:31:28.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.293+0000 7fad2d800640 1 -- 172.21.15.196:0/3671383154 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad2809bf80 con 0x7fad280d17f0 2024-04-28T13:31:28.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.293+0000 7fe458e00640 1 --2- 172.21.15.196:0/809114213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe43c004800 0x7fe43c004c80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe448009fc0 tx=0x7fe43c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3b7ab624fd493a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.292+0000 7f41dec00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d80d3300 con 0x7f41d8162a30 2024-04-28T13:31:28.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.292+0000 7f41dec00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f41d80d6020 con 0x7f41d8162e10 2024-04-28T13:31:28.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.292+0000 7f41dec00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f41d80d3020 con 0x7f41d80d3e00 2024-04-28T13:31:28.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.292+0000 7f41dd800640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f41d8162a30 0x7f41d80d36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56046/0 (socket says 172.21.15.196:56046) 2024-04-28T13:31:28.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.292+0000 7f41dd800640 1 -- 172.21.15.196:0/2758214219 learned_addr learned my addr 172.21.15.196:0/2758214219 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:28.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.293+0000 7f41de200640 1 --2- 172.21.15.196:0/2758214219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d80d3e00 0x7f41d80d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.291+0000 7f849a200640 1 -- start start 2024-04-28T13:31:28.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.293+0000 7f849a200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84940aed70 0x7f8494148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.293+0000 7f849a200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84940b5c90 0x7f84941561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.293+0000 7f849a200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f849415b0e0 0x7f84941566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.293+0000 7f849a200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f849415b4c0 con 0x7f84940b5c90 2024-04-28T13:31:28.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.293+0000 7f849a200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8494072cd0 con 0x7f84940aed70 2024-04-28T13:31:28.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.293+0000 7f849a200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f84940af2e0 con 0x7f849415b0e0 2024-04-28T13:31:28.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.293+0000 7f41dce00640 1 --2- 172.21.15.196:0/2758214219 >> v2:172.21.15.196:3300/0 conn(0x7f41d8162e10 0x7f41d80c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7f8493e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f849415b0e0 0x7f84941566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7f6aca000640 1 -- 172.21.15.196:0/3108271108 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8037070 con 0x7f6ac403c0a0 2024-04-28T13:31:28.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7f6aca000640 1 -- 172.21.15.196:0/3108271108 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab802ae40 con 0x7f6ac403c0a0 2024-04-28T13:31:28.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7f6aca000640 1 -- 172.21.15.196:0/3108271108 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8032660 con 0x7f6ac403c0a0 2024-04-28T13:31:28.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7fe453e00640 1 -- 172.21.15.196:0/809114213 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448009c10 con 0x7fe43c004800 2024-04-28T13:31:28.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7fe453e00640 1 -- 172.21.15.196:0/809114213 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe448036070 con 0x7fe43c004800 2024-04-28T13:31:28.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7fe453e00640 1 -- 172.21.15.196:0/809114213 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448031670 con 0x7fe43c004800 2024-04-28T13:31:28.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7f86fe000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86f815f370 0x7f86f80cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7f86fe000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f86f815f750 0x7f86f80b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.295+0000 7f86fe000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86f80d03a0 con 0x7f86f80d37d0 2024-04-28T13:31:28.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.295+0000 7f86fe000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f86f80b4d10 con 0x7f86f815f750 2024-04-28T13:31:28.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.295+0000 7f86fe000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f86f80d0680 con 0x7f86f815f370 2024-04-28T13:31:28.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7f41d7e00640 1 -- 172.21.15.196:0/2758214219 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2809398211 0 0) 0x7f41d80d3300 con 0x7f41d8162a30 2024-04-28T13:31:28.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7f41d7e00640 1 -- 172.21.15.196:0/2758214219 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41c0003520 con 0x7f41d8162a30 2024-04-28T13:31:28.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7f41de200640 1 -- 172.21.15.196:0/2758214219 >> v2:172.21.15.196:3300/0 conn(0x7f41d8162e10 msgr2=0x7f41d80c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7f41de200640 1 --2- 172.21.15.196:0/2758214219 >> v2:172.21.15.196:3300/0 conn(0x7f41d8162e10 0x7f41d80c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7f41de200640 1 -- 172.21.15.196:0/2758214219 >> v1:172.21.15.196:6789/0 conn(0x7f41d8162a30 legacy=0x7f41d80d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7f41de200640 1 -- 172.21.15.196:0/2758214219 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d80d5d40 con 0x7f41d80d3e00 2024-04-28T13:31:28.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7f8493400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84940aed70 0x7f8494148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7f8492a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84940b5c90 0x7f84941561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.294+0000 7f8492a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84940b5c90 0x7f84941561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35632/0 (socket says 172.21.15.196:35632) 2024-04-28T13:31:28.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.295+0000 7f8492a00640 1 -- 172.21.15.196:0/1935790059 learned_addr learned my addr 172.21.15.196:0/1935790059 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.295+0000 7fe45c200640 1 -- 172.21.15.196:0/809114213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe43c004800 msgr2=0x7fe43c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.295+0000 7fe45c200640 1 --2- 172.21.15.196:0/809114213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe43c004800 0x7fe43c004c80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe448009fc0 tx=0x7fe43c005830 comp rx=0 tx=0).stop 2024-04-28T13:31:28.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7fe45c200640 1 -- 172.21.15.196:0/809114213 shutdown_connections 2024-04-28T13:31:28.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7fe45c200640 1 --2- 172.21.15.196:0/809114213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe43c004800 0x7fe43c004c80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7fe45c200640 1 -- 172.21.15.196:0/809114213 >> 172.21.15.196:0/809114213 conn(0x7fe4540d0d30 msgr2=0x7fe4540d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7f8492a00640 1 -- 172.21.15.196:0/1935790059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f849415b0e0 msgr2=0x7f84941566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7f8492a00640 1 --2- 172.21.15.196:0/1935790059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f849415b0e0 0x7f84941566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7f8492a00640 1 -- 172.21.15.196:0/1935790059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84940aed70 msgr2=0x7f8494148730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7f8492a00640 1 --2- 172.21.15.196:0/1935790059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84940aed70 0x7f8494148730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7f8492a00640 1 -- 172.21.15.196:0/1935790059 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849409bbe0 con 0x7f84940b5c90 2024-04-28T13:31:28.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7f8493400640 1 --2- 172.21.15.196:0/1935790059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84940aed70 0x7f8494148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:31:28.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7f8492a00640 1 --2- 172.21.15.196:0/1935790059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84940b5c90 0x7f84941561a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f848400c500 tx=0x7f848400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.295+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3108271108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ac403c0a0 msgr2=0x7f6ac40e7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.295+0000 7f6ad0a00640 1 --2- 172.21.15.196:0/3108271108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ac403c0a0 0x7f6ac40e7780 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80097a0 tx=0x7f6ab802a130 comp rx=0 tx=0).stop 2024-04-28T13:31:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3108271108 shutdown_connections 2024-04-28T13:31:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7f6ad0a00640 1 --2- 172.21.15.196:0/3108271108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ac4053cb0 0x7f6ac4054140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7f6ad0a00640 1 --2- 172.21.15.196:0/3108271108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ac403c0a0 0x7f6ac40e7780 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3108271108 >> 172.21.15.196:0/3108271108 conn(0x7f6ac4058ee0 msgr2=0x7f6ac40592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.297+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3108271108 shutdown_connections 2024-04-28T13:31:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7f86f6a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f86f815f750 0x7f86f80b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7f86f6a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f86f815f750 0x7f86f80b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35634/0 (socket says 172.21.15.196:35634) 2024-04-28T13:31:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7f86f6a00640 1 -- 172.21.15.196:0/1765611657 learned_addr learned my addr 172.21.15.196:0/1765611657 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7f86f7400640 1 --2- 172.21.15.196:0/1765611657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86f815f370 0x7f86f80cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.297+0000 7f86f6a00640 1 -- 172.21.15.196:0/1765611657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86f815f370 msgr2=0x7f86f80cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7fad2d800640 1 -- 172.21.15.196:0/3671383154 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fad2809cb90 con 0x7fad280d17f0 2024-04-28T13:31:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.296+0000 7fad1ea00640 1 -- 172.21.15.196:0/3671383154 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad1403d9e0 con 0x7fad280d17f0 2024-04-28T13:31:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.297+0000 7f848b400640 1 -- 172.21.15.196:0/1935790059 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484017070 con 0x7f84940b5c90 2024-04-28T13:31:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.297+0000 7f848b400640 1 -- 172.21.15.196:0/1935790059 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8484005ce0 con 0x7f84940b5c90 2024-04-28T13:31:28.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.297+0000 7f41de200640 1 --2- 172.21.15.196:0/2758214219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d80d3e00 0x7f41d80d03e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009d90 tx=0x7f41d002a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b29e229e8d8944f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.297+0000 7f41d7e00640 1 -- 172.21.15.196:0/2758214219 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0035020 con 0x7f41d80d3e00 2024-04-28T13:31:28.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.297+0000 7f41d7e00640 1 -- 172.21.15.196:0/2758214219 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d0007540 con 0x7f41d80d3e00 2024-04-28T13:31:28.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.297+0000 7fe45c200640 1 -- 172.21.15.196:0/809114213 shutdown_connections 2024-04-28T13:31:28.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.297+0000 7f41d7e00640 1 -- 172.21.15.196:0/2758214219 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc001670 con 0x7f41d80d3e00 2024-04-28T13:31:28.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.298+0000 7fad1ea00640 1 --2- 172.21.15.196:0/3671383154 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad0806c9c0 0x7fad0806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.298+0000 7fad1ea00640 1 -- 172.21.15.196:0/3671383154 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fad140ac370 con 0x7fad280d17f0 2024-04-28T13:31:28.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.298+0000 7fad1ea00640 1 --2- 172.21.15.196:0/3671383154 >> [v2:172.21.15.196:6822/2304700074,v1:172.21.15.196:6825/2304700074] conn(0x7fad08074be0 0x7fad08077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.298+0000 7fad1ea00640 1 -- 172.21.15.196:0/3671383154 --> [v2:172.21.15.196:6822/2304700074,v1:172.21.15.196:6825/2304700074] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fad14008eb0 con 0x7fad08074be0 2024-04-28T13:31:28.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.298+0000 7fad1ea00640 1 -- 172.21.15.196:0/3671383154 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fad140b3050 con 0x7fad280d17f0 2024-04-28T13:31:28.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.297+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3108271108 wait complete. 2024-04-28T13:31:28.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.298+0000 7f849a200640 1 -- 172.21.15.196:0/1935790059 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849409bec0 con 0x7f84940b5c90 2024-04-28T13:31:28.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.299+0000 7f848b400640 1 -- 172.21.15.196:0/1935790059 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484007330 con 0x7f84940b5c90 2024-04-28T13:31:28.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.297+0000 7f86f6a00640 1 --2- 172.21.15.196:0/1765611657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86f815f370 0x7f86f80cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.297+0000 7f86f6a00640 1 -- 172.21.15.196:0/1765611657 >> v1:172.21.15.196:6789/0 conn(0x7f86f80d37d0 legacy=0x7f86f80d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.297+0000 7f86f6a00640 1 -- 172.21.15.196:0/1765611657 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f80d0960 con 0x7f86f815f750 2024-04-28T13:31:28.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.299+0000 7fad27400640 1 --2- 172.21.15.196:0/3671383154 >> [v2:172.21.15.196:6822/2304700074,v1:172.21.15.196:6825/2304700074] conn(0x7fad08074be0 0x7fad08077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.299+0000 7f86f6a00640 1 --2- 172.21.15.196:0/1765611657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86f815f750 0x7f86f80b47d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009870 tx=0x7f86e402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f63e8c0f316b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.299+0000 7fad26000640 1 --2- 172.21.15.196:0/3671383154 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad0806c9c0 0x7fad0806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.299+0000 7f849a200640 1 -- 172.21.15.196:0/1935790059 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849409c480 con 0x7f84940b5c90 2024-04-28T13:31:28.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.299+0000 7f82ef200640 1 Processor -- start 2024-04-28T13:31:28.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.299+0000 7f82ef200640 1 -- start start 2024-04-28T13:31:28.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.300+0000 7f86f6000640 1 -- 172.21.15.196:0/1765611657 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4037070 con 0x7f86f815f750 2024-04-28T13:31:28.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.300+0000 7f86f6000640 1 -- 172.21.15.196:0/1765611657 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e402ad90 con 0x7f86f815f750 2024-04-28T13:31:28.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.300+0000 7f86f6000640 1 -- 172.21.15.196:0/1765611657 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4032640 con 0x7f86f815f750 2024-04-28T13:31:28.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.298+0000 7f41dec00640 1 -- 172.21.15.196:0/2758214219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d80d3e00 msgr2=0x7f41d80d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.298+0000 7f41dec00640 1 --2- 172.21.15.196:0/2758214219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d80d3e00 0x7f41d80d03e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009d90 tx=0x7f41d002a560 comp rx=0 tx=0).stop 2024-04-28T13:31:28.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.298+0000 7f41dec00640 1 -- 172.21.15.196:0/2758214219 shutdown_connections 2024-04-28T13:31:28.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.298+0000 7f41dec00640 1 --2- 172.21.15.196:0/2758214219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d80d3e00 0x7f41d80d03e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.318 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.298+0000 7f41dec00640 1 --2- 172.21.15.196:0/2758214219 >> v2:172.21.15.196:3300/0 conn(0x7f41d8162e10 0x7f41d80c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.318 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.299+0000 7f41dec00640 1 -- 172.21.15.196:0/2758214219 >> 172.21.15.196:0/2758214219 conn(0x7f41d80d5110 msgr2=0x7f41d80d5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.318 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.299+0000 7f41dec00640 1 -- 172.21.15.196:0/2758214219 shutdown_connections 2024-04-28T13:31:28.318 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.299+0000 7f41dec00640 1 -- 172.21.15.196:0/2758214219 wait complete. 2024-04-28T13:31:28.318 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.298+0000 7fe45c200640 1 -- 172.21.15.196:0/809114213 wait complete. 2024-04-28T13:31:28.318 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.300+0000 7fe45c200640 1 Processor -- start 2024-04-28T13:31:28.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.301+0000 7fe45c200640 1 -- start start 2024-04-28T13:31:28.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.301+0000 7f86fe000640 1 -- 172.21.15.196:0/1765611657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86f815f750 msgr2=0x7f86f80b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.301+0000 7f86fe000640 1 --2- 172.21.15.196:0/1765611657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86f815f750 0x7f86f80b47d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009870 tx=0x7f86e402a100 comp rx=0 tx=0).stop 2024-04-28T13:31:28.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.301+0000 7f41dec00640 1 Processor -- start 2024-04-28T13:31:28.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.302+0000 7f41dec00640 1 -- start start 2024-04-28T13:31:28.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.301+0000 7fad27400640 1 --2- 172.21.15.196:0/3671383154 >> [v2:172.21.15.196:6822/2304700074,v1:172.21.15.196:6825/2304700074] conn(0x7fad08074be0 0x7fad08077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.301+0000 7fad26000640 1 --2- 172.21.15.196:0/3671383154 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad0806c9c0 0x7fad0806ee80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fad2809d230 tx=0x7fad18007f10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.301+0000 7f82ef200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f82e80d3ee0 0x7f82e80cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.301+0000 7f82ef200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82e815f750 0x7f82e80b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.301+0000 7f82ef200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e80d03a0 con 0x7f82e815f370 2024-04-28T13:31:28.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.301+0000 7f82ef200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f82e80b4d10 con 0x7f82e80d3ee0 2024-04-28T13:31:28.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.301+0000 7f82ef200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f82e80d0680 con 0x7f82e815f750 2024-04-28T13:31:28.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.302+0000 7f82ed200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82e815f750 0x7f82e80b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.302+0000 7f82ed200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82e815f750 0x7f82e80b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:44710/0 (socket says 172.21.15.196:44710) 2024-04-28T13:31:28.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.302+0000 7f82ed200640 1 -- 172.21.15.196:0/1531132251 learned_addr learned my addr 172.21.15.196:0/1531132251 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.303+0000 7fad1ea00640 1 -- 172.21.15.196:0/3671383154 <== osd.1 v2:172.21.15.196:6822/2304700074 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fad14008eb0 con 0x7fad08074be0 2024-04-28T13:31:28.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.303+0000 7f41dec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d80d3e00 0x7f41d80ba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.302+0000 7f82ed200640 1 -- 172.21.15.196:0/1531132251 >> v2:172.21.15.196:3300/0 conn(0x7f82e80d3ee0 msgr2=0x7f82e80cfd70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T13:31:28.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.303+0000 7f41dec00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d8162a30 0x7f41d80a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.303+0000 7f41dec00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d80bae00 0x7f41d80ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.303+0000 7f41dec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41d80b6140 con 0x7f41d80d3e00 2024-04-28T13:31:28.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.303+0000 7f41dec00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f41d80b6800 con 0x7f41d8162a30 2024-04-28T13:31:28.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.303+0000 7f41dec00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f41d80b62c0 con 0x7f41d80bae00 2024-04-28T13:31:28.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7fa410800640 1 Processor -- start 2024-04-28T13:31:28.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7f41dd800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d80d3e00 0x7f41d80ba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7f41dce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d8162a30 0x7f41d80a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7f41dce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d8162a30 0x7f41d80a21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:39574/0 (socket says 172.21.15.196:39574) 2024-04-28T13:31:28.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7f41dce00640 1 -- 172.21.15.196:0/636379091 learned_addr learned my addr 172.21.15.196:0/636379091 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7fa410800640 1 -- start start 2024-04-28T13:31:28.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.303+0000 7fe45c200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe43c004800 0x7fe4541e9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7f41de200640 1 --2- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d80bae00 0x7f41d80ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7fe45c200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe454163020 0x7fe4541fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7fe45c200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4540ae8e0 0x7fe4540aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7fe45c200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe45414d210 con 0x7fe4540ae8e0 2024-04-28T13:31:28.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7fe45c200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe45406eea0 con 0x7fe43c004800 2024-04-28T13:31:28.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7fe45c200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe45414d390 con 0x7fe454163020 2024-04-28T13:31:28.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.302+0000 7f6ad0a00640 1 Processor -- start 2024-04-28T13:31:28.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.302+0000 7f6ad0a00640 1 -- start start 2024-04-28T13:31:28.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7f6ad0a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ac403c0a0 0x7f6ac40e6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7f6ad0a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ac4053cb0 0x7f6ac40d53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7f6ad0a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ac40e3160 0x7f6ac40e0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7f6ad0a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac40e3540 con 0x7f6ac40e3160 2024-04-28T13:31:28.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7f6ad0a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac4006640 con 0x7f6ac4053cb0 2024-04-28T13:31:28.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7fe45a200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4540ae8e0 0x7fe4540aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7fe459800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe43c004800 0x7fe4541e9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7fe458e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe454163020 0x7fe4541fd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7f6ad0a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6ac4006b60 con 0x7f6ac403c0a0 2024-04-28T13:31:28.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7fe459800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe43c004800 0x7fe4541e9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:39582/0 (socket says 172.21.15.196:39582) 2024-04-28T13:31:28.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.304+0000 7fe459800640 1 -- 172.21.15.196:0/2207551420 learned_addr learned my addr 172.21.15.196:0/2207551420 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7f8488c00640 1 -- 172.21.15.196:0/1935790059 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8494148c70 con 0x7f84940b5c90 2024-04-28T13:31:28.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7f6acaa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ac4053cb0 0x7f6ac40d53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7f6acaa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ac4053cb0 0x7f6ac40d53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:39594/0 (socket says 172.21.15.196:39594) 2024-04-28T13:31:28.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7f6acaa00640 1 -- 172.21.15.196:0/3116811413 learned_addr learned my addr 172.21.15.196:0/3116811413 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7f6acbe00640 1 --2- 172.21.15.196:0/3116811413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ac40e3160 0x7f6ac40e0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7f848b400640 1 -- 172.21.15.196:0/1935790059 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f848401a070 con 0x7f84940b5c90 2024-04-28T13:31:28.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7fe459800640 1 -- 172.21.15.196:0/2207551420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe454163020 msgr2=0x7fe4541fd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7fe459800640 1 --2- 172.21.15.196:0/2207551420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe454163020 0x7fe4541fd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7fe459800640 1 -- 172.21.15.196:0/2207551420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4540ae8e0 msgr2=0x7fe4540aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7f6acb400640 1 --2- 172.21.15.196:0/3116811413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ac403c0a0 0x7f6ac40e6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7f6acbe00640 1 -- 172.21.15.196:0/3116811413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ac403c0a0 msgr2=0x7f6ac40e6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7f6acbe00640 1 --2- 172.21.15.196:0/3116811413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ac403c0a0 0x7f6ac40e6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7f6acbe00640 1 -- 172.21.15.196:0/3116811413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ac4053cb0 msgr2=0x7f6ac40d53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7f86fe000640 1 -- 172.21.15.196:0/1765611657 shutdown_connections 2024-04-28T13:31:28.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7f6acbe00640 1 --2- 172.21.15.196:0/3116811413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ac4053cb0 0x7f6ac40d53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7f6acbe00640 1 -- 172.21.15.196:0/3116811413 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac40d58f0 con 0x7f6ac40e3160 2024-04-28T13:31:28.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7fa410800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4080d3320 0x7fa4080b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7fa410800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa4080d51a0 0x7fa4080d5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7fa410800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4080d2440 con 0x7fa40815f750 2024-04-28T13:31:28.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7fa410800640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa4080d0030 con 0x7fa4080d51a0 2024-04-28T13:31:28.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7fa410800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa4080d2720 con 0x7fa4080d3320 2024-04-28T13:31:28.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7f82e7400640 1 --2- 172.21.15.196:0/1531132251 >> v2:172.21.15.196:3300/0 conn(0x7f82e80d3ee0 0x7f82e80cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7fe459800640 1 --2- 172.21.15.196:0/2207551420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4540ae8e0 0x7fe4540aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7fe459800640 1 -- 172.21.15.196:0/2207551420 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4541e8ab0 con 0x7fe43c004800 2024-04-28T13:31:28.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7fe45a200640 1 --2- 172.21.15.196:0/2207551420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4540ae8e0 0x7fe4540aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:31:28.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7f86fe000640 1 --2- 172.21.15.196:0/1765611657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86f815f750 0x7f86f80b47d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7f86fe000640 1 --2- 172.21.15.196:0/1765611657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86f815f370 0x7f86f80cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.305+0000 7f86fe000640 1 -- 172.21.15.196:0/1765611657 >> 172.21.15.196:0/1765611657 conn(0x7f86f808ec10 msgr2=0x7f86f809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7f82ed200640 1 --2- 172.21.15.196:0/1531132251 >> v2:172.21.15.196:3300/0 conn(0x7f82e80d3ee0 0x7f82e80cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7fa40de00640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa40815f750 0x7fa4080d3b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.196:36430/0 (socket says 172.21.15.196:36430) 2024-04-28T13:31:28.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7fa40de00640 1 -- 172.21.15.196:0/3025333336 learned_addr learned my addr 172.21.15.196:0/3025333336 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:28.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7f86fe000640 1 -- 172.21.15.196:0/1765611657 shutdown_connections 2024-04-28T13:31:28.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7f86fe000640 1 -- 172.21.15.196:0/1765611657 wait complete. 2024-04-28T13:31:28.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7f848b400640 1 --2- 172.21.15.196:0/1935790059 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f847406c940 0x7f847406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7f848b400640 1 -- 172.21.15.196:0/1935790059 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f848408af00 con 0x7f84940b5c90 2024-04-28T13:31:28.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7fe459800640 1 --2- 172.21.15.196:0/2207551420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe43c004800 0x7fe4541e9b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe44c007370 tx=0x7fe44c007830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7fa40e800640 1 --2- 172.21.15.196:0/3025333336 >> v2:172.21.15.203:3300/0 conn(0x7fa4080d51a0 0x7fa4080d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7f848b400640 1 --2- 172.21.15.196:0/1935790059 >> [v2:172.21.15.196:6807/377093821,v1:172.21.15.196:6809/377093821] conn(0x7f8474074bc0 0x7f8474076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7f41de200640 1 -- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d8162a30 msgr2=0x7f41d80a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7f41de200640 1 --2- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d8162a30 0x7f41d80a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7f41de200640 1 -- 172.21.15.196:0/636379091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d80d3e00 msgr2=0x7f41d80ba8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7f848b400640 1 -- 172.21.15.196:0/1935790059 --> [v2:172.21.15.196:6807/377093821,v1:172.21.15.196:6809/377093821] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8474077690 con 0x7f8474074bc0 2024-04-28T13:31:28.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7fa40d400640 1 --2- 172.21.15.196:0/3025333336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4080d3320 0x7fa4080b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7f848b400640 1 -- 172.21.15.196:0/1935790059 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f848408d520 con 0x7f84940b5c90 2024-04-28T13:31:28.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7f8493e00640 1 --2- 172.21.15.196:0/1935790059 >> [v2:172.21.15.196:6807/377093821,v1:172.21.15.196:6809/377093821] conn(0x7f8474074bc0 0x7f8474076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7fe451600640 1 -- 172.21.15.196:0/2207551420 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c01a070 con 0x7fe43c004800 2024-04-28T13:31:28.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7fe451600640 1 -- 172.21.15.196:0/2207551420 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe44c00fa00 con 0x7fe43c004800 2024-04-28T13:31:28.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7fe451600640 1 -- 172.21.15.196:0/2207551420 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00fdd0 con 0x7fe43c004800 2024-04-28T13:31:28.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.308+0000 7fe45c200640 1 -- 172.21.15.196:0/2207551420 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4541e8d90 con 0x7fe43c004800 2024-04-28T13:31:28.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.308+0000 7fa40e800640 1 -- 172.21.15.196:0/3025333336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4080d3320 msgr2=0x7fa4080b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.308+0000 7f8493400640 1 --2- 172.21.15.196:0/1935790059 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f847406c940 0x7f847406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7f82ed200640 1 -- 172.21.15.196:0/1531132251 >> v1:172.21.15.203:6789/0 conn(0x7f82e815f370 legacy=0x7f82e80d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.306+0000 7f82ed200640 1 -- 172.21.15.196:0/1531132251 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e80d0960 con 0x7f82e815f750 2024-04-28T13:31:28.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.308+0000 7f82e7400640 1 --2- 172.21.15.196:0/1531132251 >> v2:172.21.15.196:3300/0 conn(0x7f82e80d3ee0 0x7f82e80cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:28.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7f41de200640 1 --2- 172.21.15.196:0/636379091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d80d3e00 0x7f41d80ba8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.308+0000 7f82ed200640 1 --2- 172.21.15.196:0/1531132251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82e815f750 0x7f82e80b47d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009870 tx=0x7f82d802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14cf2edb88de34f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.308+0000 7fa40e800640 1 --2- 172.21.15.196:0/3025333336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4080d3320 0x7fa4080b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.308+0000 7fa40e800640 1 -- 172.21.15.196:0/3025333336 >> v1:172.21.15.203:6789/0 conn(0x7fa40815f750 legacy=0x7fa4080d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.308+0000 7fa40e800640 1 -- 172.21.15.196:0/3025333336 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4080d2a00 con 0x7fa4080d51a0 2024-04-28T13:31:28.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.309+0000 7f6acbe00640 1 --2- 172.21.15.196:0/3116811413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ac40e3160 0x7f6ac40e0db0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6acc06aae0 tx=0x7f6acc069eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.309+0000 7f86fe000640 1 Processor -- start 2024-04-28T13:31:28.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.309+0000 7f82e6a00640 1 -- 172.21.15.196:0/1531132251 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d8037070 con 0x7f82e815f750 2024-04-28T13:31:28.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.309+0000 7f82e6a00640 1 -- 172.21.15.196:0/1531132251 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d8032440 con 0x7f82e815f750 2024-04-28T13:31:28.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.309+0000 7f82e6a00640 1 -- 172.21.15.196:0/1531132251 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d80327f0 con 0x7f82e815f750 2024-04-28T13:31:28.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.308+0000 7fe45c200640 1 -- 172.21.15.196:0/2207551420 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4541e9350 con 0x7fe43c004800 2024-04-28T13:31:28.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.309+0000 7fe45c200640 1 -- 172.21.15.196:0/2207551420 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe45406a290 con 0x7fe43c004800 2024-04-28T13:31:28.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.309+0000 7f8493e00640 1 --2- 172.21.15.196:0/1935790059 >> [v2:172.21.15.196:6807/377093821,v1:172.21.15.196:6809/377093821] conn(0x7f8474074bc0 0x7f8474076fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.309+0000 7f8493400640 1 --2- 172.21.15.196:0/1935790059 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f847406c940 0x7f847406ee00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8480006fd0 tx=0x7f8480008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.309+0000 7f86fe000640 1 -- start start 2024-04-28T13:31:28.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7f41de200640 1 -- 172.21.15.196:0/636379091 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d809f2a0 con 0x7f41d80bae00 2024-04-28T13:31:28.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.307+0000 7f41dd800640 1 --2- 172.21.15.196:0/636379091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d80d3e00 0x7f41d80ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:31:28.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.309+0000 7f41de200640 1 --2- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d80bae00 0x7f41d80ce3b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f41d0031000 tx=0x7f41d0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7f82ef200640 1 -- 172.21.15.196:0/1531132251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82e815f750 msgr2=0x7f82e80b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7f82ef200640 1 --2- 172.21.15.196:0/1531132251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82e815f750 0x7f82e80b47d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009870 tx=0x7f82d802a270 comp rx=0 tx=0).stop 2024-04-28T13:31:28.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.312+0000 7fe451600640 1 -- 172.21.15.196:0/2207551420 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe44c01d360 con 0x7fe43c004800 2024-04-28T13:31:28.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.311+0000 7f848b400640 1 -- 172.21.15.196:0/1935790059 <== osd.2 v2:172.21.15.196:6807/377093821 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8474077690 con 0x7f8474074bc0 2024-04-28T13:31:28.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7f41d5600640 1 -- 172.21.15.196:0/636379091 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d003f070 con 0x7f41d80bae00 2024-04-28T13:31:28.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7f41dec00640 1 -- 172.21.15.196:0/636379091 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d80a16e0 con 0x7f41d80bae00 2024-04-28T13:31:28.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7f41dec00640 1 -- 172.21.15.196:0/636379091 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d809f580 con 0x7f41d80bae00 2024-04-28T13:31:28.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7f41d5600640 1 -- 172.21.15.196:0/636379091 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d0004240 con 0x7f41d80bae00 2024-04-28T13:31:28.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7f41d5600640 1 -- 172.21.15.196:0/636379091 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0035030 con 0x7f41d80bae00 2024-04-28T13:31:28.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7f82ef200640 1 -- 172.21.15.196:0/1531132251 shutdown_connections 2024-04-28T13:31:28.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7f82ef200640 1 --2- 172.21.15.196:0/1531132251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82e815f750 0x7f82e80b47d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7f82ef200640 1 --2- 172.21.15.196:0/1531132251 >> v2:172.21.15.196:3300/0 conn(0x7f82e80d3ee0 0x7f82e80cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7f82ef200640 1 -- 172.21.15.196:0/1531132251 >> 172.21.15.196:0/1531132251 conn(0x7f82e808ec10 msgr2=0x7f82e809e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.311+0000 7f82ef200640 1 -- 172.21.15.196:0/1531132251 shutdown_connections 2024-04-28T13:31:28.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7fa40e800640 1 --2- 172.21.15.196:0/3025333336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4080d51a0 0x7fa4080d5630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa404009d90 tx=0x7fa40402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e9a23a518ddbf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.312+0000 7f41dec00640 1 -- 172.21.15.196:0/636379091 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f41b4001030 con 0x7f41d80bae00 2024-04-28T13:31:28.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.311+0000 7fa40ca00640 1 -- 172.21.15.196:0/3025333336 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404007740 con 0x7fa4080d51a0 2024-04-28T13:31:28.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.311+0000 7fa40ca00640 1 -- 172.21.15.196:0/3025333336 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4040078e0 con 0x7fa4080d51a0 2024-04-28T13:31:28.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.311+0000 7fa40ca00640 1 -- 172.21.15.196:0/3025333336 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404007cb0 con 0x7fa4080d51a0 2024-04-28T13:31:28.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.312+0000 7fa410800640 1 -- 172.21.15.196:0/3025333336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4080d51a0 msgr2=0x7fa4080d5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.312+0000 7fa410800640 1 --2- 172.21.15.196:0/3025333336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4080d51a0 0x7fa4080d5630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa404009d90 tx=0x7fa40402a660 comp rx=0 tx=0).stop 2024-04-28T13:31:28.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.312+0000 7fa410800640 1 -- 172.21.15.196:0/3025333336 shutdown_connections 2024-04-28T13:31:28.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.312+0000 7fa410800640 1 --2- 172.21.15.196:0/3025333336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4080d51a0 0x7fa4080d5630 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.312+0000 7fa410800640 1 --2- 172.21.15.196:0/3025333336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4080d3320 0x7fa4080b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.312+0000 7fa410800640 1 -- 172.21.15.196:0/3025333336 >> 172.21.15.196:0/3025333336 conn(0x7fa40808ec20 msgr2=0x7fa40809e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.312+0000 7fa410800640 1 -- 172.21.15.196:0/3025333336 shutdown_connections 2024-04-28T13:31:28.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.312+0000 7fa410800640 1 -- 172.21.15.196:0/3025333336 wait complete. 2024-04-28T13:31:28.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7f6abf400640 1 -- 172.21.15.196:0/3116811413 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc089070 con 0x7f6ac40e3160 2024-04-28T13:31:28.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.312+0000 7f6abf400640 1 -- 172.21.15.196:0/3116811413 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6acc084610 con 0x7f6ac40e3160 2024-04-28T13:31:28.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.312+0000 7f6abf400640 1 -- 172.21.15.196:0/3116811413 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc084a00 con 0x7f6ac40e3160 2024-04-28T13:31:28.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.313+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3116811413 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac40e5070 con 0x7f6ac40e3160 2024-04-28T13:31:28.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.313+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3116811413 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac40e5630 con 0x7f6ac40e3160 2024-04-28T13:31:28.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7f86fe000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86f80d37d0 0x7f86f81f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7f86fe000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86f815f370 0x7f86f80cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.310+0000 7f86fe000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86f815f750 0x7f86f80ba440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.311+0000 7f86fe000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86f80b51c0 con 0x7f86f80d37d0 2024-04-28T13:31:28.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.311+0000 7f86fe000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f86f80d0f20 con 0x7f86f815f750 2024-04-28T13:31:28.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.311+0000 7f86fe000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f86f80b5340 con 0x7f86f815f370 2024-04-28T13:31:28.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.311+0000 7f86f7400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86f80d37d0 0x7f86f81f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.311+0000 7f86f7400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86f80d37d0 0x7f86f81f5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35680/0 (socket says 172.21.15.196:35680) 2024-04-28T13:31:28.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.311+0000 7f86f7400640 1 -- 172.21.15.196:0/3108656208 learned_addr learned my addr 172.21.15.196:0/3108656208 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.313+0000 7f86f7400640 1 -- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86f815f370 msgr2=0x7f86f80cb5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-28T13:31:28.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.313+0000 7f86f7400640 1 --2- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86f815f370 0x7f86f80cb5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.313+0000 7f86f7400640 1 -- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86f815f750 msgr2=0x7f86f80ba440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.313+0000 7f86f7e00640 1 --2- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86f815f750 0x7f86f80ba440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.311+0000 7f82ef200640 1 -- 172.21.15.196:0/1531132251 wait complete. 2024-04-28T13:31:28.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.314+0000 7f82ef200640 1 Processor -- start 2024-04-28T13:31:28.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.314+0000 7fe451600640 1 --2- 172.21.15.196:0/2207551420 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe43806c920 0x7fe43806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.314+0000 7fe451600640 1 -- 172.21.15.196:0/2207551420 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe44c08bb20 con 0x7fe43c004800 2024-04-28T13:31:28.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.314+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3116811413 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6ac4005390 con 0x7f6ac40e3160 2024-04-28T13:31:28.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.314+0000 7f86f7400640 1 --2- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86f815f750 0x7f86f80ba440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.314+0000 7f86f7400640 1 -- 172.21.15.196:0/3108656208 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f80cbb30 con 0x7f86f80d37d0 2024-04-28T13:31:28.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.314+0000 7f86f7e00640 1 --2- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86f815f750 0x7f86f80ba440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:28.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.314+0000 7f82ef200640 1 -- start start 2024-04-28T13:31:28.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.315+0000 7fe451600640 1 --2- 172.21.15.196:0/2207551420 >> [v2:172.21.15.196:6800/3595094355,v1:172.21.15.196:6801/3595094355] conn(0x7fe438074ac0 0x7fe438076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.315+0000 7fe451600640 1 -- 172.21.15.196:0/2207551420 --> [v2:172.21.15.196:6800/3595094355,v1:172.21.15.196:6801/3595094355] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe438077590 con 0x7fe438074ac0 2024-04-28T13:31:28.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.315+0000 7fe451600640 1 -- 172.21.15.196:0/2207551420 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe44c08bed0 con 0x7fe43c004800 2024-04-28T13:31:28.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.315+0000 7fe458e00640 1 --2- 172.21.15.196:0/2207551420 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe43806c920 0x7fe43806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.315+0000 7fa410800640 1 Processor -- start 2024-04-28T13:31:28.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.315+0000 7f86f7400640 1 --2- 172.21.15.196:0/3108656208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86f80d37d0 0x7f86f81f5170 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f86e800e9b0 tx=0x7f86e800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.315+0000 7fa410800640 1 -- start start 2024-04-28T13:31:28.350 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.316+0000 7f82ef200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82e80d3ee0 0x7f82e8160c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.350 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.316+0000 7fa410800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4080d3320 0x7fa4080c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.350 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.316+0000 7fa410800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4080d51a0 0x7fa408160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.350 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.316+0000 7fa410800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa40815f750 0x7fa408154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.350 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f82ef200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82e815f370 0x7f82e81f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.351 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f86ef400640 1 -- 172.21.15.196:0/3108656208 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8017070 con 0x7f86f80d37d0 2024-04-28T13:31:28.351 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f86ef400640 1 -- 172.21.15.196:0/3108656208 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e800c9a0 con 0x7f86f80d37d0 2024-04-28T13:31:28.351 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f86ef400640 1 -- 172.21.15.196:0/3108656208 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e800cd90 con 0x7f86f80d37d0 2024-04-28T13:31:28.351 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7fa410800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4080d08d0 con 0x7fa40815f750 2024-04-28T13:31:28.351 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7fa410800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa4080d0750 con 0x7fa4080d3320 2024-04-28T13:31:28.351 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7fa410800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa4080d0a50 con 0x7fa4080d51a0 2024-04-28T13:31:28.352 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f82ef200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82e815f750 0x7f82e80b9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.352 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f82e7e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82e80d3ee0 0x7f82e8160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.352 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f82e7e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82e80d3ee0 0x7f82e8160c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:44758/0 (socket says 172.21.15.196:44758) 2024-04-28T13:31:28.352 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f82ef200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82e80b51c0 con 0x7f82e815f750 2024-04-28T13:31:28.352 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f82e7e00640 1 -- 172.21.15.196:0/561753753 learned_addr learned my addr 172.21.15.196:0/561753753 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.352 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f82ef200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f82e80d0f20 con 0x7f82e815f370 2024-04-28T13:31:28.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f82ef200640 1 -- 172.21.15.196:0/561753753 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f82e80b5340 con 0x7f82e80d3ee0 2024-04-28T13:31:28.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f82e7400640 1 --2- 172.21.15.196:0/561753753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82e815f370 0x7f82e81f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7fe458e00640 1 --2- 172.21.15.196:0/2207551420 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe43806c920 0x7fe43806ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe448035000 tx=0x7fe44802f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.318+0000 7fe45a200640 1 --2- 172.21.15.196:0/2207551420 >> [v2:172.21.15.196:6800/3595094355,v1:172.21.15.196:6801/3595094355] conn(0x7fe438074ac0 0x7fe438076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f6abf400640 1 -- 172.21.15.196:0/3116811413 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6acc084ba0 con 0x7f6ac40e3160 2024-04-28T13:31:28.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.318+0000 7f6abf400640 1 --2- 172.21.15.196:0/3116811413 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6aac06c8f0 0x7f6aac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.318+0000 7f6abf400640 1 -- 172.21.15.196:0/3116811413 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6acc0c4560 con 0x7f6ac40e3160 2024-04-28T13:31:28.354 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f86fe000640 1 -- 172.21.15.196:0/3108656208 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f81f5850 con 0x7f86f80d37d0 2024-04-28T13:31:28.354 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f86fe000640 1 -- 172.21.15.196:0/3108656208 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f81f5e10 con 0x7f86f80d37d0 2024-04-28T13:31:28.354 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.316+0000 7f41d5600640 1 -- 172.21.15.196:0/636379091 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41d0042070 con 0x7f41d80bae00 2024-04-28T13:31:28.355 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.318+0000 7fa40e800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa40815f750 0x7fa408154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.355 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.318+0000 7fa40d400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4080d51a0 0x7fa408160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.355 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.318+0000 7fa40de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4080d3320 0x7fa4080c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.355 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.318+0000 7fa40de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4080d3320 0x7fa4080c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:39622/0 (socket says 172.21.15.196:39622) 2024-04-28T13:31:28.355 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.318+0000 7fa40de00640 1 -- 172.21.15.196:0/1110636039 learned_addr learned my addr 172.21.15.196:0/1110636039 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.355 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.318+0000 7fa40d400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4080d51a0 0x7fa408160540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:44760/0 (socket says 172.21.15.196:44760) 2024-04-28T13:31:28.355 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.319+0000 7f6abf400640 1 --2- 172.21.15.196:0/3116811413 >> [v2:172.21.15.196:6815/176565235,v1:172.21.15.196:6817/176565235] conn(0x7f6aac075770 0x7f6aac077c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.356 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.319+0000 7f41d5600640 1 --2- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f41b806c8f0 0x7f41b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.356 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.319+0000 7f41d5600640 1 -- 172.21.15.196:0/636379091 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f41d00ad080 con 0x7f41d80bae00 2024-04-28T13:31:28.356 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.319+0000 7fe45a200640 1 --2- 172.21.15.196:0/2207551420 >> [v2:172.21.15.196:6800/3595094355,v1:172.21.15.196:6801/3595094355] conn(0x7fe438074ac0 0x7fe438076ee0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.356 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.319+0000 7f6acb400640 1 --2- 172.21.15.196:0/3116811413 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6aac06c8f0 0x7f6aac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.319+0000 7f86ecc00640 1 -- 172.21.15.196:0/3108656208 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86cc001030 con 0x7f86f80d37d0 2024-04-28T13:31:28.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.319+0000 7f41d5600640 1 --2- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:6824/2058013113,v1:172.21.15.203:6825/2058013113] conn(0x7f41b8074b10 0x7f41b8076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.318+0000 7fa40e800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa40815f750 0x7fa408154650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35688/0 (socket says 172.21.15.196:35688) 2024-04-28T13:31:28.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.319+0000 7f41d5600640 1 -- 172.21.15.196:0/636379091 --> [v2:172.21.15.203:6824/2058013113,v1:172.21.15.203:6825/2058013113] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41b80775e0 con 0x7f41b8074b10 2024-04-28T13:31:28.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.319+0000 7f41d5600640 1 -- 172.21.15.196:0/636379091 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f41d00ad470 con 0x7f41d80bae00 2024-04-28T13:31:28.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.319+0000 7f6abf400640 1 -- 172.21.15.196:0/3116811413 --> [v2:172.21.15.196:6815/176565235,v1:172.21.15.196:6817/176565235] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6aac0782e0 con 0x7f6aac075770 2024-04-28T13:31:28.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.319+0000 7f6abf400640 1 -- 172.21.15.196:0/3116811413 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6acc0c2ef0 con 0x7f6ac40e3160 2024-04-28T13:31:28.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.319+0000 7f6acaa00640 1 --2- 172.21.15.196:0/3116811413 >> [v2:172.21.15.196:6815/176565235,v1:172.21.15.196:6817/176565235] conn(0x7f6aac075770 0x7f6aac077c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.317+0000 7f82ed200640 1 --2- 172.21.15.196:0/561753753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82e815f750 0x7f82e80b9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.318+0000 7f82ed200640 1 -- 172.21.15.196:0/561753753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82e80d3ee0 msgr2=0x7f82e8160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.318+0000 7f82ed200640 1 --2- 172.21.15.196:0/561753753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82e80d3ee0 0x7f82e8160c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.318+0000 7f82ed200640 1 -- 172.21.15.196:0/561753753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82e815f370 msgr2=0x7f82e81f24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.318+0000 7f82ed200640 1 --2- 172.21.15.196:0/561753753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82e815f370 0x7f82e81f24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.318+0000 7f82ed200640 1 -- 172.21.15.196:0/561753753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e81f29e0 con 0x7f82e815f750 2024-04-28T13:31:28.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.319+0000 7f82ed200640 1 --2- 172.21.15.196:0/561753753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82e815f750 0x7f82e80b9e00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f82d8031000 tx=0x7f82d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.320+0000 7fa40d400640 1 -- 172.21.15.196:0/1110636039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4080d3320 msgr2=0x7fa4080c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.320+0000 7fa40d400640 1 --2- 172.21.15.196:0/1110636039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4080d3320 0x7fa4080c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.320+0000 7fa40d400640 1 -- 172.21.15.196:0/1110636039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa40815f750 msgr2=0x7fa408154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.320+0000 7fa40d400640 1 --2- 172.21.15.196:0/1110636039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa40815f750 0x7fa408154650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.320+0000 7f41dce00640 1 --2- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:6824/2058013113,v1:172.21.15.203:6825/2058013113] conn(0x7f41b8074b10 0x7f41b8076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.320+0000 7f82dfe00640 1 -- 172.21.15.196:0/561753753 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d8037040 con 0x7f82e815f750 2024-04-28T13:31:28.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.320+0000 7f82dfe00640 1 -- 172.21.15.196:0/561753753 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d803fbf0 con 0x7f82e815f750 2024-04-28T13:31:28.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.320+0000 7f82dfe00640 1 -- 172.21.15.196:0/561753753 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d80325f0 con 0x7f82e815f750 2024-04-28T13:31:28.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.320+0000 7fa40d400640 1 -- 172.21.15.196:0/1110636039 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4080b2040 con 0x7fa4080d51a0 2024-04-28T13:31:28.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.320+0000 7fa40e800640 1 --2- 172.21.15.196:0/1110636039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa40815f750 0x7fa408154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:31:28.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.320+0000 7f6acaa00640 1 --2- 172.21.15.196:0/3116811413 >> [v2:172.21.15.196:6815/176565235,v1:172.21.15.196:6817/176565235] conn(0x7f6aac075770 0x7f6aac077c10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.320+0000 7fe451600640 1 -- 172.21.15.196:0/2207551420 <== osd.0 v2:172.21.15.196:6800/3595094355 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe438077590 con 0x7fe438074ac0 2024-04-28T13:31:28.362 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.321+0000 7f6acb400640 1 --2- 172.21.15.196:0/3116811413 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6aac06c8f0 0x7f6aac06edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0007fd0 tx=0x7f6ac0007be0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.362 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.321+0000 7f41dd800640 1 --2- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f41b806c8f0 0x7f41b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.362 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.321+0000 7fa40d400640 1 --2- 172.21.15.196:0/1110636039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4080d51a0 0x7fa408160540 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800e850 tx=0x7fa3f800ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.321+0000 7f41dce00640 1 --2- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:6824/2058013113,v1:172.21.15.203:6825/2058013113] conn(0x7f41b8074b10 0x7f41b8076f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.321+0000 7fa402000640 1 -- 172.21.15.196:0/1110636039 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8017070 con 0x7fa4080d51a0 2024-04-28T13:31:28.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.321+0000 7fa402000640 1 -- 172.21.15.196:0/1110636039 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f800c940 con 0x7fa4080d51a0 2024-04-28T13:31:28.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.321+0000 7fa402000640 1 -- 172.21.15.196:0/1110636039 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800cd10 con 0x7fa4080d51a0 2024-04-28T13:31:28.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.321+0000 7fa410800640 1 -- 172.21.15.196:0/1110636039 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4080b2320 con 0x7fa4080d51a0 2024-04-28T13:31:28.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.322+0000 7fa410800640 1 -- 172.21.15.196:0/1110636039 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4080b28e0 con 0x7fa4080d51a0 2024-04-28T13:31:28.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.322+0000 7f6abf400640 1 -- 172.21.15.196:0/3116811413 <== osd.3 v2:172.21.15.196:6815/176565235 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6aac0782e0 con 0x7f6aac075770 2024-04-28T13:31:28.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.321+0000 7f82ef200640 1 -- 172.21.15.196:0/561753753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e81f5160 con 0x7f82e815f750 2024-04-28T13:31:28.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.321+0000 7f82ef200640 1 -- 172.21.15.196:0/561753753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e81f5720 con 0x7f82e815f750 2024-04-28T13:31:28.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.322+0000 7f41dd800640 1 --2- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f41b806c8f0 0x7f41b806edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41d80a1610 tx=0x7f41cc001970 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.322+0000 7f82ef200640 1 -- 172.21.15.196:0/561753753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f82e81f5de0 con 0x7f82e815f750 2024-04-28T13:31:28.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.323+0000 7f86ef400640 1 -- 172.21.15.196:0/3108656208 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86e801a070 con 0x7f86f80d37d0 2024-04-28T13:31:28.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.324+0000 7f41d5600640 1 -- 172.21.15.196:0/636379091 <== osd.5 v2:172.21.15.203:6824/2058013113 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f41b80775e0 con 0x7f41b8074b10 2024-04-28T13:31:28.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.325+0000 7f86ef400640 1 --2- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f86d806c9c0 0x7f86d806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.325+0000 7f86ef400640 1 -- 172.21.15.196:0/3108656208 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f86e808bae0 con 0x7f86f80d37d0 2024-04-28T13:31:28.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.325+0000 7f86ef400640 1 --2- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:6800/1325487575,v1:172.21.15.203:6801/1325487575] conn(0x7f86d8074bc0 0x7f86d8076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.325+0000 7f86ef400640 1 -- 172.21.15.196:0/3108656208 --> [v2:172.21.15.203:6800/1325487575,v1:172.21.15.203:6801/1325487575] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86d8077690 con 0x7f86d8074bc0 2024-04-28T13:31:28.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.325+0000 7f86ef400640 1 -- 172.21.15.196:0/3108656208 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f86e8055b20 con 0x7f86f80d37d0 2024-04-28T13:31:28.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.323+0000 7fa410800640 1 -- 172.21.15.196:0/1110636039 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa4080693d0 con 0x7fa4080d51a0 2024-04-28T13:31:28.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.325+0000 7f82dfe00640 1 -- 172.21.15.196:0/561753753 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82d804a480 con 0x7f82e815f750 2024-04-28T13:31:28.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.325+0000 7fa402000640 1 -- 172.21.15.196:0/1110636039 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3f801a070 con 0x7fa4080d51a0 2024-04-28T13:31:28.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.326+0000 7f86f6a00640 1 --2- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f86d806c9c0 0x7f86d806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.326+0000 7f86f7e00640 1 --2- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:6800/1325487575,v1:172.21.15.203:6801/1325487575] conn(0x7f86d8074bc0 0x7f86d8076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.326+0000 7fa402000640 1 --2- 172.21.15.196:0/1110636039 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa3ec06c9c0 0x7fa3ec06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.327+0000 7fa402000640 1 -- 172.21.15.196:0/1110636039 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa3f808bd50 con 0x7fa4080d51a0 2024-04-28T13:31:28.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.327+0000 7fa402000640 1 --2- 172.21.15.196:0/1110636039 >> [v2:172.21.15.203:6802/2023229662,v1:172.21.15.203:6805/2023229662] conn(0x7fa3ec074c40 0x7fa3ec077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.327+0000 7fa402000640 1 -- 172.21.15.196:0/1110636039 --> [v2:172.21.15.203:6802/2023229662,v1:172.21.15.203:6805/2023229662] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3ec077710 con 0x7fa3ec074c40 2024-04-28T13:31:28.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.327+0000 7fa402000640 1 -- 172.21.15.196:0/1110636039 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa3f808c140 con 0x7fa4080d51a0 2024-04-28T13:31:28.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.327+0000 7fa40de00640 1 --2- 172.21.15.196:0/1110636039 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa3ec06c9c0 0x7fa3ec06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.327+0000 7f82dfe00640 1 --2- 172.21.15.196:0/561753753 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f82c406c9c0 0x7f82c406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.327+0000 7f82dfe00640 1 -- 172.21.15.196:0/561753753 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f82d80b4a00 con 0x7f82e815f750 2024-04-28T13:31:28.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.327+0000 7f82dfe00640 1 --2- 172.21.15.196:0/561753753 >> [v2:172.21.15.203:6804/2209636555,v1:172.21.15.203:6807/2209636555] conn(0x7f82c4074be0 0x7f82c4077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.327+0000 7f82dfe00640 1 -- 172.21.15.196:0/561753753 --> [v2:172.21.15.203:6804/2209636555,v1:172.21.15.203:6807/2209636555] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f82d8001d30 con 0x7f82c4074be0 2024-04-28T13:31:28.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.327+0000 7f82dfe00640 1 -- 172.21.15.196:0/561753753 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f82d80b4df0 con 0x7f82e815f750 2024-04-28T13:31:28.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.328+0000 7f82e7400640 1 --2- 172.21.15.196:0/561753753 >> [v2:172.21.15.203:6804/2209636555,v1:172.21.15.203:6807/2209636555] conn(0x7f82c4074be0 0x7f82c4077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.328+0000 7f82e7e00640 1 --2- 172.21.15.196:0/561753753 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f82c406c9c0 0x7f82c406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.328+0000 7fa40e800640 1 --2- 172.21.15.196:0/1110636039 >> [v2:172.21.15.203:6802/2023229662,v1:172.21.15.203:6805/2023229662] conn(0x7fa3ec074c40 0x7fa3ec077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.329+0000 7f86f7e00640 1 --2- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:6800/1325487575,v1:172.21.15.203:6801/1325487575] conn(0x7f86d8074bc0 0x7f86d8076fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.329+0000 7f86f6a00640 1 --2- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f86d806c9c0 0x7f86d806ee80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f86e4034000 tx=0x7f86e402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.329+0000 7fa40de00640 1 --2- 172.21.15.196:0/1110636039 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa3ec06c9c0 0x7fa3ec06ee80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc005880 tx=0x7fa3fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.329+0000 7f82e7400640 1 --2- 172.21.15.196:0/561753753 >> [v2:172.21.15.203:6804/2209636555,v1:172.21.15.203:6807/2209636555] conn(0x7f82c4074be0 0x7f82c4077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.330+0000 7f82e7e00640 1 --2- 172.21.15.196:0/561753753 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f82c406c9c0 0x7f82c406ee80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f82e0007fd0 tx=0x7f82e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.330+0000 7fa40e800640 1 --2- 172.21.15.196:0/1110636039 >> [v2:172.21.15.203:6802/2023229662,v1:172.21.15.203:6805/2023229662] conn(0x7fa3ec074c40 0x7fa3ec077060 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.331+0000 7f86ef400640 1 -- 172.21.15.196:0/3108656208 <== osd.4 v2:172.21.15.203:6800/1325487575 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86d8077690 con 0x7f86d8074bc0 2024-04-28T13:31:28.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.332+0000 7f82dfe00640 1 -- 172.21.15.196:0/561753753 <== osd.6 v2:172.21.15.203:6804/2209636555 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f82d8001d30 con 0x7f82c4074be0 2024-04-28T13:31:28.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.333+0000 7fa402000640 1 -- 172.21.15.196:0/1110636039 <== osd.7 v2:172.21.15.203:6802/2023229662 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa3ec077710 con 0x7fa3ec074c40 2024-04-28T13:31:28.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.342+0000 7f8488c00640 1 -- 172.21.15.196:0/1935790059 --> [v2:172.21.15.196:6807/377093821,v1:172.21.15.196:6809/377093821] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f849406a290 con 0x7f8474074bc0 2024-04-28T13:31:28.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.343+0000 7f848b400640 1 -- 172.21.15.196:0/1935790059 <== osd.2 v2:172.21.15.196:6807/377093821 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f849406a290 con 0x7f8474074bc0 2024-04-28T13:31:28.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.343+0000 7fad2d800640 1 -- 172.21.15.196:0/3671383154 --> [v2:172.21.15.196:6822/2304700074,v1:172.21.15.196:6825/2304700074] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fad2806a290 con 0x7fad08074be0 2024-04-28T13:31:28.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.344+0000 7fad1ea00640 1 -- 172.21.15.196:0/3671383154 <== osd.1 v2:172.21.15.196:6822/2304700074 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fad2806a290 con 0x7fad08074be0 2024-04-28T13:31:28.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.346+0000 7f8488c00640 1 -- 172.21.15.196:0/1935790059 >> [v2:172.21.15.196:6807/377093821,v1:172.21.15.196:6809/377093821] conn(0x7f8474074bc0 msgr2=0x7f8474076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.346+0000 7f8488c00640 1 --2- 172.21.15.196:0/1935790059 >> [v2:172.21.15.196:6807/377093821,v1:172.21.15.196:6809/377093821] conn(0x7f8474074bc0 0x7f8474076fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.346+0000 7f8488c00640 1 -- 172.21.15.196:0/1935790059 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f847406c940 msgr2=0x7f847406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.346+0000 7f8488c00640 1 --2- 172.21.15.196:0/1935790059 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f847406c940 0x7f847406ee00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8480006fd0 tx=0x7f8480008040 comp rx=0 tx=0).stop 2024-04-28T13:31:28.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.346+0000 7f8488c00640 1 -- 172.21.15.196:0/1935790059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84940b5c90 msgr2=0x7f84941561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.346+0000 7f8488c00640 1 --2- 172.21.15.196:0/1935790059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84940b5c90 0x7f84941561a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f848400c500 tx=0x7f848400c9c0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.346+0000 7f8493e00640 1 -- 172.21.15.196:0/1935790059 reap_dead start 2024-04-28T13:31:28.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.347+0000 7f8488c00640 1 -- 172.21.15.196:0/1935790059 shutdown_connections 2024-04-28T13:31:28.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.347+0000 7f8488c00640 1 -- 172.21.15.196:0/1935790059 >> 172.21.15.196:0/1935790059 conn(0x7f84940b2990 msgr2=0x7f849415efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.347+0000 7f8488c00640 1 -- 172.21.15.196:0/1935790059 shutdown_connections 2024-04-28T13:31:28.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.347+0000 7f8488c00640 1 -- 172.21.15.196:0/1935790059 wait complete. 2024-04-28T13:31:28.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.350+0000 7fad13e00640 1 -- 172.21.15.196:0/3671383154 >> [v2:172.21.15.196:6822/2304700074,v1:172.21.15.196:6825/2304700074] conn(0x7fad08074be0 msgr2=0x7fad08077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.350+0000 7fad13e00640 1 --2- 172.21.15.196:0/3671383154 >> [v2:172.21.15.196:6822/2304700074,v1:172.21.15.196:6825/2304700074] conn(0x7fad08074be0 0x7fad08077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.350+0000 7fad13e00640 1 -- 172.21.15.196:0/3671383154 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad0806c9c0 msgr2=0x7fad0806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.350+0000 7fad13e00640 1 --2- 172.21.15.196:0/3671383154 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad0806c9c0 0x7fad0806ee80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fad2809d230 tx=0x7fad18007f10 comp rx=0 tx=0).stop 2024-04-28T13:31:28.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.350+0000 7fad13e00640 1 -- 172.21.15.196:0/3671383154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad280d17f0 msgr2=0x7fad2809e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.350+0000 7fad13e00640 1 --2- 172.21.15.196:0/3671383154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad280d17f0 0x7fad2809e140 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fad14031000 tx=0x7fad140046c0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.350+0000 7fad27400640 1 -- 172.21.15.196:0/3671383154 reap_dead start 2024-04-28T13:31:28.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.352+0000 7fad13e00640 1 -- 172.21.15.196:0/3671383154 shutdown_connections 2024-04-28T13:31:28.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.352+0000 7fad13e00640 1 -- 172.21.15.196:0/3671383154 >> 172.21.15.196:0/3671383154 conn(0x7fad2806c6f0 msgr2=0x7fad280cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.352+0000 7fad13e00640 1 -- 172.21.15.196:0/3671383154 shutdown_connections 2024-04-28T13:31:28.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.352+0000 7fad13e00640 1 -- 172.21.15.196:0/3671383154 wait complete. 2024-04-28T13:31:28.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.353+0000 7f41dec00640 1 -- 172.21.15.196:0/636379091 --> [v2:172.21.15.203:6824/2058013113,v1:172.21.15.203:6825/2058013113] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f41b4002dd0 con 0x7f41b8074b10 2024-04-28T13:31:28.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.354+0000 7f41d5600640 1 -- 172.21.15.196:0/636379091 <== osd.5 v2:172.21.15.203:6824/2058013113 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f41b4002dd0 con 0x7f41b8074b10 2024-04-28T13:31:28.378 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.354+0000 7f41dec00640 1 -- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:6824/2058013113,v1:172.21.15.203:6825/2058013113] conn(0x7f41b8074b10 msgr2=0x7f41b8076f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.378 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.354+0000 7f41dec00640 1 --2- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:6824/2058013113,v1:172.21.15.203:6825/2058013113] conn(0x7f41b8074b10 0x7f41b8076f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.378 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.355+0000 7f41dec00640 1 -- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f41b806c8f0 msgr2=0x7f41b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.355+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3116811413 --> [v2:172.21.15.196:6815/176565235,v1:172.21.15.196:6817/176565235] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6ac4056bf0 con 0x7f6aac075770 2024-04-28T13:31:28.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.355+0000 7f41dec00640 1 --2- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f41b806c8f0 0x7f41b806edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41d80a1610 tx=0x7f41cc001970 comp rx=0 tx=0).stop 2024-04-28T13:31:28.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.355+0000 7f41dec00640 1 -- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d80bae00 msgr2=0x7f41d80ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.355+0000 7f41dec00640 1 --2- 172.21.15.196:0/636379091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d80bae00 0x7f41d80ce3b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f41d0031000 tx=0x7f41d0004a40 comp rx=0 tx=0).stop 2024-04-28T13:31:28.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.355+0000 7f41de200640 1 -- 172.21.15.196:0/636379091 reap_dead start 2024-04-28T13:31:28.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.356+0000 7f41dec00640 1 -- 172.21.15.196:0/636379091 shutdown_connections 2024-04-28T13:31:28.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.356+0000 7f41dec00640 1 -- 172.21.15.196:0/636379091 >> 172.21.15.196:0/636379091 conn(0x7f41d80d5110 msgr2=0x7f41d80b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.356+0000 7f41dec00640 1 -- 172.21.15.196:0/636379091 shutdown_connections 2024-04-28T13:31:28.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.356+0000 7f41dec00640 1 -- 172.21.15.196:0/636379091 wait complete. 2024-04-28T13:31:28.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.356+0000 7f6abf400640 1 -- 172.21.15.196:0/3116811413 <== osd.3 v2:172.21.15.196:6815/176565235 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6ac4056bf0 con 0x7f6aac075770 2024-04-28T13:31:28.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.356+0000 7fe45c200640 1 -- 172.21.15.196:0/2207551420 --> [v2:172.21.15.196:6800/3595094355,v1:172.21.15.196:6801/3595094355] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe4541639e0 con 0x7fe438074ac0 2024-04-28T13:31:28.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.358+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3116811413 >> [v2:172.21.15.196:6815/176565235,v1:172.21.15.196:6817/176565235] conn(0x7f6aac075770 msgr2=0x7f6aac077c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.359+0000 7fe451600640 1 -- 172.21.15.196:0/2207551420 <== osd.0 v2:172.21.15.196:6800/3595094355 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe4541639e0 con 0x7fe438074ac0 2024-04-28T13:31:28.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.358+0000 7f6ad0a00640 1 --2- 172.21.15.196:0/3116811413 >> [v2:172.21.15.196:6815/176565235,v1:172.21.15.196:6817/176565235] conn(0x7f6aac075770 0x7f6aac077c10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.358+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3116811413 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6aac06c8f0 msgr2=0x7f6aac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.358+0000 7f6ad0a00640 1 --2- 172.21.15.196:0/3116811413 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6aac06c8f0 0x7f6aac06edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0007fd0 tx=0x7f6ac0007be0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.358+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3116811413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ac40e3160 msgr2=0x7f6ac40e0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.358+0000 7f6ad0a00640 1 --2- 172.21.15.196:0/3116811413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ac40e3160 0x7f6ac40e0db0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6acc06aae0 tx=0x7f6acc069eb0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.358+0000 7f6acbe00640 1 -- 172.21.15.196:0/3116811413 reap_dead start 2024-04-28T13:31:28.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.359+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3116811413 shutdown_connections 2024-04-28T13:31:28.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.359+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3116811413 >> 172.21.15.196:0/3116811413 conn(0x7f6ac4058ee0 msgr2=0x7f6ac4054120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.359+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3116811413 shutdown_connections 2024-04-28T13:31:28.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.359+0000 7f6ad0a00640 1 -- 172.21.15.196:0/3116811413 wait complete. 2024-04-28T13:31:28.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.361+0000 7fe45c200640 1 -- 172.21.15.196:0/2207551420 >> [v2:172.21.15.196:6800/3595094355,v1:172.21.15.196:6801/3595094355] conn(0x7fe438074ac0 msgr2=0x7fe438076ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.361+0000 7fe45c200640 1 --2- 172.21.15.196:0/2207551420 >> [v2:172.21.15.196:6800/3595094355,v1:172.21.15.196:6801/3595094355] conn(0x7fe438074ac0 0x7fe438076ee0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.361+0000 7fe45c200640 1 -- 172.21.15.196:0/2207551420 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe43806c920 msgr2=0x7fe43806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.361+0000 7fe45c200640 1 --2- 172.21.15.196:0/2207551420 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe43806c920 0x7fe43806ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe448035000 tx=0x7fe44802f000 comp rx=0 tx=0).stop 2024-04-28T13:31:28.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.361+0000 7fe45c200640 1 -- 172.21.15.196:0/2207551420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe43c004800 msgr2=0x7fe4541e9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.361+0000 7fe45c200640 1 --2- 172.21.15.196:0/2207551420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe43c004800 0x7fe4541e9b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe44c007370 tx=0x7fe44c007830 comp rx=0 tx=0).stop 2024-04-28T13:31:28.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.363+0000 7fe45a200640 1 -- 172.21.15.196:0/2207551420 reap_dead start 2024-04-28T13:31:28.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.363+0000 7f86ecc00640 1 -- 172.21.15.196:0/3108656208 --> [v2:172.21.15.203:6800/1325487575,v1:172.21.15.203:6801/1325487575] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86cc002dd0 con 0x7f86d8074bc0 2024-04-28T13:31:28.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.363+0000 7fe45c200640 1 -- 172.21.15.196:0/2207551420 shutdown_connections 2024-04-28T13:31:28.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.363+0000 7fe45c200640 1 -- 172.21.15.196:0/2207551420 >> 172.21.15.196:0/2207551420 conn(0x7fe4540d0d30 msgr2=0x7fe45406f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.364+0000 7fe45c200640 1 -- 172.21.15.196:0/2207551420 shutdown_connections 2024-04-28T13:31:28.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.364+0000 7fe45c200640 1 -- 172.21.15.196:0/2207551420 wait complete. 2024-04-28T13:31:28.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.365+0000 7fa410800640 1 -- 172.21.15.196:0/1110636039 --> [v2:172.21.15.203:6802/2023229662,v1:172.21.15.203:6805/2023229662] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa408069e50 con 0x7fa3ec074c40 2024-04-28T13:31:28.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.365+0000 7f86ef400640 1 -- 172.21.15.196:0/3108656208 <== osd.4 v2:172.21.15.203:6800/1325487575 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f86cc002dd0 con 0x7f86d8074bc0 2024-04-28T13:31:28.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.365+0000 7f86ecc00640 1 -- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:6800/1325487575,v1:172.21.15.203:6801/1325487575] conn(0x7f86d8074bc0 msgr2=0x7f86d8076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.365+0000 7f86ecc00640 1 --2- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:6800/1325487575,v1:172.21.15.203:6801/1325487575] conn(0x7f86d8074bc0 0x7f86d8076fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.366+0000 7fa402000640 1 -- 172.21.15.196:0/1110636039 <== osd.7 v2:172.21.15.203:6802/2023229662 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa408069e50 con 0x7fa3ec074c40 2024-04-28T13:31:28.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.366+0000 7f86ecc00640 1 -- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f86d806c9c0 msgr2=0x7f86d806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.366+0000 7f86ecc00640 1 --2- 172.21.15.196:0/3108656208 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f86d806c9c0 0x7f86d806ee80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f86e4034000 tx=0x7f86e402e000 comp rx=0 tx=0).stop 2024-04-28T13:31:28.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.366+0000 7f86ecc00640 1 -- 172.21.15.196:0/3108656208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86f80d37d0 msgr2=0x7f86f81f5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.366+0000 7f86ecc00640 1 --2- 172.21.15.196:0/3108656208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86f80d37d0 0x7f86f81f5170 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f86e800e9b0 tx=0x7f86e800ee70 comp rx=0 tx=0).stop 2024-04-28T13:31:28.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.367+0000 7f86f7e00640 1 -- 172.21.15.196:0/3108656208 reap_dead start 2024-04-28T13:31:28.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.367+0000 7fa410800640 1 -- 172.21.15.196:0/1110636039 >> [v2:172.21.15.203:6802/2023229662,v1:172.21.15.203:6805/2023229662] conn(0x7fa3ec074c40 msgr2=0x7fa3ec077060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.367+0000 7f86ecc00640 1 -- 172.21.15.196:0/3108656208 shutdown_connections 2024-04-28T13:31:28.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.367+0000 7f86ecc00640 1 -- 172.21.15.196:0/3108656208 >> 172.21.15.196:0/3108656208 conn(0x7f86f808ec10 msgr2=0x7f86f806e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.367+0000 7fa410800640 1 --2- 172.21.15.196:0/1110636039 >> [v2:172.21.15.203:6802/2023229662,v1:172.21.15.203:6805/2023229662] conn(0x7fa3ec074c40 0x7fa3ec077060 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.367+0000 7fa410800640 1 -- 172.21.15.196:0/1110636039 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa3ec06c9c0 msgr2=0x7fa3ec06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.367+0000 7fa410800640 1 --2- 172.21.15.196:0/1110636039 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa3ec06c9c0 0x7fa3ec06ee80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc005880 tx=0x7fa3fc009040 comp rx=0 tx=0).stop 2024-04-28T13:31:28.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.367+0000 7fa410800640 1 -- 172.21.15.196:0/1110636039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4080d51a0 msgr2=0x7fa408160540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.367+0000 7fa410800640 1 --2- 172.21.15.196:0/1110636039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4080d51a0 0x7fa408160540 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800e850 tx=0x7fa3f800ed10 comp rx=0 tx=0).stop 2024-04-28T13:31:28.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.367+0000 7fa40e800640 1 -- 172.21.15.196:0/1110636039 reap_dead start 2024-04-28T13:31:28.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.367+0000 7fa410800640 1 -- 172.21.15.196:0/1110636039 shutdown_connections 2024-04-28T13:31:28.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.367+0000 7fa410800640 1 -- 172.21.15.196:0/1110636039 >> 172.21.15.196:0/1110636039 conn(0x7fa40808ec20 msgr2=0x7fa408155480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.368+0000 7fa410800640 1 -- 172.21.15.196:0/1110636039 shutdown_connections 2024-04-28T13:31:28.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.368+0000 7f86ecc00640 1 -- 172.21.15.196:0/3108656208 shutdown_connections 2024-04-28T13:31:28.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.368+0000 7fa410800640 1 -- 172.21.15.196:0/1110636039 wait complete. 2024-04-28T13:31:28.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.368+0000 7f86ecc00640 1 -- 172.21.15.196:0/3108656208 wait complete. 2024-04-28T13:31:28.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.368+0000 7f82ef200640 1 -- 172.21.15.196:0/561753753 --> [v2:172.21.15.203:6804/2209636555,v1:172.21.15.203:6807/2209636555] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f82e80d5a10 con 0x7f82c4074be0 2024-04-28T13:31:28.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.370+0000 7f82dfe00640 1 -- 172.21.15.196:0/561753753 <== osd.6 v2:172.21.15.203:6804/2209636555 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f82e80d5a10 con 0x7f82c4074be0 2024-04-28T13:31:28.392 INFO:teuthology.orchestra.run.smithi196.stdout:55834574854 2024-04-28T13:31:28.393 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-28T13:31:28.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.372+0000 7f82ef200640 1 -- 172.21.15.196:0/561753753 >> [v2:172.21.15.203:6804/2209636555,v1:172.21.15.203:6807/2209636555] conn(0x7f82c4074be0 msgr2=0x7f82c4077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.372+0000 7f82ef200640 1 --2- 172.21.15.196:0/561753753 >> [v2:172.21.15.203:6804/2209636555,v1:172.21.15.203:6807/2209636555] conn(0x7f82c4074be0 0x7f82c4077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.372+0000 7f82ef200640 1 -- 172.21.15.196:0/561753753 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f82c406c9c0 msgr2=0x7f82c406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.395 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.372+0000 7f82ef200640 1 --2- 172.21.15.196:0/561753753 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f82c406c9c0 0x7f82c406ee80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f82e0007fd0 tx=0x7f82e0009040 comp rx=0 tx=0).stop 2024-04-28T13:31:28.395 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.372+0000 7f82ef200640 1 -- 172.21.15.196:0/561753753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82e815f750 msgr2=0x7f82e80b9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.395 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.372+0000 7f82ef200640 1 --2- 172.21.15.196:0/561753753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82e815f750 0x7f82e80b9e00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f82d8031000 tx=0x7f82d80046c0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.395 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.372+0000 7f82ed200640 1 -- 172.21.15.196:0/561753753 reap_dead start 2024-04-28T13:31:28.395 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.373+0000 7f82ef200640 1 -- 172.21.15.196:0/561753753 shutdown_connections 2024-04-28T13:31:28.395 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.373+0000 7f82ef200640 1 -- 172.21.15.196:0/561753753 >> 172.21.15.196:0/561753753 conn(0x7f82e808ec10 msgr2=0x7f82e80d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.395 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.373+0000 7f82ef200640 1 -- 172.21.15.196:0/561753753 shutdown_connections 2024-04-28T13:31:28.395 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.373+0000 7f82ef200640 1 -- 172.21.15.196:0/561753753 wait complete. 2024-04-28T13:31:28.396 INFO:teuthology.orchestra.run.smithi196.stdout:55834574853 2024-04-28T13:31:28.396 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-28T13:31:28.397 INFO:teuthology.orchestra.run.smithi196.stdout:55834574854 2024-04-28T13:31:28.397 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-28T13:31:28.398 INFO:teuthology.orchestra.run.smithi196.stdout:55834574855 2024-04-28T13:31:28.398 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-28T13:31:28.399 INFO:teuthology.orchestra.run.smithi196.stdout:55834574854 2024-04-28T13:31:28.399 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-28T13:31:28.400 INFO:teuthology.orchestra.run.smithi196.stdout:55834574856 2024-04-28T13:31:28.400 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-28T13:31:28.401 INFO:teuthology.orchestra.run.smithi196.stdout:55834574855 2024-04-28T13:31:28.401 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-28T13:31:28.402 INFO:teuthology.orchestra.run.smithi196.stdout:55834574855 2024-04-28T13:31:28.403 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-28T13:31:28.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.716+0000 7f88d5400640 1 Processor -- start 2024-04-28T13:31:28.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.717+0000 7f88d5400640 1 -- start start 2024-04-28T13:31:28.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.720+0000 7f88d5400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f88d0160970 0x7f88d00d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.720+0000 7f88d5400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f88d00ceef0 0x7f88d00d1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.720+0000 7f88d5400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88d0070e10 con 0x7f88d00720d0 2024-04-28T13:31:28.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.720+0000 7f88d5400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f88d00710f0 con 0x7f88d0160970 2024-04-28T13:31:28.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.720+0000 7f88d5400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f88d00713d0 con 0x7f88d00ceef0 2024-04-28T13:31:28.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.721+0000 7f88ce800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f88d0160970 0x7f88d00d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.721+0000 7f88cf200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f88d00ceef0 0x7f88d00d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.721+0000 7f88cf200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f88d00ceef0 0x7f88d00d1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:39640/0 (socket says 172.21.15.196:39640) 2024-04-28T13:31:28.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.721+0000 7f88cf200640 1 -- 172.21.15.196:0/694206293 learned_addr learned my addr 172.21.15.196:0/694206293 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.722+0000 7f88cd400640 1 -- 172.21.15.196:0/694206293 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2725004224 0 0) 0x7f88d0070e10 con 0x7f88d00720d0 2024-04-28T13:31:28.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.722+0000 7f88cd400640 1 -- 172.21.15.196:0/694206293 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88b4007930 con 0x7f88d00720d0 2024-04-28T13:31:28.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.723+0000 7f88cd400640 1 -- 172.21.15.196:0/694206293 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 266012231 0 0) 0x7f88b4007930 con 0x7f88d00720d0 2024-04-28T13:31:28.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.723+0000 7f88cd400640 1 -- 172.21.15.196:0/694206293 >> v2:172.21.15.203:3300/0 conn(0x7f88d00ceef0 msgr2=0x7f88d00d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.723+0000 7f88cd400640 1 --2- 172.21.15.196:0/694206293 >> v2:172.21.15.203:3300/0 conn(0x7f88d00ceef0 0x7f88d00d1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.723+0000 7f88cd400640 1 -- 172.21.15.196:0/694206293 >> v2:172.21.15.196:3300/0 conn(0x7f88d0160970 msgr2=0x7f88d00d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.723+0000 7f88cd400640 1 --2- 172.21.15.196:0/694206293 >> v2:172.21.15.196:3300/0 conn(0x7f88d0160970 0x7f88d00d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.723+0000 7f88cd400640 1 -- 172.21.15.196:0/694206293 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d00b58a0 con 0x7f88d00720d0 2024-04-28T13:31:28.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.723+0000 7f88cd400640 1 -- 172.21.15.196:0/694206293 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f88bc001670 con 0x7f88d00720d0 2024-04-28T13:31:28.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.723+0000 7f88cd400640 1 -- 172.21.15.196:0/694206293 >> v1:172.21.15.196:6789/0 conn(0x7f88d00720d0 legacy=0x7f88d009ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.723+0000 7f88cd400640 1 --2- 172.21.15.196:0/694206293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b4007dc0 0x7f88b40081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.723+0000 7f88cd400640 1 --2- 172.21.15.196:0/694206293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4008c10 0x7f88b4009090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.723+0000 7f88cd400640 1 --2- 172.21.15.196:0/694206293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88b40095d0 0x7f88b400fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.723+0000 7f88cd400640 1 -- 172.21.15.196:0/694206293 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f88d00710f0 con 0x7f88b4008c10 2024-04-28T13:31:28.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.723+0000 7f88cd400640 1 -- 172.21.15.196:0/694206293 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f88d00713d0 con 0x7f88b4007dc0 2024-04-28T13:31:28.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.723+0000 7f88cd400640 1 -- 172.21.15.196:0/694206293 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f88b4010360 con 0x7f88b40095d0 2024-04-28T13:31:28.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.724+0000 7f88cde00640 1 --2- 172.21.15.196:0/694206293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4008c10 0x7f88b4009090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.724+0000 7f88cde00640 1 -- 172.21.15.196:0/694206293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88b40095d0 msgr2=0x7f88b400fdd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T13:31:28.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.724+0000 7f88cde00640 1 --2- 172.21.15.196:0/694206293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88b40095d0 0x7f88b400fdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.724+0000 7f88cde00640 1 -- 172.21.15.196:0/694206293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b4007dc0 msgr2=0x7f88b40081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.724+0000 7f88cde00640 1 --2- 172.21.15.196:0/694206293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b4007dc0 0x7f88b40081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.724+0000 7f88cde00640 1 -- 172.21.15.196:0/694206293 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d00b58a0 con 0x7f88b4008c10 2024-04-28T13:31:28.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.726+0000 7f1f34e00640 1 Processor -- start 2024-04-28T13:31:28.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.726+0000 7f88cde00640 1 --2- 172.21.15.196:0/694206293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4008c10 0x7f88b4009090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00c1b0 tx=0x7f88b4009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6f1ed51ee8f9b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.726+0000 7f88cf200640 1 -- 172.21.15.196:0/694206293 reap_dead start 2024-04-28T13:31:28.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.726+0000 7f1f34e00640 1 -- start start 2024-04-28T13:31:28.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.727+0000 7f88cd400640 1 -- 172.21.15.196:0/694206293 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc038070 con 0x7f88b4008c10 2024-04-28T13:31:28.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.727+0000 7f88cd400640 1 -- 172.21.15.196:0/694206293 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88bc033680 con 0x7f88b4008c10 2024-04-28T13:31:28.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.727+0000 7f88cd400640 1 -- 172.21.15.196:0/694206293 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc038070 con 0x7f88b4008c10 2024-04-28T13:31:28.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.728+0000 7f88d5400640 1 -- 172.21.15.196:0/694206293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4008c10 msgr2=0x7f88b4009090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.728+0000 7f88d5400640 1 --2- 172.21.15.196:0/694206293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4008c10 0x7f88b4009090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00c1b0 tx=0x7f88b4009c40 comp rx=0 tx=0).stop 2024-04-28T13:31:28.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.729+0000 7f88d5400640 1 -- 172.21.15.196:0/694206293 shutdown_connections 2024-04-28T13:31:28.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.729+0000 7f88d5400640 1 --2- 172.21.15.196:0/694206293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4008c10 0x7f88b4009090 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.729+0000 7f88d5400640 1 -- 172.21.15.196:0/694206293 >> 172.21.15.196:0/694206293 conn(0x7f88d00b4a10 msgr2=0x7f88d00b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.729+0000 7f1f34e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1f30152230 0x7f1f3015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.729+0000 7f1f34e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1f300b63a0 0x7f1f300b6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.729+0000 7f1f34e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f3015c720 con 0x7f1f3015ed50 2024-04-28T13:31:28.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.729+0000 7f1f34e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1f300b6d50 con 0x7f1f30152230 2024-04-28T13:31:28.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.729+0000 7f1f34e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f1f300ce940 con 0x7f1f300b63a0 2024-04-28T13:31:28.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.729+0000 7f88d5400640 1 -- 172.21.15.196:0/694206293 shutdown_connections 2024-04-28T13:31:28.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.729+0000 7f88d5400640 1 -- 172.21.15.196:0/694206293 wait complete. 2024-04-28T13:31:28.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.730+0000 7f1f2e800640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f1f3015ed50 0x7f1f3015f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.196:36434/0 (socket says 172.21.15.196:36434) 2024-04-28T13:31:28.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.730+0000 7f1f2e800640 1 -- 172.21.15.196:0/640551270 learned_addr learned my addr 172.21.15.196:0/640551270 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:28.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.730+0000 7f1f2d400640 1 --2- 172.21.15.196:0/640551270 >> v2:172.21.15.203:3300/0 conn(0x7f1f300b63a0 0x7f1f300b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.731+0000 7f88d5400640 1 Processor -- start 2024-04-28T13:31:28.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.732+0000 7f88d5400640 1 -- start start 2024-04-28T13:31:28.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.731+0000 7f1f2ca00640 1 -- 172.21.15.196:0/640551270 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2501516827 0 0) 0x7f1f28001490 con 0x7f1f3015ed50 2024-04-28T13:31:28.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.733+0000 7f1f2ca00640 1 -- 172.21.15.196:0/640551270 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f14003520 con 0x7f1f3015ed50 2024-04-28T13:31:28.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.730+0000 7f1f2de00640 1 --2- 172.21.15.196:0/640551270 >> v2:172.21.15.196:3300/0 conn(0x7f1f30152230 0x7f1f3015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.733+0000 7fd7d0400640 1 Processor -- start 2024-04-28T13:31:28.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.733+0000 7f88d5400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4008c10 0x7f88d015fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.733+0000 7f88d5400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d00cc070 0x7f88d0159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.733+0000 7f88d5400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d00cc4d0 0x7f88d015a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.733+0000 7f88d5400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f88d00b5530 con 0x7f88b4008c10 2024-04-28T13:31:28.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.733+0000 7f88d5400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0160d50 con 0x7f88d00cc070 2024-04-28T13:31:28.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.733+0000 7f88d5400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f88d00717c0 con 0x7f88d00cc4d0 2024-04-28T13:31:28.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.733+0000 7fd7d0400640 1 -- start start 2024-04-28T13:31:28.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.734+0000 7f88ce800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4008c10 0x7f88d015fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.734+0000 7f88ce800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4008c10 0x7f88d015fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35718/0 (socket says 172.21.15.196:35718) 2024-04-28T13:31:28.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.734+0000 7f88ce800640 1 -- 172.21.15.196:0/1962924672 learned_addr learned my addr 172.21.15.196:0/1962924672 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.734+0000 7f88cde00640 1 --2- 172.21.15.196:0/1962924672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d00cc070 0x7f88d0159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.734+0000 7f88ce800640 1 -- 172.21.15.196:0/1962924672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d00cc4d0 msgr2=0x7f88d015a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.734+0000 7f88ce800640 1 --2- 172.21.15.196:0/1962924672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d00cc4d0 0x7f88d015a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.734+0000 7f88ce800640 1 -- 172.21.15.196:0/1962924672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d00cc070 msgr2=0x7f88d0159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.734+0000 7f88ce800640 1 --2- 172.21.15.196:0/1962924672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d00cc070 0x7f88d0159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.734+0000 7f88ce800640 1 -- 172.21.15.196:0/1962924672 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d00cb2b0 con 0x7f88b4008c10 2024-04-28T13:31:28.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.733+0000 7f1f2d400640 1 -- 172.21.15.196:0/640551270 >> v2:172.21.15.196:3300/0 conn(0x7f1f30152230 msgr2=0x7f1f3015c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.733+0000 7f1f2d400640 1 --2- 172.21.15.196:0/640551270 >> v2:172.21.15.196:3300/0 conn(0x7f1f30152230 0x7f1f3015c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.733+0000 7f1f2d400640 1 -- 172.21.15.196:0/640551270 >> v1:172.21.15.203:6789/0 conn(0x7f1f3015ed50 legacy=0x7f1f3015f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.733+0000 7f1f2d400640 1 -- 172.21.15.196:0/640551270 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3006eea0 con 0x7f1f300b63a0 2024-04-28T13:31:28.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.735+0000 7f88cde00640 1 --2- 172.21.15.196:0/1962924672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d00cc070 0x7f88d0159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:28.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.735+0000 7f88ce800640 1 --2- 172.21.15.196:0/1962924672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4008c10 0x7f88d015fc90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f88c000e7d0 tx=0x7f88c000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.735+0000 7fd7d0400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd7c806c510 0x7fd7c814cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.736+0000 7fd7d0400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7c80b7600 0x7fd7c814d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.736+0000 7fd7d0400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7c8070740 con 0x7fd7c806c8f0 2024-04-28T13:31:28.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.736+0000 7fd7d0400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd7c8070a20 con 0x7fd7c806c510 2024-04-28T13:31:28.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.736+0000 7fd7d0400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c8070d00 con 0x7fd7c80b7600 2024-04-28T13:31:28.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.735+0000 7f88c6a00640 1 -- 172.21.15.196:0/1962924672 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0017070 con 0x7f88b4008c10 2024-04-28T13:31:28.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.736+0000 7fb6c1200640 1 Processor -- start 2024-04-28T13:31:28.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.736+0000 7f88c6a00640 1 -- 172.21.15.196:0/1962924672 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c000c8e0 con 0x7f88b4008c10 2024-04-28T13:31:28.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.736+0000 7f88c6a00640 1 -- 172.21.15.196:0/1962924672 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c000ccb0 con 0x7f88b4008c10 2024-04-28T13:31:28.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.736+0000 7fb6c1200640 1 -- start start 2024-04-28T13:31:28.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.736+0000 7f1f2d400640 1 --2- 172.21.15.196:0/640551270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f300b63a0 0x7f1f300b6810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1f30159330 tx=0x7f1f1c00ab30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=754aa7e99b82c68d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.737+0000 7f6f03a00640 1 Processor -- start 2024-04-28T13:31:28.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.736+0000 7fd7cd000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fd7c806c8f0 0x7fd7c80b6ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56078/0 (socket says 172.21.15.196:56078) 2024-04-28T13:31:28.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.736+0000 7fd7cd000640 1 -- 172.21.15.196:0/4224268435 learned_addr learned my addr 172.21.15.196:0/4224268435 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:28.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.736+0000 7fd7ce400640 1 --2- 172.21.15.196:0/4224268435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7c80b7600 0x7fd7c814d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.737+0000 7fd7cda00640 1 --2- 172.21.15.196:0/4224268435 >> v2:172.21.15.196:3300/0 conn(0x7fd7c806c510 0x7fd7c814cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.737+0000 7f6f03a00640 1 -- start start 2024-04-28T13:31:28.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.737+0000 7f1f2ca00640 1 -- 172.21.15.196:0/640551270 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28001bc0 con 0x7f1f300b63a0 2024-04-28T13:31:28.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.737+0000 7f1f2ca00640 1 -- 172.21.15.196:0/640551270 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f1c008bf0 con 0x7f1f300b63a0 2024-04-28T13:31:28.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.737+0000 7fd7c7e00640 1 -- 172.21.15.196:0/4224268435 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3973812394 0 0) 0x7fd7c8070740 con 0x7fd7c806c8f0 2024-04-28T13:31:28.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.737+0000 7fd7c7e00640 1 -- 172.21.15.196:0/4224268435 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7b0003520 con 0x7fd7c806c8f0 2024-04-28T13:31:28.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.737+0000 7f88d5400640 1 -- 172.21.15.196:0/1962924672 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d015b710 con 0x7f88b4008c10 2024-04-28T13:31:28.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.737+0000 7f1f2ca00640 1 -- 172.21.15.196:0/640551270 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28001bc0 con 0x7f1f300b63a0 2024-04-28T13:31:28.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.737+0000 7fd7c7e00640 1 -- 172.21.15.196:0/4224268435 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3764138469 0 0) 0x7fd7b0003520 con 0x7fd7c806c8f0 2024-04-28T13:31:28.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.737+0000 7fd7c7e00640 1 -- 172.21.15.196:0/4224268435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7c80b7600 msgr2=0x7fd7c814d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.737+0000 7fd7c7e00640 1 --2- 172.21.15.196:0/4224268435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7c80b7600 0x7fd7c814d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.737+0000 7fd7c7e00640 1 -- 172.21.15.196:0/4224268435 >> v2:172.21.15.196:3300/0 conn(0x7fd7c806c510 msgr2=0x7fd7c814cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.738+0000 7f88d5400640 1 -- 172.21.15.196:0/1962924672 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d015d790 con 0x7f88b4008c10 2024-04-28T13:31:28.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.739+0000 7f1f34e00640 1 -- 172.21.15.196:0/640551270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f300b63a0 msgr2=0x7f1f300b6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.739+0000 7f1f34e00640 1 --2- 172.21.15.196:0/640551270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f300b63a0 0x7f1f300b6810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1f30159330 tx=0x7f1f1c00ab30 comp rx=0 tx=0).stop 2024-04-28T13:31:28.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.739+0000 7f6f03a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6efc15f4d0 0x7f6efc0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.739+0000 7f6f03a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6efc0d2e20 0x7f6efc0d5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.739+0000 7f1f34e00640 1 -- 172.21.15.196:0/640551270 shutdown_connections 2024-04-28T13:31:28.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.739+0000 7f1f34e00640 1 --2- 172.21.15.196:0/640551270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f300b63a0 0x7f1f300b6810 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.739+0000 7f88d5400640 1 -- 172.21.15.196:0/1962924672 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d006a290 con 0x7f88b4008c10 2024-04-28T13:31:28.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.739+0000 7f1f34e00640 1 --2- 172.21.15.196:0/640551270 >> v2:172.21.15.196:3300/0 conn(0x7f1f30152230 0x7f1f3015c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.739+0000 7f1f34e00640 1 -- 172.21.15.196:0/640551270 >> 172.21.15.196:0/640551270 conn(0x7f1f300b2ae0 msgr2=0x7f1f300b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.739+0000 7f6f03a00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6efc0b2550 con 0x7f6efc0d5640 2024-04-28T13:31:28.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.739+0000 7f6f03a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6efc0b6180 con 0x7f6efc0d2e20 2024-04-28T13:31:28.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.739+0000 7f6f03a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f6efc0b2830 con 0x7f6efc15f4d0 2024-04-28T13:31:28.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.740+0000 7f6efbe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6efc0d2e20 0x7f6efc0d5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.740+0000 7f1f34e00640 1 -- 172.21.15.196:0/640551270 shutdown_connections 2024-04-28T13:31:28.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.740+0000 7f2c45a00640 1 Processor -- start 2024-04-28T13:31:28.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.738+0000 7fd7c7e00640 1 --2- 172.21.15.196:0/4224268435 >> v2:172.21.15.196:3300/0 conn(0x7fd7c806c510 0x7fd7c814cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.738+0000 7fd7c7e00640 1 -- 172.21.15.196:0/4224268435 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c8070fe0 con 0x7fd7c806c8f0 2024-04-28T13:31:28.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.738+0000 7fd7c7e00640 1 -- 172.21.15.196:0/4224268435 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fd7bc001670 con 0x7fd7c806c8f0 2024-04-28T13:31:28.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.738+0000 7fd7c7e00640 1 -- 172.21.15.196:0/4224268435 >> v1:172.21.15.196:6789/0 conn(0x7fd7c806c8f0 legacy=0x7fd7c80b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.738+0000 7fd7c7e00640 1 --2- 172.21.15.196:0/4224268435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b00039b0 0x7fd7b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.738+0000 7fd7c7e00640 1 --2- 172.21.15.196:0/4224268435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7b0004800 0x7fd7b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.738+0000 7fd7c7e00640 1 --2- 172.21.15.196:0/4224268435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7b00051c0 0x7fd7b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.738+0000 7fd7c7e00640 1 -- 172.21.15.196:0/4224268435 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c8070a20 con 0x7fd7b00039b0 2024-04-28T13:31:28.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.738+0000 7fd7c7e00640 1 -- 172.21.15.196:0/4224268435 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c8070d00 con 0x7fd7b0004800 2024-04-28T13:31:28.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.738+0000 7fd7c7e00640 1 -- 172.21.15.196:0/4224268435 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b000c090 con 0x7fd7b00051c0 2024-04-28T13:31:28.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.738+0000 7fd7cda00640 1 --2- 172.21.15.196:0/4224268435 >> v2:172.21.15.196:3300/0 conn(0x7fd7c806c510 0x7fd7c814cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:31:28.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.739+0000 7fd7cd000640 1 --2- 172.21.15.196:0/4224268435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7b0004800 0x7fd7b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.740+0000 7f1f34e00640 1 -- 172.21.15.196:0/640551270 wait complete. 2024-04-28T13:31:28.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.739+0000 7fd7cda00640 1 --2- 172.21.15.196:0/4224268435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b00039b0 0x7fd7b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.740+0000 7f6efbe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6efc0d2e20 0x7f6efc0d5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35752/0 (socket says 172.21.15.196:35752) 2024-04-28T13:31:28.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.740+0000 7f6efbe00640 1 -- 172.21.15.196:0/1955473931 learned_addr learned my addr 172.21.15.196:0/1955473931 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.740+0000 7fb6c1200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6bc14c600 0x7fb6bc0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.740+0000 7fb6c1200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb6bc0d1460 0x7fb6bc072020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.741+0000 7fb6b9400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb6bc0d1460 0x7fb6bc072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.741+0000 7fb6b9400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb6bc0d1460 0x7fb6bc072020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:39690/0 (socket says 172.21.15.196:39690) 2024-04-28T13:31:28.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.741+0000 7f6efbe00640 1 -- 172.21.15.196:0/1955473931 >> v2:172.21.15.203:3300/0 conn(0x7f6efc15f4d0 msgr2=0x7f6efc0b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.741+0000 7f6f01000640 1 --2- 172.21.15.196:0/1955473931 >> v2:172.21.15.203:3300/0 conn(0x7f6efc15f4d0 0x7f6efc0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.741+0000 7fb6b9e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6bc14c600 0x7fb6bc0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.741+0000 7fb6b9400640 1 -- 172.21.15.196:0/2734220529 learned_addr learned my addr 172.21.15.196:0/2734220529 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.741+0000 7fb6c1200640 1 -- 172.21.15.196:0/2734220529 --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6bc0ceef0 con 0x7fb6bc072560 2024-04-28T13:31:28.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.741+0000 7fb6c1200640 1 -- 172.21.15.196:0/2734220529 --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb6bc14c9e0 con 0x7fb6bc0d1460 2024-04-28T13:31:28.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.741+0000 7fb6c1200640 1 -- 172.21.15.196:0/2734220529 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb6bc0b4df0 con 0x7fb6bc14c600 2024-04-28T13:31:28.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.741+0000 7f6efbe00640 1 --2- 172.21.15.196:0/1955473931 >> v2:172.21.15.203:3300/0 conn(0x7f6efc15f4d0 0x7f6efc0b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.741+0000 7f6efbe00640 1 -- 172.21.15.196:0/1955473931 >> v1:172.21.15.203:6789/0 conn(0x7f6efc0d5640 legacy=0x7f6efc0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.741+0000 7f6efbe00640 1 -- 172.21.15.196:0/1955473931 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6efc0b2b10 con 0x7f6efc0d2e20 2024-04-28T13:31:28.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7fd7ce400640 1 --2- 172.21.15.196:0/4224268435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7b00051c0 0x7fd7b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7fcaa2c00640 1 Processor -- start 2024-04-28T13:31:28.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7f6f01000640 1 --2- 172.21.15.196:0/1955473931 >> v2:172.21.15.203:3300/0 conn(0x7f6efc15f4d0 0x7f6efc0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:28.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.740+0000 7f2c45a00640 1 -- start start 2024-04-28T13:31:28.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7f2c45a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2c40072020 0x7f2c400724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7f2c45a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c400ceef0 con 0x7f2c4014c600 2024-04-28T13:31:28.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7f2c45a00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c400cf1d0 con 0x7f2c400b3de0 2024-04-28T13:31:28.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7f2c45a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f2c4014c9e0 con 0x7f2c40072020 2024-04-28T13:31:28.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7fd7cd000640 1 -- 172.21.15.196:0/4224268435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7b00051c0 msgr2=0x7fd7b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7fd7cd000640 1 --2- 172.21.15.196:0/4224268435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7b00051c0 0x7fd7b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7fd7cd000640 1 -- 172.21.15.196:0/4224268435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b00039b0 msgr2=0x7fd7b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7fd7cd000640 1 --2- 172.21.15.196:0/4224268435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b00039b0 0x7fd7b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7fd7cd000640 1 -- 172.21.15.196:0/4224268435 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c8070fe0 con 0x7fd7b0004800 2024-04-28T13:31:28.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7fcaa2c00640 1 -- start start 2024-04-28T13:31:28.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7fb6b9400640 1 -- 172.21.15.196:0/2734220529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6bc14c600 msgr2=0x7fb6bc0d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7fb6b9400640 1 --2- 172.21.15.196:0/2734220529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6bc14c600 0x7fb6bc0d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7fb6b9400640 1 -- 172.21.15.196:0/2734220529 >> v1:172.21.15.203:6789/0 conn(0x7fb6bc072560 legacy=0x7fb6bc06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.742+0000 7fb6b9400640 1 -- 172.21.15.196:0/2734220529 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6bc0cf1d0 con 0x7fb6bc0d1460 2024-04-28T13:31:28.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.743+0000 7f2c3e000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2c40072020 0x7f2c400724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.743+0000 7f6efbe00640 1 --2- 172.21.15.196:0/1955473931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6efc0d2e20 0x7f6efc0d5100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6efc08f070 tx=0x7f6ee802a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480c5c8326eb175e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.744+0000 7f2c3f400640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f2c400b3de0 0x7f2c400b4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.196:36462/0 (socket says 172.21.15.196:36462) 2024-04-28T13:31:28.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.744+0000 7f2c3f400640 1 -- 172.21.15.196:0/713631661 learned_addr learned my addr 172.21.15.196:0/713631661 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:28.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.744+0000 7fb6b9400640 1 --2- 172.21.15.196:0/2734220529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6bc0d1460 0x7fb6bc072020 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009870 tx=0x7fb6a402a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a3132921f1aa6e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.744+0000 7fcaa2c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca9c14c600 0x7fca9c0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.744+0000 7fcaa2c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca9c0ceef0 con 0x7fca9c0d1460 2024-04-28T13:31:28.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.744+0000 7fcaa2c00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca9c0cf1d0 con 0x7fca9c072730 2024-04-28T13:31:28.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.744+0000 7fcaa2c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fca9c14c9e0 con 0x7fca9c14c600 2024-04-28T13:31:28.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.744+0000 7f2c3d600640 1 -- 172.21.15.196:0/713631661 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 460180526 0 0) 0x7f2c400cf1d0 con 0x7f2c400b3de0 2024-04-28T13:31:28.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.745+0000 7f2c3d600640 1 -- 172.21.15.196:0/713631661 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c24003520 con 0x7f2c400b3de0 2024-04-28T13:31:28.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.745+0000 7fb6b8a00640 1 -- 172.21.15.196:0/2734220529 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4037070 con 0x7fb6bc0d1460 2024-04-28T13:31:28.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.745+0000 7fcaa0c00640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fca9c072730 0x7fca9c0b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.196:36468/0 (socket says 172.21.15.196:36468) 2024-04-28T13:31:28.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.745+0000 7fcaa0c00640 1 -- 172.21.15.196:0/4074888475 learned_addr learned my addr 172.21.15.196:0/4074888475 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:28.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.745+0000 7fca9be00640 1 --2- 172.21.15.196:0/4074888475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca9c14c600 0x7fca9c0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.745+0000 7fb6b8a00640 1 -- 172.21.15.196:0/2734220529 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a4032440 con 0x7fb6bc0d1460 2024-04-28T13:31:28.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.745+0000 7f2c3e000640 1 -- 172.21.15.196:0/713631661 >> v1:172.21.15.203:6789/0 conn(0x7f2c400b3de0 legacy=0x7f2c400b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.745+0000 7fb6b8a00640 1 -- 172.21.15.196:0/2734220529 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4032810 con 0x7fb6bc0d1460 2024-04-28T13:31:28.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.744+0000 7f88c6a00640 1 -- 172.21.15.196:0/1962924672 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88c001a070 con 0x7f88b4008c10 2024-04-28T13:31:28.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.744+0000 7f88c6a00640 1 --2- 172.21.15.196:0/1962924672 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88b006c970 0x7f88b006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.745+0000 7f88c6a00640 1 -- 172.21.15.196:0/1962924672 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f88c008c140 con 0x7f88b4008c10 2024-04-28T13:31:28.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.745+0000 7f88cde00640 1 --2- 172.21.15.196:0/1962924672 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88b006c970 0x7f88b006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.744+0000 7fd7cd000640 1 --2- 172.21.15.196:0/4224268435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7b0004800 0x7fd7b0004c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc01d360 tx=0x7fd7b0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74f5f6c62c62a919 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.745+0000 7fd7c7e00640 1 -- 172.21.15.196:0/4224268435 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc007e10 con 0x7fd7b0004800 2024-04-28T13:31:28.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.745+0000 7fd7c7e00640 1 -- 172.21.15.196:0/4224268435 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7bc045070 con 0x7fd7b0004800 2024-04-28T13:31:28.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.745+0000 7fd7c7e00640 1 -- 172.21.15.196:0/4224268435 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc005d10 con 0x7fd7b0004800 2024-04-28T13:31:28.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.745+0000 7f2c3e000640 1 -- 172.21.15.196:0/713631661 >> v1:172.21.15.196:6789/0 conn(0x7f2c4014c600 legacy=0x7f2c400d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.746+0000 7fb6c1200640 1 -- 172.21.15.196:0/2734220529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6bc0d1460 msgr2=0x7fb6bc072020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.746+0000 7fb6c1200640 1 --2- 172.21.15.196:0/2734220529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6bc0d1460 0x7fb6bc072020 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009870 tx=0x7fb6a402a290 comp rx=0 tx=0).stop 2024-04-28T13:31:28.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.746+0000 7fca9b400640 1 -- 172.21.15.196:0/4074888475 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 407189839 0 0) 0x7fca9c0cf1d0 con 0x7fca9c072730 2024-04-28T13:31:28.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.746+0000 7fca9b400640 1 -- 172.21.15.196:0/4074888475 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca80003520 con 0x7fca9c072730 2024-04-28T13:31:28.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.746+0000 7f1f34e00640 1 Processor -- start 2024-04-28T13:31:28.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.747+0000 7f1f34e00640 1 -- start start 2024-04-28T13:31:28.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.746+0000 7fb6c1200640 1 -- 172.21.15.196:0/2734220529 shutdown_connections 2024-04-28T13:31:28.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.746+0000 7fb6c1200640 1 --2- 172.21.15.196:0/2734220529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6bc0d1460 0x7fb6bc072020 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.746+0000 7fb6c1200640 1 --2- 172.21.15.196:0/2734220529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6bc14c600 0x7fb6bc0d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.746+0000 7fb6c1200640 1 -- 172.21.15.196:0/2734220529 >> 172.21.15.196:0/2734220529 conn(0x7fb6bc0cd830 msgr2=0x7fb6bc0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.746+0000 7fb6c1200640 1 -- 172.21.15.196:0/2734220529 shutdown_connections 2024-04-28T13:31:28.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.745+0000 7f2c3e000640 1 -- 172.21.15.196:0/713631661 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c400cf4b0 con 0x7f2c40072020 2024-04-28T13:31:28.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.747+0000 7fd7d0400640 1 -- 172.21.15.196:0/4224268435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7b0004800 msgr2=0x7fd7b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.747+0000 7fd7d0400640 1 --2- 172.21.15.196:0/4224268435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7b0004800 0x7fd7b0004c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc01d360 tx=0x7fd7b0005830 comp rx=0 tx=0).stop 2024-04-28T13:31:28.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.747+0000 7f88cde00640 1 --2- 172.21.15.196:0/1962924672 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88b006c970 0x7f88b006ee30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88bc035000 tx=0x7f88bc009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.747+0000 7f2c3e000640 1 --2- 172.21.15.196:0/713631661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c40072020 0x7f2c400724b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0097a0 tx=0x7f2c2c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ba951a263b880f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.747+0000 7fd7ce400640 1 -- 172.21.15.196:0/4224268435 reap_dead start 2024-04-28T13:31:28.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.746+0000 7fca9be00640 1 -- 172.21.15.196:0/4074888475 >> v1:172.21.15.203:6789/0 conn(0x7fca9c072730 legacy=0x7fca9c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.746+0000 7fca9be00640 1 -- 172.21.15.196:0/4074888475 >> v1:172.21.15.196:6789/0 conn(0x7fca9c0d1460 legacy=0x7fca9c072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.746+0000 7fca9be00640 1 -- 172.21.15.196:0/4074888475 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c0cf4b0 con 0x7fca9c14c600 2024-04-28T13:31:28.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.747+0000 7f2c3d600640 1 -- 172.21.15.196:0/713631661 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c037070 con 0x7f2c40072020 2024-04-28T13:31:28.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.747+0000 7fd7d0400640 1 -- 172.21.15.196:0/4224268435 shutdown_connections 2024-04-28T13:31:28.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.747+0000 7fd7d0400640 1 -- 172.21.15.196:0/4224268435 >> 172.21.15.196:0/4224268435 conn(0x7fd7c806f050 msgr2=0x7fd7c806f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.747+0000 7fb6c1200640 1 -- 172.21.15.196:0/2734220529 wait complete. 2024-04-28T13:31:28.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.747+0000 7f2c3d600640 1 -- 172.21.15.196:0/713631661 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2c032440 con 0x7f2c40072020 2024-04-28T13:31:28.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.747+0000 7f2c3d600640 1 -- 172.21.15.196:0/713631661 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0327f0 con 0x7f2c40072020 2024-04-28T13:31:28.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.748+0000 7fd7d0400640 1 -- 172.21.15.196:0/4224268435 shutdown_connections 2024-04-28T13:31:28.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.748+0000 7fca9be00640 1 --2- 172.21.15.196:0/4074888475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca9c14c600 0x7fca9c0d0f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fca840097d0 tx=0x7fca8402a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60e8121f445cc59b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.748+0000 7fabfaa00640 1 Processor -- start 2024-04-28T13:31:28.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.748+0000 7fabfaa00640 1 -- start start 2024-04-28T13:31:28.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.748+0000 7fca9b400640 1 -- 172.21.15.196:0/4074888475 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84037070 con 0x7fca9c14c600 2024-04-28T13:31:28.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.748+0000 7fca9b400640 1 -- 172.21.15.196:0/4074888475 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca84032440 con 0x7fca9c14c600 2024-04-28T13:31:28.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.748+0000 7fca9b400640 1 -- 172.21.15.196:0/4074888475 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84032810 con 0x7fca9c14c600 2024-04-28T13:31:28.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.748+0000 7f1f34e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f300b63a0 0x7f1f300b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.749+0000 7f1f34e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f30152230 0x7f1f300cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.749+0000 7fd7d0400640 1 -- 172.21.15.196:0/4224268435 wait complete. 2024-04-28T13:31:28.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.749+0000 7fb6c1200640 1 Processor -- start 2024-04-28T13:31:28.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.749+0000 7fb6c1200640 1 -- start start 2024-04-28T13:31:28.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.749+0000 7f2c45a00640 1 -- 172.21.15.196:0/713631661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c40072020 msgr2=0x7f2c400724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.749+0000 7f2c45a00640 1 --2- 172.21.15.196:0/713631661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c40072020 0x7f2c400724b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0097a0 tx=0x7f2c2c02a230 comp rx=0 tx=0).stop 2024-04-28T13:31:28.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.749+0000 7fcaa2c00640 1 -- 172.21.15.196:0/4074888475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca9c14c600 msgr2=0x7fca9c0d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.749+0000 7fcaa2c00640 1 --2- 172.21.15.196:0/4074888475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca9c14c600 0x7fca9c0d0f20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fca840097d0 tx=0x7fca8402a160 comp rx=0 tx=0).stop 2024-04-28T13:31:28.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.749+0000 7f1f2de00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f300b63a0 0x7f1f300b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7f1f2de00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f300b63a0 0x7f1f300b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:44820/0 (socket says 172.21.15.196:44820) 2024-04-28T13:31:28.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7f6efb400640 1 -- 172.21.15.196:0/1955473931 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8037070 con 0x7f6efc0d2e20 2024-04-28T13:31:28.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7f2c45a00640 1 -- 172.21.15.196:0/713631661 shutdown_connections 2024-04-28T13:31:28.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7f2c45a00640 1 --2- 172.21.15.196:0/713631661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c40072020 0x7f2c400724b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7f2c45a00640 1 -- 172.21.15.196:0/713631661 >> 172.21.15.196:0/713631661 conn(0x7f2c400cd830 msgr2=0x7f2c400cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7fabfaa00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fabf4070570 0x7fabf40b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7f6efb400640 1 -- 172.21.15.196:0/1955473931 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee8032440 con 0x7f6efc0d2e20 2024-04-28T13:31:28.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7f6efb400640 1 -- 172.21.15.196:0/1955473931 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8032810 con 0x7f6efc0d2e20 2024-04-28T13:31:28.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7fabfaa00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf415fce0 con 0x7fabf415f6a0 2024-04-28T13:31:28.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7fabfaa00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf415ffc0 con 0x7fabf409ba40 2024-04-28T13:31:28.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7fabfaa00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fabf4160610 con 0x7fabf4070570 2024-04-28T13:31:28.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7f1f2d400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f30152230 0x7f1f300cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7f2c45a00640 1 -- 172.21.15.196:0/713631661 shutdown_connections 2024-04-28T13:31:28.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7fcaa2c00640 1 -- 172.21.15.196:0/4074888475 shutdown_connections 2024-04-28T13:31:28.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7fcaa2c00640 1 --2- 172.21.15.196:0/4074888475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca9c14c600 0x7fca9c0d0f20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7fcaa2c00640 1 -- 172.21.15.196:0/4074888475 >> 172.21.15.196:0/4074888475 conn(0x7fca9c0cd830 msgr2=0x7fca9c0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7f1f2de00640 1 -- 172.21.15.196:0/2613822165 learned_addr learned my addr 172.21.15.196:0/2613822165 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7f2c45a00640 1 -- 172.21.15.196:0/713631661 wait complete. 2024-04-28T13:31:28.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7fabf3e00640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fabf409ba40 0x7fabf409be20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.196:36470/0 (socket says 172.21.15.196:36470) 2024-04-28T13:31:28.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7fabf3e00640 1 -- 172.21.15.196:0/554471906 learned_addr learned my addr 172.21.15.196:0/554471906 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:28.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7f1f34e00640 1 --2- 172.21.15.196:0/2613822165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f3015ed50 0x7f1f300b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7fabf8a00640 1 --2- 172.21.15.196:0/554471906 >> v2:172.21.15.196:3300/0 conn(0x7fabf4070570 0x7fabf40b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7f1f34e00640 1 -- 172.21.15.196:0/2613822165 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f300cee70 con 0x7f1f30152230 2024-04-28T13:31:28.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7f6f03a00640 1 -- 172.21.15.196:0/1955473931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6efc0d2e20 msgr2=0x7f6efc0d5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7fb6c1200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6bc072560 0x7fb6bc162b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7fb6c1200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6bc0d1460 0x7fb6bc0cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7fb6c1200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6bc14c600 0x7fb6bc1e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7f6f03a00640 1 --2- 172.21.15.196:0/1955473931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6efc0d2e20 0x7f6efc0d5100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6efc08f070 tx=0x7f6ee802a1f0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7f1f34e00640 1 -- 172.21.15.196:0/2613822165 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1f300ceb90 con 0x7f1f3015ed50 2024-04-28T13:31:28.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7fabf2a00640 1 -- 172.21.15.196:0/554471906 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4219393642 0 0) 0x7fabf415ffc0 con 0x7fabf409ba40 2024-04-28T13:31:28.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7fabf2a00640 1 -- 172.21.15.196:0/554471906 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabd8003520 con 0x7fabf409ba40 2024-04-28T13:31:28.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7f1f2e800640 1 --2- 172.21.15.196:0/2613822165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f3015ed50 0x7f1f300b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7fb6b9e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6bc072560 0x7fb6bc162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.752+0000 7fb6c1200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc0cfa70 con 0x7fb6bc072560 2024-04-28T13:31:28.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.752+0000 7fb6c1200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc0cf790 con 0x7fb6bc0d1460 2024-04-28T13:31:28.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.752+0000 7fb6c1200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb6bc0cff90 con 0x7fb6bc14c600 2024-04-28T13:31:28.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.752+0000 7f88c6a00640 1 -- 172.21.15.196:0/1962924672 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f88c005a5c0 con 0x7f88b4008c10 2024-04-28T13:31:28.781 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.752+0000 7fb6b9e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6bc072560 0x7fb6bc162b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35780/0 (socket says 172.21.15.196:35780) 2024-04-28T13:31:28.781 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.752+0000 7fb6b9e00640 1 -- 172.21.15.196:0/3126007439 learned_addr learned my addr 172.21.15.196:0/3126007439 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.781 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.752+0000 7fb6ba800640 1 --2- 172.21.15.196:0/3126007439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6bc14c600 0x7fb6bc1e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.781 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.752+0000 7fb6b9400640 1 --2- 172.21.15.196:0/3126007439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6bc0d1460 0x7fb6bc0cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.781 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7f1f34e00640 1 -- 172.21.15.196:0/2613822165 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1f30152610 con 0x7f1f300b63a0 2024-04-28T13:31:28.781 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.752+0000 7fabf8a00640 1 -- 172.21.15.196:0/554471906 >> v1:172.21.15.203:6789/0 conn(0x7fabf409ba40 legacy=0x7fabf409be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.752+0000 7fabf8a00640 1 -- 172.21.15.196:0/554471906 >> v1:172.21.15.196:6789/0 conn(0x7fabf415f6a0 legacy=0x7fabf40d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:28.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.752+0000 7fabf8a00640 1 -- 172.21.15.196:0/554471906 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf41602a0 con 0x7fabf4070570 2024-04-28T13:31:28.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.752+0000 7fd7d0400640 1 Processor -- start 2024-04-28T13:31:28.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.752+0000 7f2c45a00640 1 Processor -- start 2024-04-28T13:31:28.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7f6f03a00640 1 -- 172.21.15.196:0/1955473931 shutdown_connections 2024-04-28T13:31:28.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7f6f03a00640 1 --2- 172.21.15.196:0/1955473931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6efc0d2e20 0x7f6efc0d5100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7f6f03a00640 1 --2- 172.21.15.196:0/1955473931 >> v2:172.21.15.203:3300/0 conn(0x7f6efc15f4d0 0x7f6efc0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.751+0000 7f6f03a00640 1 -- 172.21.15.196:0/1955473931 >> 172.21.15.196:0/1955473931 conn(0x7f6efc08ebf0 msgr2=0x7f6efc09e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.752+0000 7f6f03a00640 1 -- 172.21.15.196:0/1955473931 shutdown_connections 2024-04-28T13:31:28.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.752+0000 7fb6b9e00640 1 -- 172.21.15.196:0/3126007439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6bc14c600 msgr2=0x7fb6bc1e9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.784 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.753+0000 7f6f03a00640 1 -- 172.21.15.196:0/1955473931 wait complete. 2024-04-28T13:31:28.784 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.753+0000 7f2c45a00640 1 -- start start 2024-04-28T13:31:28.784 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.753+0000 7fb6b9e00640 1 --2- 172.21.15.196:0/3126007439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6bc14c600 0x7fb6bc1e9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.784 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.753+0000 7fb6b9e00640 1 -- 172.21.15.196:0/3126007439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6bc0d1460 msgr2=0x7fb6bc0cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.784 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.753+0000 7fb6b9e00640 1 --2- 172.21.15.196:0/3126007439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6bc0d1460 0x7fb6bc0cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.753+0000 7fb6b9e00640 1 -- 172.21.15.196:0/3126007439 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6bc163070 con 0x7fb6bc072560 2024-04-28T13:31:28.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.753+0000 7fd7d0400640 1 -- start start 2024-04-28T13:31:28.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7fcaa2c00640 1 -- 172.21.15.196:0/4074888475 shutdown_connections 2024-04-28T13:31:28.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.750+0000 7fcaa2c00640 1 -- 172.21.15.196:0/4074888475 wait complete. 2024-04-28T13:31:28.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.753+0000 7fcaa2c00640 1 Processor -- start 2024-04-28T13:31:28.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.753+0000 7f1f2de00640 1 -- 172.21.15.196:0/2613822165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f3015ed50 msgr2=0x7f1f300b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.753+0000 7fcaa2c00640 1 -- start start 2024-04-28T13:31:28.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.753+0000 7f1f2de00640 1 --2- 172.21.15.196:0/2613822165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f3015ed50 0x7f1f300b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.753+0000 7fb6b9e00640 1 --2- 172.21.15.196:0/3126007439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6bc072560 0x7fb6bc162b30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400efc0 tx=0x7fb6b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.754+0000 7f1f2de00640 1 -- 172.21.15.196:0/2613822165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f30152230 msgr2=0x7f1f300cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.754+0000 7f1f2de00640 1 --2- 172.21.15.196:0/2613822165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f30152230 0x7f1f300cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.754+0000 7f1f2de00640 1 -- 172.21.15.196:0/2613822165 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f300cc810 con 0x7f1f300b63a0 2024-04-28T13:31:28.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.754+0000 7fb6b2000640 1 -- 172.21.15.196:0/3126007439 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4018020 con 0x7fb6bc072560 2024-04-28T13:31:28.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.754+0000 7fb6b2000640 1 -- 172.21.15.196:0/3126007439 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b4014480 con 0x7fb6bc072560 2024-04-28T13:31:28.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.754+0000 7fb6b2000640 1 -- 172.21.15.196:0/3126007439 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4014850 con 0x7fb6bc072560 2024-04-28T13:31:28.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.754+0000 7f1f2d400640 1 --2- 172.21.15.196:0/2613822165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f30152230 0x7f1f300cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:31:28.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.755+0000 7f2c45a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c40072020 0x7f2c40162bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.755+0000 7f6f03a00640 1 Processor -- start 2024-04-28T13:31:28.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.755+0000 7f1f2de00640 1 --2- 172.21.15.196:0/2613822165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f300b63a0 0x7f1f300b3460 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000e9b0 tx=0x7f1f2000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.755+0000 7fd7d0400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b0004800 0x7fd7c8140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.755+0000 7f2c45a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400b3de0 0x7f2c400afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.755+0000 7fcaa2c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca9c072730 0x7fca9c0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.755+0000 7f2c45a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c4014c600 0x7f2c400ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.755+0000 7fd7d0400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7c81427a0 0x7fd7c8165100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.754+0000 7fb6c1200640 1 -- 172.21.15.196:0/3126007439 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6bc0ccac0 con 0x7fb6bc072560 2024-04-28T13:31:28.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.754+0000 7fb6c1200640 1 -- 172.21.15.196:0/3126007439 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6bc0cbf40 con 0x7fb6bc072560 2024-04-28T13:31:28.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.755+0000 7fcaa2c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca9c0d1460 0x7fca9c1486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.755+0000 7f2c45a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c400cff90 con 0x7f2c4014c600 2024-04-28T13:31:28.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.755+0000 7fd7d0400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7c815d960 0x7fd7c8148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.755+0000 7fd7d0400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c814d4d0 con 0x7fd7b0004800 2024-04-28T13:31:28.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.755+0000 7fd7d0400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c814d350 con 0x7fd7c815d960 2024-04-28T13:31:28.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.755+0000 7fd7d0400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c80b4fb0 con 0x7fd7c81427a0 2024-04-28T13:31:28.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.755+0000 7fcaa2c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c14c600 0x7fca9c148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7f2c45a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2c400cfa70 con 0x7f2c40072020 2024-04-28T13:31:28.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7f2c3ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c40072020 0x7f2c40162bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7f2c45a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2c400d1630 con 0x7f2c400b3de0 2024-04-28T13:31:28.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7f2c3ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c40072020 0x7f2c40162bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:39738/0 (socket says 172.21.15.196:39738) 2024-04-28T13:31:28.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7f2c3ea00640 1 -- 172.21.15.196:0/3350089994 learned_addr learned my addr 172.21.15.196:0/3350089994 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7fd7cda00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b0004800 0x7fd7c8140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7f6f03a00640 1 -- start start 2024-04-28T13:31:28.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7f2c3e000640 1 --2- 172.21.15.196:0/3350089994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400b3de0 0x7f2c400afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7f1f26000640 1 -- 172.21.15.196:0/2613822165 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20017070 con 0x7f1f300b63a0 2024-04-28T13:31:28.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7f1f26000640 1 -- 172.21.15.196:0/2613822165 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f2000c9a0 con 0x7f1f300b63a0 2024-04-28T13:31:28.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7fb6c1200640 1 -- 172.21.15.196:0/3126007439 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb694005290 con 0x7fb6bc072560 2024-04-28T13:31:28.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7fca9be00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca9c072730 0x7fca9c0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7fca9be00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca9c072730 0x7fca9c0c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:44852/0 (socket says 172.21.15.196:44852) 2024-04-28T13:31:28.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7fcaa0c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c14c600 0x7fca9c148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7fcaa0c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c14c600 0x7fca9c148be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35804/0 (socket says 172.21.15.196:35804) 2024-04-28T13:31:28.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7fca93400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca9c0d1460 0x7fca9c1486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7fcaa2c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c0cff90 con 0x7fca9c14c600 2024-04-28T13:31:28.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7fcaa2c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c0cfa70 con 0x7fca9c0d1460 2024-04-28T13:31:28.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7fcaa2c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fca9c0ceb50 con 0x7fca9c072730 2024-04-28T13:31:28.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7fca9be00640 1 -- 172.21.15.196:0/1609633609 learned_addr learned my addr 172.21.15.196:0/1609633609 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7fd7cd000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7c81427a0 0x7fd7c8165100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7fd7cd000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7c81427a0 0x7fd7c8165100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:44858/0 (socket says 172.21.15.196:44858) 2024-04-28T13:31:28.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7fd7cd000640 1 -- 172.21.15.196:0/1764461038 learned_addr learned my addr 172.21.15.196:0/1764461038 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7fd7ce400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7c815d960 0x7fd7c8148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7fabf8a00640 1 --2- 172.21.15.196:0/554471906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf4070570 0x7fabf40b5530 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fabec009d90 tx=0x7fabec02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b88b374695ae842 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7f1f26000640 1 -- 172.21.15.196:0/2613822165 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2000cd90 con 0x7f1f300b63a0 2024-04-28T13:31:28.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7f1f34e00640 1 -- 172.21.15.196:0/2613822165 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3014d480 con 0x7f1f300b63a0 2024-04-28T13:31:28.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7fabf2a00640 1 -- 172.21.15.196:0/554471906 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec007740 con 0x7fabf4070570 2024-04-28T13:31:28.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7f1f34e00640 1 -- 172.21.15.196:0/2613822165 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f3014da40 con 0x7f1f300b63a0 2024-04-28T13:31:28.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7f6f03a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d5640 0x7f6efc0cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7f6f03a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6efc15f4d0 0x7f6efc1f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7f6f03a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6efc159830 0x7f6efc1f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7f6f03a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc0d5a70 con 0x7f6efc159830 2024-04-28T13:31:28.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7f6f03a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc0b30d0 con 0x7f6efc15f4d0 2024-04-28T13:31:28.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7f6f03a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6efc0b4600 con 0x7f6efc0d5640 2024-04-28T13:31:28.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.756+0000 7f2c3f400640 1 --2- 172.21.15.196:0/3350089994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c4014c600 0x7f2c400ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7f2c3f400640 1 -- 172.21.15.196:0/3350089994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400b3de0 msgr2=0x7f2c400afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7f2c3f400640 1 --2- 172.21.15.196:0/3350089994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400b3de0 0x7f2c400afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7f2c3f400640 1 -- 172.21.15.196:0/3350089994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c40072020 msgr2=0x7f2c40162bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7fabf2a00640 1 -- 172.21.15.196:0/554471906 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabec0078e0 con 0x7fabf4070570 2024-04-28T13:31:28.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.757+0000 7fabf2a00640 1 -- 172.21.15.196:0/554471906 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec007c90 con 0x7fabf4070570 2024-04-28T13:31:28.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.758+0000 7f6f01000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d5640 0x7f6efc0cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.758+0000 7f6f01000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d5640 0x7f6efc0cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:44866/0 (socket says 172.21.15.196:44866) 2024-04-28T13:31:28.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.758+0000 7f6f01000640 1 -- 172.21.15.196:0/689372654 learned_addr learned my addr 172.21.15.196:0/689372654 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.758+0000 7fca9be00640 1 -- 172.21.15.196:0/1609633609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca9c0d1460 msgr2=0x7fca9c1486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.758+0000 7fca9be00640 1 --2- 172.21.15.196:0/1609633609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca9c0d1460 0x7fca9c1486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.758+0000 7fca9be00640 1 -- 172.21.15.196:0/1609633609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c14c600 msgr2=0x7fca9c148be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.758+0000 7fca9be00640 1 --2- 172.21.15.196:0/1609633609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c14c600 0x7fca9c148be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.758+0000 7fabfaa00640 1 -- 172.21.15.196:0/554471906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf4070570 msgr2=0x7fabf40b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.758+0000 7f6f01a00640 1 --2- 172.21.15.196:0/689372654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6efc159830 0x7f6efc1f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7f1f34e00640 1 -- 172.21.15.196:0/2613822165 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f300b6880 con 0x7f1f300b63a0 2024-04-28T13:31:28.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.758+0000 7f2c3f400640 1 --2- 172.21.15.196:0/3350089994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c40072020 0x7f2c40162bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.758+0000 7f2c3f400640 1 -- 172.21.15.196:0/3350089994 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c400b02d0 con 0x7f2c4014c600 2024-04-28T13:31:28.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7f2c3ea00640 1 --2- 172.21.15.196:0/3350089994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c40072020 0x7f2c40162bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:31:28.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.758+0000 7fd7cd000640 1 -- 172.21.15.196:0/1764461038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7c815d960 msgr2=0x7fd7c8148ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.758+0000 7fd7cd000640 1 --2- 172.21.15.196:0/1764461038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7c815d960 0x7fd7c8148ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7fd7cd000640 1 -- 172.21.15.196:0/1764461038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b0004800 msgr2=0x7fd7c8140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7fd7cd000640 1 --2- 172.21.15.196:0/1764461038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b0004800 0x7fd7c8140160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7fd7cd000640 1 -- 172.21.15.196:0/1764461038 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c8141eb0 con 0x7fd7c81427a0 2024-04-28T13:31:28.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.758+0000 7fca9be00640 1 -- 172.21.15.196:0/1609633609 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c0cc380 con 0x7fca9c072730 2024-04-28T13:31:28.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7fca9be00640 1 --2- 172.21.15.196:0/1609633609 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca9c072730 0x7fca9c0c9df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fca840097d0 tx=0x7fca840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.758+0000 7fabfaa00640 1 --2- 172.21.15.196:0/554471906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf4070570 0x7fabf40b5530 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fabec009d90 tx=0x7fabec02a6a0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7fabfaa00640 1 -- 172.21.15.196:0/554471906 shutdown_connections 2024-04-28T13:31:28.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7fabfaa00640 1 --2- 172.21.15.196:0/554471906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf4070570 0x7fabf40b5530 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7fabfaa00640 1 -- 172.21.15.196:0/554471906 >> 172.21.15.196:0/554471906 conn(0x7fabf40b4a10 msgr2=0x7fabf40b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:28.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7f6efbe00640 1 --2- 172.21.15.196:0/689372654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6efc15f4d0 0x7f6efc1f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7f6f01a00640 1 -- 172.21.15.196:0/689372654 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d5640 msgr2=0x7f6efc0cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7f6f01a00640 1 --2- 172.21.15.196:0/689372654 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d5640 0x7f6efc0cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7f6f01a00640 1 -- 172.21.15.196:0/689372654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6efc15f4d0 msgr2=0x7f6efc1f8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7f2c3f400640 1 --2- 172.21.15.196:0/3350089994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c4014c600 0x7f2c400ae4a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000c3f0 tx=0x7f2c3000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7fd7cd000640 1 --2- 172.21.15.196:0/1764461038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7c81427a0 0x7fd7c8165100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc01d480 tx=0x7fd7bc001970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7fabfaa00640 1 -- 172.21.15.196:0/554471906 shutdown_connections 2024-04-28T13:31:28.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7f6f01a00640 1 --2- 172.21.15.196:0/689372654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6efc15f4d0 0x7f6efc1f8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.759+0000 7f6f01a00640 1 -- 172.21.15.196:0/689372654 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6efc159cc0 con 0x7f6efc159830 2024-04-28T13:31:28.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.760+0000 7f6efbe00640 1 --2- 172.21.15.196:0/689372654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6efc15f4d0 0x7f6efc1f8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:28.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.760+0000 7fd7c5600640 1 -- 172.21.15.196:0/1764461038 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc001ca0 con 0x7fd7c81427a0 2024-04-28T13:31:28.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.760+0000 7f2c36a00640 1 -- 172.21.15.196:0/3350089994 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c30017070 con 0x7f2c4014c600 2024-04-28T13:31:28.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.760+0000 7fca98c00640 1 -- 172.21.15.196:0/1609633609 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84037070 con 0x7fca9c072730 2024-04-28T13:31:28.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.760+0000 7fca98c00640 1 -- 172.21.15.196:0/1609633609 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca84038040 con 0x7fca9c072730 2024-04-28T13:31:28.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.760+0000 7fca98c00640 1 -- 172.21.15.196:0/1609633609 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84004490 con 0x7fca9c072730 2024-04-28T13:31:28.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.760+0000 7f6f01a00640 1 --2- 172.21.15.196:0/689372654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6efc159830 0x7f6efc1f57a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400c760 tx=0x7f6ef400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.760+0000 7fb6b2000640 1 -- 172.21.15.196:0/3126007439 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6b4005d10 con 0x7fb6bc072560 2024-04-28T13:31:28.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.761+0000 7fb6b2000640 1 --2- 172.21.15.196:0/3126007439 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb69806c940 0x7fb69806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.762+0000 7fb6b2000640 1 -- 172.21.15.196:0/3126007439 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb6b408c760 con 0x7fb6bc072560 2024-04-28T13:31:28.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.760+0000 7f2c36a00640 1 -- 172.21.15.196:0/3350089994 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c30005ce0 con 0x7f2c4014c600 2024-04-28T13:31:28.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.762+0000 7f2c36a00640 1 -- 172.21.15.196:0/3350089994 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c30007310 con 0x7f2c4014c600 2024-04-28T13:31:28.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.762+0000 7f2c45a00640 1 -- 172.21.15.196:0/3350089994 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c400aecf0 con 0x7f2c4014c600 2024-04-28T13:31:28.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.762+0000 7f2c45a00640 1 -- 172.21.15.196:0/3350089994 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c400af2b0 con 0x7f2c4014c600 2024-04-28T13:31:28.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.760+0000 7fd7c5600640 1 -- 172.21.15.196:0/1764461038 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7bc007480 con 0x7fd7c81427a0 2024-04-28T13:31:28.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.760+0000 7fd7c5600640 1 -- 172.21.15.196:0/1764461038 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc004e40 con 0x7fd7c81427a0 2024-04-28T13:31:28.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.760+0000 7fd7d0400640 1 -- 172.21.15.196:0/1764461038 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c813fc80 con 0x7fd7c81427a0 2024-04-28T13:31:28.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.760+0000 7fd7d0400640 1 -- 172.21.15.196:0/1764461038 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c815df20 con 0x7fd7c81427a0 2024-04-28T13:31:28.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.761+0000 7fcaa2c00640 1 -- 172.21.15.196:0/1609633609 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9c0cc660 con 0x7fca9c072730 2024-04-28T13:31:28.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.762+0000 7fcaa2c00640 1 -- 172.21.15.196:0/1609633609 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9c130540 con 0x7fca9c072730 2024-04-28T13:31:28.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.760+0000 7fabfaa00640 1 -- 172.21.15.196:0/554471906 wait complete. 2024-04-28T13:31:28.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.762+0000 7fabfaa00640 1 Processor -- start 2024-04-28T13:31:28.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.762+0000 7fb6b9400640 1 --2- 172.21.15.196:0/3126007439 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb69806c940 0x7fb69806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.762+0000 7f6ef8c00640 1 -- 172.21.15.196:0/689372654 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4016020 con 0x7f6efc159830 2024-04-28T13:31:28.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.763+0000 7f6ef8c00640 1 -- 172.21.15.196:0/689372654 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef4006d50 con 0x7f6efc159830 2024-04-28T13:31:28.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.763+0000 7f6ef8c00640 1 -- 172.21.15.196:0/689372654 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4003bb0 con 0x7f6efc159830 2024-04-28T13:31:28.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.763+0000 7f6f03a00640 1 -- 172.21.15.196:0/689372654 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6efc0ca6f0 con 0x7f6efc159830 2024-04-28T13:31:28.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.763+0000 7f6f03a00640 1 -- 172.21.15.196:0/689372654 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6efc0cac90 con 0x7f6efc159830 2024-04-28T13:31:28.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.763+0000 7fd7d0400640 1 -- 172.21.15.196:0/1764461038 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c80693d0 con 0x7fd7c81427a0 2024-04-28T13:31:28.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.764+0000 7f2c45a00640 1 -- 172.21.15.196:0/3350089994 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c400ceb50 con 0x7f2c4014c600 2024-04-28T13:31:28.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.762+0000 7fabfaa00640 1 -- start start 2024-04-28T13:31:28.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.764+0000 7fabfaa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf4070570 0x7fabf41f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.764+0000 7fabfaa00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabf409ba40 0x7fabf40c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.764+0000 7fabfaa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabf415f6a0 0x7fabf4162e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.764+0000 7fabfaa00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fabf40d1b20 con 0x7fabf4070570 2024-04-28T13:31:28.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.764+0000 7fabfaa00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fabf40d19a0 con 0x7fabf415f6a0 2024-04-28T13:31:28.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.764+0000 7fabfaa00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fabf40d1ca0 con 0x7fabf409ba40 2024-04-28T13:31:28.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.763+0000 7f1f26000640 1 -- 172.21.15.196:0/2613822165 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f2001a070 con 0x7f1f300b63a0 2024-04-28T13:31:28.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.764+0000 7f1f26000640 1 --2- 172.21.15.196:0/2613822165 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1f1006c9f0 0x7f1f1006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.764+0000 7f1f26000640 1 -- 172.21.15.196:0/2613822165 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1f2008c210 con 0x7f1f300b63a0 2024-04-28T13:31:28.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.765+0000 7fabf8a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabf415f6a0 0x7fabf4162e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.765+0000 7fabf3400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabf409ba40 0x7fabf40c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.765+0000 7fabf3400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabf409ba40 0x7fabf40c9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:44882/0 (socket says 172.21.15.196:44882) 2024-04-28T13:31:28.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.765+0000 7fb6b9400640 1 --2- 172.21.15.196:0/3126007439 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb69806c940 0x7fb69806ee00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4034000 tx=0x7fb6a402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.765+0000 7fabf3400640 1 -- 172.21.15.196:0/1438843108 learned_addr learned my addr 172.21.15.196:0/1438843108 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:28.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.765+0000 7fabf3e00640 1 --2- 172.21.15.196:0/1438843108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf4070570 0x7fabf41f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.765+0000 7f6f03a00640 1 -- 172.21.15.196:0/689372654 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6efc06a290 con 0x7f6efc159830 2024-04-28T13:31:28.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.765+0000 7f1f2d400640 1 --2- 172.21.15.196:0/2613822165 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1f1006c9f0 0x7f1f1006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.765+0000 7fcaa2c00640 1 -- 172.21.15.196:0/1609633609 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9c0ccb80 con 0x7fca9c072730 2024-04-28T13:31:28.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.765+0000 7fabf3400640 1 -- 172.21.15.196:0/1438843108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabf415f6a0 msgr2=0x7fabf4162e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.765+0000 7fabf3400640 1 --2- 172.21.15.196:0/1438843108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabf415f6a0 0x7fabf4162e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.766+0000 7fabf3400640 1 -- 172.21.15.196:0/1438843108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf4070570 msgr2=0x7fabf41f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:28.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.766+0000 7fabf3400640 1 --2- 172.21.15.196:0/1438843108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf4070570 0x7fabf41f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:28.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.766+0000 7fabf3400640 1 -- 172.21.15.196:0/1438843108 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf40ca8d0 con 0x7fabf409ba40 2024-04-28T13:31:28.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.766+0000 7fabf3e00640 1 --2- 172.21.15.196:0/1438843108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf4070570 0x7fabf41f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:31:28.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.766+0000 7fabf3400640 1 --2- 172.21.15.196:0/1438843108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabf409ba40 0x7fabf40c9ff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fabe000b3f0 tx=0x7fabe000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.766+0000 7f1f2d400640 1 --2- 172.21.15.196:0/2613822165 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1f1006c9f0 0x7f1f1006eeb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c033000 tx=0x7f1f1c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.768+0000 7fb6b2000640 1 -- 172.21.15.196:0/3126007439 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb6b4022030 con 0x7fb6bc072560 2024-04-28T13:31:28.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.771+0000 7f2c36a00640 1 -- 172.21.15.196:0/3350089994 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c3001a070 con 0x7f2c4014c600 2024-04-28T13:31:28.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.772+0000 7f6ef8c00640 1 -- 172.21.15.196:0/689372654 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ef400a850 con 0x7f6efc159830 2024-04-28T13:31:28.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.772+0000 7f2c36a00640 1 --2- 172.21.15.196:0/3350089994 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2c2006c940 0x7f2c2006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.772+0000 7f2c36a00640 1 -- 172.21.15.196:0/3350089994 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2c3008b170 con 0x7f2c4014c600 2024-04-28T13:31:28.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.773+0000 7f2c3ea00640 1 --2- 172.21.15.196:0/3350089994 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2c2006c940 0x7f2c2006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.773+0000 7f6ef8c00640 1 --2- 172.21.15.196:0/689372654 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6edc06c940 0x7f6edc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.773+0000 7f6ef8c00640 1 -- 172.21.15.196:0/689372654 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6ef408c4d0 con 0x7f6efc159830 2024-04-28T13:31:28.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.774+0000 7f6f01000640 1 --2- 172.21.15.196:0/689372654 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6edc06c940 0x7f6edc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.774+0000 7f2c3ea00640 1 --2- 172.21.15.196:0/3350089994 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2c2006c940 0x7f2c2006ee00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800f370 tx=0x7f2c38009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.775+0000 7f6f01000640 1 --2- 172.21.15.196:0/689372654 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6edc06c940 0x7f6edc06ee00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000ab80 tx=0x7f6ef0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.776+0000 7f2c36a00640 1 -- 172.21.15.196:0/3350089994 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2c400ceb50 con 0x7f2c4014c600 2024-04-28T13:31:28.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.779+0000 7f1f26000640 1 -- 172.21.15.196:0/2613822165 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f300b6880 con 0x7f1f300b63a0 2024-04-28T13:31:28.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.781+0000 7f6ef8c00640 1 -- 172.21.15.196:0/689372654 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ef4020020 con 0x7f6efc159830 2024-04-28T13:31:28.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.780+0000 7fabebe00640 1 -- 172.21.15.196:0/1438843108 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0015020 con 0x7fabf409ba40 2024-04-28T13:31:28.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.780+0000 7fabebe00640 1 -- 172.21.15.196:0/1438843108 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe0011830 con 0x7fabf409ba40 2024-04-28T13:31:28.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.780+0000 7fabebe00640 1 -- 172.21.15.196:0/1438843108 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0011c20 con 0x7fabf409ba40 2024-04-28T13:31:28.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.781+0000 7fd7c5600640 1 -- 172.21.15.196:0/1764461038 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd7bc04c420 con 0x7fd7c81427a0 2024-04-28T13:31:28.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.782+0000 7fd7c5600640 1 --2- 172.21.15.196:0/1764461038 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7a806c920 0x7fd7a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.782+0000 7fd7c5600640 1 -- 172.21.15.196:0/1764461038 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd7bc0bdb40 con 0x7fd7c81427a0 2024-04-28T13:31:28.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.783+0000 7fca98c00640 1 -- 172.21.15.196:0/1609633609 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca8403e750 con 0x7fca9c072730 2024-04-28T13:31:28.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.781+0000 7fabfaa00640 1 -- 172.21.15.196:0/1438843108 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf40cabb0 con 0x7fabf409ba40 2024-04-28T13:31:28.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.781+0000 7fabfaa00640 1 -- 172.21.15.196:0/1438843108 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf40cb170 con 0x7fabf409ba40 2024-04-28T13:31:28.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.784+0000 7fca98c00640 1 --2- 172.21.15.196:0/1609633609 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fca7806c940 0x7fca7806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.784+0000 7fca98c00640 1 -- 172.21.15.196:0/1609633609 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fca840ad2a0 con 0x7fca9c072730 2024-04-28T13:31:28.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.784+0000 7fd7cda00640 1 --2- 172.21.15.196:0/1764461038 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7a806c920 0x7fd7a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.784+0000 7fca93400640 1 --2- 172.21.15.196:0/1609633609 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fca7806c940 0x7fca7806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.784+0000 7fabfaa00640 1 -- 172.21.15.196:0/1438843108 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf40cb370 con 0x7fabf409ba40 2024-04-28T13:31:28.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.785+0000 7fd7cda00640 1 --2- 172.21.15.196:0/1764461038 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7a806c920 0x7fd7a806ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80d1340 tx=0x7fd7b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.785+0000 7fca93400640 1 --2- 172.21.15.196:0/1609633609 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fca7806c940 0x7fca7806ee00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0cd760 tx=0x7fca8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.795+0000 7fd7c5600640 1 -- 172.21.15.196:0/1764461038 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd7bc05a070 con 0x7fd7c81427a0 2024-04-28T13:31:28.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.807+0000 7fca98c00640 1 -- 172.21.15.196:0/1609633609 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca9c0ccb80 con 0x7fca9c072730 2024-04-28T13:31:28.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.809+0000 7fabebe00640 1 -- 172.21.15.196:0/1438843108 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fabe0018020 con 0x7fabf409ba40 2024-04-28T13:31:28.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.811+0000 7fabebe00640 1 --2- 172.21.15.196:0/1438843108 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fabd406c940 0x7fabd406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:28.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.811+0000 7fabebe00640 1 -- 172.21.15.196:0/1438843108 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fabe008bdd0 con 0x7fabf409ba40 2024-04-28T13:31:28.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.812+0000 7fabf3e00640 1 --2- 172.21.15.196:0/1438843108 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fabd406c940 0x7fabd406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:28.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.813+0000 7fabf3e00640 1 --2- 172.21.15.196:0/1438843108 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fabd406c940 0x7fabd406ee00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fabe40011e0 tx=0x7fabe4007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:28.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:28.817+0000 7fabebe00640 1 -- 172.21.15.196:0/1438843108 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fabe001f020 con 0x7fabf409ba40 2024-04-28T13:31:29.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.177+0000 7f88d5400640 1 -- 172.21.15.196:0/1962924672 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f88d00b5200 con 0x7f88b4008c10 2024-04-28T13:31:29.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.177+0000 7f88c6a00640 1 -- 172.21.15.196:0/1962924672 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f88c0059ec0 con 0x7f88b4008c10 2024-04-28T13:31:29.179 INFO:teuthology.orchestra.run.smithi196.stdout:55834574853 2024-04-28T13:31:29.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.181+0000 7f88d5400640 1 -- 172.21.15.196:0/1962924672 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88b006c970 msgr2=0x7f88b006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.181+0000 7f88d5400640 1 --2- 172.21.15.196:0/1962924672 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88b006c970 0x7f88b006ee30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88bc035000 tx=0x7f88bc009030 comp rx=0 tx=0).stop 2024-04-28T13:31:29.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.182+0000 7f88d5400640 1 -- 172.21.15.196:0/1962924672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4008c10 msgr2=0x7f88d015fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.182+0000 7f88d5400640 1 --2- 172.21.15.196:0/1962924672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4008c10 0x7f88d015fc90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f88c000e7d0 tx=0x7f88c000ec90 comp rx=0 tx=0).stop 2024-04-28T13:31:29.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.182+0000 7f88d5400640 1 -- 172.21.15.196:0/1962924672 shutdown_connections 2024-04-28T13:31:29.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.182+0000 7f88d5400640 1 --2- 172.21.15.196:0/1962924672 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88b006c970 0x7f88b006ee30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.183+0000 7f88d5400640 1 --2- 172.21.15.196:0/1962924672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d00cc4d0 0x7f88d015a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.183+0000 7f88d5400640 1 --2- 172.21.15.196:0/1962924672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d00cc070 0x7f88d0159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.183+0000 7f88d5400640 1 --2- 172.21.15.196:0/1962924672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4008c10 0x7f88d015fc90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.183+0000 7f88d5400640 1 -- 172.21.15.196:0/1962924672 >> 172.21.15.196:0/1962924672 conn(0x7f88d00b4a10 msgr2=0x7f88d0070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:29.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.183+0000 7f88d5400640 1 -- 172.21.15.196:0/1962924672 shutdown_connections 2024-04-28T13:31:29.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.184+0000 7f88d5400640 1 -- 172.21.15.196:0/1962924672 wait complete. 2024-04-28T13:31:29.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.190+0000 7f2c45a00640 1 -- 172.21.15.196:0/3350089994 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f2c4006a290 con 0x7f2c4014c600 2024-04-28T13:31:29.192 INFO:teuthology.orchestra.run.smithi196.stdout:55834574853 2024-04-28T13:31:29.193 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.191+0000 7f2c36a00640 1 -- 172.21.15.196:0/3350089994 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2c30021020 con 0x7f2c4014c600 2024-04-28T13:31:29.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.195+0000 7f2c2be00640 1 -- 172.21.15.196:0/3350089994 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2c2006c940 msgr2=0x7f2c2006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.195+0000 7f2c2be00640 1 --2- 172.21.15.196:0/3350089994 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2c2006c940 0x7f2c2006ee00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800f370 tx=0x7f2c38009040 comp rx=0 tx=0).stop 2024-04-28T13:31:29.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.195+0000 7f2c2be00640 1 -- 172.21.15.196:0/3350089994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c4014c600 msgr2=0x7f2c400ae4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.195+0000 7f2c2be00640 1 --2- 172.21.15.196:0/3350089994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c4014c600 0x7f2c400ae4a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000c3f0 tx=0x7f2c3000c7f0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.195+0000 7f2c2be00640 1 -- 172.21.15.196:0/3350089994 shutdown_connections 2024-04-28T13:31:29.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.195+0000 7f2c2be00640 1 --2- 172.21.15.196:0/3350089994 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2c2006c940 0x7f2c2006ee00 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800f370 tx=0x7f2c38009040 comp rx=0 tx=0).stop 2024-04-28T13:31:29.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.195+0000 7f2c2be00640 1 --2- 172.21.15.196:0/3350089994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c4014c600 0x7f2c400ae4a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.195+0000 7f2c2be00640 1 --2- 172.21.15.196:0/3350089994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400b3de0 0x7f2c400afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.195+0000 7f2c2be00640 1 --2- 172.21.15.196:0/3350089994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c40072020 0x7f2c40162bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.195+0000 7f2c2be00640 1 -- 172.21.15.196:0/3350089994 >> 172.21.15.196:0/3350089994 conn(0x7f2c400cd830 msgr2=0x7f2c400d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:29.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.195+0000 7f2c2be00640 1 -- 172.21.15.196:0/3350089994 shutdown_connections 2024-04-28T13:31:29.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.195+0000 7f2c2be00640 1 -- 172.21.15.196:0/3350089994 wait complete. 2024-04-28T13:31:29.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-04-28T13:31:29.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.212+0000 7fb6c1200640 1 -- 172.21.15.196:0/3126007439 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb694005580 con 0x7fb6bc072560 2024-04-28T13:31:29.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.212+0000 7fb6b2000640 1 -- 172.21.15.196:0/3126007439 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb6bc163070 con 0x7fb6bc072560 2024-04-28T13:31:29.214 INFO:teuthology.orchestra.run.smithi196.stdout:55834574854 2024-04-28T13:31:29.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.215+0000 7f6f03a00640 1 -- 172.21.15.196:0/689372654 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6efc069570 con 0x7f6efc159830 2024-04-28T13:31:29.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.215+0000 7f6ef8c00640 1 -- 172.21.15.196:0/689372654 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6ef405a970 con 0x7f6efc159830 2024-04-28T13:31:29.216 INFO:teuthology.orchestra.run.smithi196.stdout:55834574853 2024-04-28T13:31:29.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.217+0000 7fb6c1200640 1 -- 172.21.15.196:0/3126007439 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb69806c940 msgr2=0x7fb69806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.217+0000 7fb6c1200640 1 --2- 172.21.15.196:0/3126007439 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb69806c940 0x7fb69806ee00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4034000 tx=0x7fb6a402e000 comp rx=0 tx=0).stop 2024-04-28T13:31:29.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.217+0000 7fb6c1200640 1 -- 172.21.15.196:0/3126007439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6bc072560 msgr2=0x7fb6bc162b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.217+0000 7fb6c1200640 1 --2- 172.21.15.196:0/3126007439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6bc072560 0x7fb6bc162b30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400efc0 tx=0x7fb6b400c490 comp rx=0 tx=0).stop 2024-04-28T13:31:29.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.218+0000 7fb6c1200640 1 -- 172.21.15.196:0/3126007439 shutdown_connections 2024-04-28T13:31:29.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.218+0000 7fb6c1200640 1 --2- 172.21.15.196:0/3126007439 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb69806c940 0x7fb69806ee00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.218+0000 7fb6c1200640 1 --2- 172.21.15.196:0/3126007439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6bc14c600 0x7fb6bc1e9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.218+0000 7fb6c1200640 1 --2- 172.21.15.196:0/3126007439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6bc0d1460 0x7fb6bc0cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.218+0000 7fb6c1200640 1 --2- 172.21.15.196:0/3126007439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6bc072560 0x7fb6bc162b30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.218+0000 7fb6c1200640 1 -- 172.21.15.196:0/3126007439 >> 172.21.15.196:0/3126007439 conn(0x7fb6bc0cd830 msgr2=0x7fb6bc0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:29.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.218+0000 7f6f03a00640 1 -- 172.21.15.196:0/689372654 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6edc06c940 msgr2=0x7f6edc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.218+0000 7f6f03a00640 1 --2- 172.21.15.196:0/689372654 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6edc06c940 0x7f6edc06ee00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000ab80 tx=0x7f6ef0008040 comp rx=0 tx=0).stop 2024-04-28T13:31:29.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.218+0000 7f6f03a00640 1 -- 172.21.15.196:0/689372654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6efc159830 msgr2=0x7f6efc1f57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.220 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.218+0000 7f6f03a00640 1 --2- 172.21.15.196:0/689372654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6efc159830 0x7f6efc1f57a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400c760 tx=0x7f6ef400cc20 comp rx=0 tx=0).stop 2024-04-28T13:31:29.220 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.218+0000 7fb6c1200640 1 -- 172.21.15.196:0/3126007439 shutdown_connections 2024-04-28T13:31:29.220 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.218+0000 7fb6c1200640 1 -- 172.21.15.196:0/3126007439 wait complete. 2024-04-28T13:31:29.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.219+0000 7f6f03a00640 1 -- 172.21.15.196:0/689372654 shutdown_connections 2024-04-28T13:31:29.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.219+0000 7f6f03a00640 1 --2- 172.21.15.196:0/689372654 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6edc06c940 0x7f6edc06ee00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.219+0000 7f6f03a00640 1 --2- 172.21.15.196:0/689372654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6efc159830 0x7f6efc1f57a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.219+0000 7f6f03a00640 1 --2- 172.21.15.196:0/689372654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6efc15f4d0 0x7f6efc1f8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.219+0000 7f6f03a00640 1 --2- 172.21.15.196:0/689372654 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d5640 0x7f6efc0cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.219+0000 7f6f03a00640 1 -- 172.21.15.196:0/689372654 >> 172.21.15.196:0/689372654 conn(0x7f6efc08ebf0 msgr2=0x7f6efc09e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:29.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.219+0000 7f6f03a00640 1 -- 172.21.15.196:0/689372654 shutdown_connections 2024-04-28T13:31:29.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.219+0000 7f6f03a00640 1 -- 172.21.15.196:0/689372654 wait complete. 2024-04-28T13:31:29.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-04-28T13:31:29.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.223+0000 7fd7d0400640 1 -- 172.21.15.196:0/1764461038 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd7c80d1bd0 con 0x7fd7c81427a0 2024-04-28T13:31:29.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.224+0000 7fd7c5600640 1 -- 172.21.15.196:0/1764461038 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd7bc08b900 con 0x7fd7c81427a0 2024-04-28T13:31:29.225 INFO:teuthology.orchestra.run.smithi196.stdout:55834574855 2024-04-28T13:31:29.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.224+0000 7f1f34e00640 1 -- 172.21.15.196:0/2613822165 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1f3006a290 con 0x7f1f300b63a0 2024-04-28T13:31:29.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.225+0000 7f1f26000640 1 -- 172.21.15.196:0/2613822165 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1f2005a060 con 0x7f1f300b63a0 2024-04-28T13:31:29.226 INFO:teuthology.orchestra.run.smithi196.stdout:55834574852 2024-04-28T13:31:29.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.229+0000 7fd7d0400640 1 -- 172.21.15.196:0/1764461038 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7a806c920 msgr2=0x7fd7a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.229+0000 7fd7d0400640 1 --2- 172.21.15.196:0/1764461038 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7a806c920 0x7fd7a806ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80d1340 tx=0x7fd7b4008040 comp rx=0 tx=0).stop 2024-04-28T13:31:29.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.229+0000 7fd7d0400640 1 -- 172.21.15.196:0/1764461038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7c81427a0 msgr2=0x7fd7c8165100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.229+0000 7fd7d0400640 1 --2- 172.21.15.196:0/1764461038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7c81427a0 0x7fd7c8165100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc01d480 tx=0x7fd7bc001970 comp rx=0 tx=0).stop 2024-04-28T13:31:29.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.229+0000 7f1f34e00640 1 -- 172.21.15.196:0/2613822165 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1f1006c9f0 msgr2=0x7f1f1006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.229+0000 7f1f34e00640 1 --2- 172.21.15.196:0/2613822165 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1f1006c9f0 0x7f1f1006eeb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c033000 tx=0x7f1f1c02e000 comp rx=0 tx=0).stop 2024-04-28T13:31:29.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.229+0000 7f1f34e00640 1 -- 172.21.15.196:0/2613822165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f300b63a0 msgr2=0x7f1f300b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.229+0000 7f1f34e00640 1 --2- 172.21.15.196:0/2613822165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f300b63a0 0x7f1f300b3460 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000e9b0 tx=0x7f1f2000ee70 comp rx=0 tx=0).stop 2024-04-28T13:31:29.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.229+0000 7f1f34e00640 1 -- 172.21.15.196:0/2613822165 shutdown_connections 2024-04-28T13:31:29.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.229+0000 7f1f34e00640 1 --2- 172.21.15.196:0/2613822165 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1f1006c9f0 0x7f1f1006eeb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.229+0000 7f1f34e00640 1 --2- 172.21.15.196:0/2613822165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f3015ed50 0x7f1f300b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.229+0000 7f1f34e00640 1 --2- 172.21.15.196:0/2613822165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f30152230 0x7f1f300cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.229+0000 7f1f34e00640 1 --2- 172.21.15.196:0/2613822165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f300b63a0 0x7f1f300b3460 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.229+0000 7f1f34e00640 1 -- 172.21.15.196:0/2613822165 >> 172.21.15.196:0/2613822165 conn(0x7f1f300b2ae0 msgr2=0x7f1f300735b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:29.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.230+0000 7fd7d0400640 1 -- 172.21.15.196:0/1764461038 shutdown_connections 2024-04-28T13:31:29.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.230+0000 7fd7d0400640 1 --2- 172.21.15.196:0/1764461038 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7a806c920 0x7fd7a806ede0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.230+0000 7fd7d0400640 1 --2- 172.21.15.196:0/1764461038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7c815d960 0x7fd7c8148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.230+0000 7fd7d0400640 1 --2- 172.21.15.196:0/1764461038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7c81427a0 0x7fd7c8165100 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.230+0000 7fd7d0400640 1 --2- 172.21.15.196:0/1764461038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b0004800 0x7fd7c8140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.230+0000 7fd7d0400640 1 -- 172.21.15.196:0/1764461038 >> 172.21.15.196:0/1764461038 conn(0x7fd7c806f050 msgr2=0x7fd7c8073100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:29.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.230+0000 7fd7d0400640 1 -- 172.21.15.196:0/1764461038 shutdown_connections 2024-04-28T13:31:29.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.230+0000 7f1f34e00640 1 -- 172.21.15.196:0/2613822165 shutdown_connections 2024-04-28T13:31:29.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.230+0000 7fd7d0400640 1 -- 172.21.15.196:0/1764461038 wait complete. 2024-04-28T13:31:29.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.230+0000 7f1f34e00640 1 -- 172.21.15.196:0/2613822165 wait complete. 2024-04-28T13:31:29.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-04-28T13:31:29.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.241+0000 7fcaa2c00640 1 -- 172.21.15.196:0/1609633609 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fca9c06a290 con 0x7fca9c072730 2024-04-28T13:31:29.244 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-04-28T13:31:29.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.242+0000 7fca98c00640 1 -- 172.21.15.196:0/1609633609 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fca8407b6b0 con 0x7fca9c072730 2024-04-28T13:31:29.244 INFO:teuthology.orchestra.run.smithi196.stdout:55834574854 2024-04-28T13:31:29.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.245+0000 7fcaa2c00640 1 -- 172.21.15.196:0/1609633609 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fca7806c940 msgr2=0x7fca7806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.245+0000 7fcaa2c00640 1 --2- 172.21.15.196:0/1609633609 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fca7806c940 0x7fca7806ee00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0cd760 tx=0x7fca8c009040 comp rx=0 tx=0).stop 2024-04-28T13:31:29.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.245+0000 7fcaa2c00640 1 -- 172.21.15.196:0/1609633609 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca9c072730 msgr2=0x7fca9c0c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.245+0000 7fcaa2c00640 1 --2- 172.21.15.196:0/1609633609 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca9c072730 0x7fca9c0c9df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fca840097d0 tx=0x7fca840046c0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.245+0000 7fcaa2c00640 1 -- 172.21.15.196:0/1609633609 shutdown_connections 2024-04-28T13:31:29.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.245+0000 7fcaa2c00640 1 --2- 172.21.15.196:0/1609633609 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fca7806c940 0x7fca7806ee00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.245+0000 7fcaa2c00640 1 --2- 172.21.15.196:0/1609633609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca9c14c600 0x7fca9c148be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.245+0000 7fcaa2c00640 1 --2- 172.21.15.196:0/1609633609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca9c0d1460 0x7fca9c1486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.245+0000 7fcaa2c00640 1 --2- 172.21.15.196:0/1609633609 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca9c072730 0x7fca9c0c9df0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.245+0000 7fcaa2c00640 1 -- 172.21.15.196:0/1609633609 >> 172.21.15.196:0/1609633609 conn(0x7fca9c0cd830 msgr2=0x7fca9c14b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:29.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.245+0000 7fcaa2c00640 1 -- 172.21.15.196:0/1609633609 shutdown_connections 2024-04-28T13:31:29.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.245+0000 7fcaa2c00640 1 -- 172.21.15.196:0/1609633609 wait complete. 2024-04-28T13:31:29.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2024-04-28T13:31:29.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.1 2024-04-28T13:31:29.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.252+0000 7fabfaa00640 1 -- 172.21.15.196:0/1438843108 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fabf406f1d0 con 0x7fabf409ba40 2024-04-28T13:31:29.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.252+0000 7fabebe00640 1 -- 172.21.15.196:0/1438843108 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fabe0059c90 con 0x7fabf409ba40 2024-04-28T13:31:29.253 INFO:teuthology.orchestra.run.smithi196.stdout:55834574854 2024-04-28T13:31:29.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.255+0000 7fabfaa00640 1 -- 172.21.15.196:0/1438843108 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fabd406c940 msgr2=0x7fabd406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.255+0000 7fabfaa00640 1 --2- 172.21.15.196:0/1438843108 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fabd406c940 0x7fabd406ee00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fabe40011e0 tx=0x7fabe4007d30 comp rx=0 tx=0).stop 2024-04-28T13:31:29.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.255+0000 7fabfaa00640 1 -- 172.21.15.196:0/1438843108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabf409ba40 msgr2=0x7fabf40c9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:29.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.255+0000 7fabfaa00640 1 --2- 172.21.15.196:0/1438843108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabf409ba40 0x7fabf40c9ff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fabe000b3f0 tx=0x7fabe000b8b0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.255+0000 7fabfaa00640 1 -- 172.21.15.196:0/1438843108 shutdown_connections 2024-04-28T13:31:29.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.255+0000 7fabfaa00640 1 --2- 172.21.15.196:0/1438843108 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fabd406c940 0x7fabd406ee00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.255+0000 7fabfaa00640 1 --2- 172.21.15.196:0/1438843108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabf415f6a0 0x7fabf4162e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.255+0000 7fabfaa00640 1 --2- 172.21.15.196:0/1438843108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabf409ba40 0x7fabf40c9ff0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.255+0000 7fabfaa00640 1 --2- 172.21.15.196:0/1438843108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf4070570 0x7fabf41f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:29.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.255+0000 7fabfaa00640 1 -- 172.21.15.196:0/1438843108 >> 172.21.15.196:0/1438843108 conn(0x7fabf40b4a10 msgr2=0x7fabf406d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:29.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.255+0000 7fabfaa00640 1 -- 172.21.15.196:0/1438843108 shutdown_connections 2024-04-28T13:31:29.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:29.255+0000 7fabfaa00640 1 -- 172.21.15.196:0/1438843108 wait complete. 2024-04-28T13:31:29.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-04-28T13:31:29.274 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-04-28T13:31:30.214 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-28T13:31:30.224 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-28T13:31:30.241 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-28T13:31:30.245 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-28T13:31:30.251 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-28T13:31:30.254 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-28T13:31:30.269 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-28T13:31:30.276 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-28T13:31:30.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.487+0000 7fe7c7000640 1 Processor -- start 2024-04-28T13:31:30.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.487+0000 7fe7c7000640 1 -- start start 2024-04-28T13:31:30.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.489+0000 7fe7c7000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe7b803c090 0x7fe7b80e7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.489+0000 7fe7c7000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7b8053ca0 0x7fe7b8054130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.489+0000 7fe7c7000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7b8054670 con 0x7fe7b80e3150 2024-04-28T13:31:30.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.489+0000 7fe7c7000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b8007b60 con 0x7fe7b803c090 2024-04-28T13:31:30.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.489+0000 7fe7c7000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b8058800 con 0x7fe7b8053ca0 2024-04-28T13:31:30.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.490+0000 7fe7bfe00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fe7b80e3150 0x7fe7b8058040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56110/0 (socket says 172.21.15.196:56110) 2024-04-28T13:31:30.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.490+0000 7fe7bfe00640 1 -- 172.21.15.196:0/2032455624 learned_addr learned my addr 172.21.15.196:0/2032455624 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:30.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.491+0000 7fe7c5000640 1 --2- 172.21.15.196:0/2032455624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7b8053ca0 0x7fe7b8054130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.492+0000 7fe7bea00640 1 -- 172.21.15.196:0/2032455624 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1809961289 0 0) 0x7fe7b8054670 con 0x7fe7b80e3150 2024-04-28T13:31:30.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.492+0000 7fe7bea00640 1 -- 172.21.15.196:0/2032455624 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7a8003520 con 0x7fe7b80e3150 2024-04-28T13:31:30.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.492+0000 7fe7c5000640 1 -- 172.21.15.196:0/2032455624 >> v2:172.21.15.203:3300/0 conn(0x7fe7b803c090 msgr2=0x7fe7b80e7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:30.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.492+0000 7fe7c5000640 1 --2- 172.21.15.196:0/2032455624 >> v2:172.21.15.203:3300/0 conn(0x7fe7b803c090 0x7fe7b80e7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.492+0000 7fe7c5000640 1 -- 172.21.15.196:0/2032455624 >> v1:172.21.15.196:6789/0 conn(0x7fe7b80e3150 legacy=0x7fe7b8058040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.492+0000 7fe7c5000640 1 -- 172.21.15.196:0/2032455624 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b8058ae0 con 0x7fe7b8053ca0 2024-04-28T13:31:30.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.494+0000 7fe7c5000640 1 --2- 172.21.15.196:0/2032455624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7b8053ca0 0x7fe7b8054130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe7c006b9c0 tx=0x7fe7c00a6f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a84d3dea2d2c581 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.495+0000 7fe7bea00640 1 -- 172.21.15.196:0/2032455624 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00b4070 con 0x7fe7b8053ca0 2024-04-28T13:31:30.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.495+0000 7fe7bea00640 1 -- 172.21.15.196:0/2032455624 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c00a7db0 con 0x7fe7b8053ca0 2024-04-28T13:31:30.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.495+0000 7fe7bea00640 1 -- 172.21.15.196:0/2032455624 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00af640 con 0x7fe7b8053ca0 2024-04-28T13:31:30.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.497+0000 7fe7c7000640 1 -- 172.21.15.196:0/2032455624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7b8053ca0 msgr2=0x7fe7b8054130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.497+0000 7fe7c7000640 1 --2- 172.21.15.196:0/2032455624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7b8053ca0 0x7fe7b8054130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe7c006b9c0 tx=0x7fe7c00a6f60 comp rx=0 tx=0).stop 2024-04-28T13:31:30.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.497+0000 7fe7c7000640 1 -- 172.21.15.196:0/2032455624 shutdown_connections 2024-04-28T13:31:30.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.497+0000 7fe7c7000640 1 --2- 172.21.15.196:0/2032455624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7b8053ca0 0x7fe7b8054130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.497+0000 7fe7c7000640 1 --2- 172.21.15.196:0/2032455624 >> v2:172.21.15.203:3300/0 conn(0x7fe7b803c090 0x7fe7b80e7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.498+0000 7fe7c7000640 1 -- 172.21.15.196:0/2032455624 >> 172.21.15.196:0/2032455624 conn(0x7fe7b8058ed0 msgr2=0x7fe7b80592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:30.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.498+0000 7fe7c7000640 1 -- 172.21.15.196:0/2032455624 shutdown_connections 2024-04-28T13:31:30.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.498+0000 7fe7c7000640 1 -- 172.21.15.196:0/2032455624 wait complete. 2024-04-28T13:31:30.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.502+0000 7fe7c7000640 1 Processor -- start 2024-04-28T13:31:30.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.502+0000 7fe7c7000640 1 -- start start 2024-04-28T13:31:30.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.504+0000 7fe7c7000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7b803c090 0x7fe7b804ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.505+0000 7fe7c7000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7b8053ca0 0x7fe7b816cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.506 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.505+0000 7fe7c7000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7b80e3150 0x7fe7b816d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.507 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.505+0000 7fe7bfe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7b803c090 0x7fe7b804ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.507 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.505+0000 7fe7bfe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7b803c090 0x7fe7b804ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:44902/0 (socket says 172.21.15.196:44902) 2024-04-28T13:31:30.507 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.505+0000 7fe7bf400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7b8053ca0 0x7fe7b816cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.507 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.506+0000 7fe7bf400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7b8053ca0 0x7fe7b816cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35830/0 (socket says 172.21.15.196:35830) 2024-04-28T13:31:30.507 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.506+0000 7fe7bfe00640 1 -- 172.21.15.196:0/2235858608 learned_addr learned my addr 172.21.15.196:0/2235858608 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:30.507 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.506+0000 7fe7c7000640 1 -- 172.21.15.196:0/2235858608 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b80e3530 con 0x7fe7b8053ca0 2024-04-28T13:31:30.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.506+0000 7fe7c7000640 1 -- 172.21.15.196:0/2235858608 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8006630 con 0x7fe7b80e3150 2024-04-28T13:31:30.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.507+0000 7fe7c7000640 1 -- 172.21.15.196:0/2235858608 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b8006b50 con 0x7fe7b803c090 2024-04-28T13:31:30.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.507+0000 7fe7c5000640 1 --2- 172.21.15.196:0/2235858608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7b80e3150 0x7fe7b816d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.508+0000 7fe7bf400640 1 -- 172.21.15.196:0/2235858608 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7b803c090 msgr2=0x7fe7b804ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.508+0000 7fe7bf400640 1 --2- 172.21.15.196:0/2235858608 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7b803c090 0x7fe7b804ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.508+0000 7fe7bf400640 1 -- 172.21.15.196:0/2235858608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7b80e3150 msgr2=0x7fe7b816d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.508+0000 7fe7bf400640 1 --2- 172.21.15.196:0/2235858608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7b80e3150 0x7fe7b816d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.508+0000 7fe7bf400640 1 -- 172.21.15.196:0/2235858608 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b80507f0 con 0x7fe7b8053ca0 2024-04-28T13:31:30.510 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.509+0000 7fe7bf400640 1 --2- 172.21.15.196:0/2235858608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7b8053ca0 0x7fe7b816cad0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00b420 tx=0x7fe7ac00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.510+0000 7fe7b3e00640 1 -- 172.21.15.196:0/2235858608 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac011830 con 0x7fe7b8053ca0 2024-04-28T13:31:30.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.510+0000 7fe7b3e00640 1 -- 172.21.15.196:0/2235858608 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ac016070 con 0x7fe7b8053ca0 2024-04-28T13:31:30.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.510+0000 7fe7b3e00640 1 -- 172.21.15.196:0/2235858608 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac011c10 con 0x7fe7b8053ca0 2024-04-28T13:31:30.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.510+0000 7fe7c7000640 1 -- 172.21.15.196:0/2235858608 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b8050ad0 con 0x7fe7b8053ca0 2024-04-28T13:31:30.513 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.512+0000 7fe7c7000640 1 -- 172.21.15.196:0/2235858608 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b8051090 con 0x7fe7b8053ca0 2024-04-28T13:31:30.516 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.513+0000 7fe7c7000640 1 -- 172.21.15.196:0/2235858608 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b8005380 con 0x7fe7b8053ca0 2024-04-28T13:31:30.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.516+0000 7fe7b3e00640 1 -- 172.21.15.196:0/2235858608 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7ac01b460 con 0x7fe7b8053ca0 2024-04-28T13:31:30.525 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.525+0000 7fe7b3e00640 1 --2- 172.21.15.196:0/2235858608 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe7a006c940 0x7fe7a006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.525+0000 7fe7b3e00640 1 -- 172.21.15.196:0/2235858608 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe7ac056cc0 con 0x7fe7b8053ca0 2024-04-28T13:31:30.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.526+0000 7fe7bfe00640 1 --2- 172.21.15.196:0/2235858608 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe7a006c940 0x7fe7a006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.529+0000 7fe7bfe00640 1 --2- 172.21.15.196:0/2235858608 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe7a006c940 0x7fe7a006ee00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4000f60 tx=0x7fe7b4007e10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.530+0000 7fa29f200640 1 Processor -- start 2024-04-28T13:31:30.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.530+0000 7fe7b3e00640 1 -- 172.21.15.196:0/2235858608 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe7ac058c20 con 0x7fe7b8053ca0 2024-04-28T13:31:30.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.530+0000 7fa29f200640 1 -- start start 2024-04-28T13:31:30.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.532+0000 7fa29f200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa2a01520d0 0x7fa2a00ae940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.532+0000 7fa29f200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a00b4ed0 con 0x7fa2a015f490 2024-04-28T13:31:30.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.532+0000 7fa29f200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a00b51b0 con 0x7fa2a00b5f70 2024-04-28T13:31:30.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.532+0000 7fa29f200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a00b5490 con 0x7fa2a01520d0 2024-04-28T13:31:30.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.533+0000 7fa29de00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa2a01520d0 0x7fa2a00ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.533+0000 7fa29e800640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa2a015f490 0x7fa2a0165110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56112/0 (socket says 172.21.15.196:56112) 2024-04-28T13:31:30.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.533+0000 7fa29e800640 1 -- 172.21.15.196:0/3975620535 learned_addr learned my addr 172.21.15.196:0/3975620535 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:30.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.534+0000 7fa29ca00640 1 -- 172.21.15.196:0/3975620535 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2618109548 0 0) 0x7fa2a00b4ed0 con 0x7fa2a015f490 2024-04-28T13:31:30.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.534+0000 7fa29ca00640 1 -- 172.21.15.196:0/3975620535 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa284003520 con 0x7fa2a015f490 2024-04-28T13:31:30.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.535+0000 7fa29ca00640 1 -- 172.21.15.196:0/3975620535 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 89374559 0 0) 0x7fa284003520 con 0x7fa2a015f490 2024-04-28T13:31:30.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.535+0000 7fa29ca00640 1 -- 172.21.15.196:0/3975620535 >> v2:172.21.15.203:3300/0 conn(0x7fa2a01520d0 msgr2=0x7fa2a00ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:30.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.535+0000 7fa29ca00640 1 --2- 172.21.15.196:0/3975620535 >> v2:172.21.15.203:3300/0 conn(0x7fa2a01520d0 0x7fa2a00ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.535+0000 7fa29ca00640 1 -- 172.21.15.196:0/3975620535 >> v1:172.21.15.203:6789/0 conn(0x7fa2a00b5f70 legacy=0x7fa2a015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.535+0000 7fa29ca00640 1 -- 172.21.15.196:0/3975620535 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a00b5770 con 0x7fa2a015f490 2024-04-28T13:31:30.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.535+0000 7fa29ca00640 1 -- 172.21.15.196:0/3975620535 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fa28c001670 con 0x7fa2a015f490 2024-04-28T13:31:30.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.535+0000 7fa29ca00640 1 -- 172.21.15.196:0/3975620535 >> v1:172.21.15.196:6789/0 conn(0x7fa2a015f490 legacy=0x7fa2a0165110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.535+0000 7fa29ca00640 1 --2- 172.21.15.196:0/3975620535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2840039b0 0x7fa284003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.535+0000 7fa29ca00640 1 --2- 172.21.15.196:0/3975620535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa284004800 0x7fa284004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.535+0000 7fa29ca00640 1 --2- 172.21.15.196:0/3975620535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2840051c0 0x7fa28400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.535+0000 7fa29ca00640 1 -- 172.21.15.196:0/3975620535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a00b5490 con 0x7fa2840051c0 2024-04-28T13:31:30.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.535+0000 7fa29ca00640 1 -- 172.21.15.196:0/3975620535 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa28400c060 con 0x7fa2840039b0 2024-04-28T13:31:30.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.535+0000 7fa29ca00640 1 -- 172.21.15.196:0/3975620535 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa28400c260 con 0x7fa284004800 2024-04-28T13:31:30.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.536+0000 7fa29e800640 1 --2- 172.21.15.196:0/3975620535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2840051c0 0x7fa28400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.536+0000 7fa29de00640 1 --2- 172.21.15.196:0/3975620535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2840039b0 0x7fa284003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.536+0000 7fa29e800640 1 -- 172.21.15.196:0/3975620535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa284004800 msgr2=0x7fa284004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T13:31:30.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.536+0000 7fa29e800640 1 --2- 172.21.15.196:0/3975620535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa284004800 0x7fa284004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.536+0000 7fa29e800640 1 -- 172.21.15.196:0/3975620535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2840039b0 msgr2=0x7fa284003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:30.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.536+0000 7fa29e800640 1 --2- 172.21.15.196:0/3975620535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2840039b0 0x7fa284003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.536+0000 7fa29e800640 1 -- 172.21.15.196:0/3975620535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a00b5770 con 0x7fa2840051c0 2024-04-28T13:31:30.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.538+0000 7fa29e800640 1 -- 172.21.15.196:0/3975620535 reap_dead start 2024-04-28T13:31:30.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.538+0000 7fa29e800640 1 --2- 172.21.15.196:0/3975620535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2840051c0 0x7fa28400b9c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa28c007fd0 tx=0x7fa284005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77f6d98bfedc2a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.538+0000 7efd80a00640 1 Processor -- start 2024-04-28T13:31:30.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.539+0000 7efd80a00640 1 -- start start 2024-04-28T13:31:30.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.540+0000 7fa29ca00640 1 -- 172.21.15.196:0/3975620535 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c007d50 con 0x7fa2840051c0 2024-04-28T13:31:30.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.540+0000 7fa29ca00640 1 -- 172.21.15.196:0/3975620535 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa28c005ad0 con 0x7fa2840051c0 2024-04-28T13:31:30.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.540+0000 7fa29ca00640 1 -- 172.21.15.196:0/3975620535 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c007070 con 0x7fa2840051c0 2024-04-28T13:31:30.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.541+0000 7efd80a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd740ebb50 0x7efd74055300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.541+0000 7fa29f200640 1 -- 172.21.15.196:0/3975620535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2840051c0 msgr2=0x7fa28400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.541+0000 7fa29f200640 1 --2- 172.21.15.196:0/3975620535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2840051c0 0x7fa28400b9c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa28c007fd0 tx=0x7fa284005830 comp rx=0 tx=0).stop 2024-04-28T13:31:30.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.542+0000 7fa29f200640 1 -- 172.21.15.196:0/3975620535 shutdown_connections 2024-04-28T13:31:30.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.542+0000 7fa29f200640 1 --2- 172.21.15.196:0/3975620535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2840051c0 0x7fa28400b9c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.542+0000 7fa29f200640 1 -- 172.21.15.196:0/3975620535 >> 172.21.15.196:0/3975620535 conn(0x7fa2a00b2990 msgr2=0x7fa2a00b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:30.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.542+0000 7efd80a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efd740061a0 0x7efd74006630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.542+0000 7efd80a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd740387e0 con 0x7efd740eb060 2024-04-28T13:31:30.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.542+0000 7efd80a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7efd74057e30 con 0x7efd740061a0 2024-04-28T13:31:30.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.542+0000 7efd80a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efd740ebfd0 con 0x7efd740ebb50 2024-04-28T13:31:30.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.542+0000 7efd7be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efd740061a0 0x7efd74006630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.543+0000 7fa29f200640 1 -- 172.21.15.196:0/3975620535 shutdown_connections 2024-04-28T13:31:30.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.543+0000 7fa29f200640 1 -- 172.21.15.196:0/3975620535 wait complete. 2024-04-28T13:31:30.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.543+0000 7efd7b400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7efd740eb060 0x7efd740eb440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56124/0 (socket says 172.21.15.196:56124) 2024-04-28T13:31:30.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.543+0000 7efd7b400640 1 -- 172.21.15.196:0/2023852170 learned_addr learned my addr 172.21.15.196:0/2023852170 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:30.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.543+0000 7efd7a000640 1 -- 172.21.15.196:0/2023852170 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1578275193 0 0) 0x7efd70001110 con 0x7efd740eb060 2024-04-28T13:31:30.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.544+0000 7efd7a000640 1 -- 172.21.15.196:0/2023852170 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd64003520 con 0x7efd740eb060 2024-04-28T13:31:30.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.544+0000 7efd7a000640 1 -- 172.21.15.196:0/2023852170 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1344374149 0 0) 0x7efd64003520 con 0x7efd740eb060 2024-04-28T13:31:30.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.544+0000 7efd7a000640 1 -- 172.21.15.196:0/2023852170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd740ebb50 msgr2=0x7efd74055300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-28T13:31:30.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.544+0000 7efd7a000640 1 --2- 172.21.15.196:0/2023852170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd740ebb50 0x7efd74055300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.544+0000 7efd7a000640 1 -- 172.21.15.196:0/2023852170 >> v2:172.21.15.196:3300/0 conn(0x7efd740061a0 msgr2=0x7efd74006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:30.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.544+0000 7efd7a000640 1 --2- 172.21.15.196:0/2023852170 >> v2:172.21.15.196:3300/0 conn(0x7efd740061a0 0x7efd74006630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.544+0000 7efd7a000640 1 -- 172.21.15.196:0/2023852170 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd74056d90 con 0x7efd740eb060 2024-04-28T13:31:30.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.544+0000 7efd7a000640 1 -- 172.21.15.196:0/2023852170 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7efd70001840 con 0x7efd740eb060 2024-04-28T13:31:30.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.544+0000 7efd7a000640 1 -- 172.21.15.196:0/2023852170 >> v1:172.21.15.196:6789/0 conn(0x7efd740eb060 legacy=0x7efd740eb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.544+0000 7efd7a000640 1 --2- 172.21.15.196:0/2023852170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd640039e0 0x7efd64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.544+0000 7efd7a000640 1 --2- 172.21.15.196:0/2023852170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd64004800 0x7efd64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.544+0000 7efd7a000640 1 --2- 172.21.15.196:0/2023852170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd64005210 0x7efd6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.544+0000 7efd7a000640 1 -- 172.21.15.196:0/2023852170 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd6400c090 con 0x7efd640039e0 2024-04-28T13:31:30.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.544+0000 7efd7a000640 1 -- 172.21.15.196:0/2023852170 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efd6400c240 con 0x7efd64004800 2024-04-28T13:31:30.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.544+0000 7efd7a000640 1 -- 172.21.15.196:0/2023852170 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efd6400c440 con 0x7efd64005210 2024-04-28T13:31:30.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.545+0000 7efd7be00640 1 --2- 172.21.15.196:0/2023852170 >> v2:172.21.15.196:3300/0 conn(0x7efd740061a0 0x7efd74006630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:31:30.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.545+0000 7efd7aa00640 1 --2- 172.21.15.196:0/2023852170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd64004800 0x7efd64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.546+0000 7efd7aa00640 1 -- 172.21.15.196:0/2023852170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd64005210 msgr2=0x7efd6400b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-28T13:31:30.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.546+0000 7efd7aa00640 1 --2- 172.21.15.196:0/2023852170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd64005210 0x7efd6400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.546+0000 7efd7aa00640 1 -- 172.21.15.196:0/2023852170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd640039e0 msgr2=0x7efd64003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:30.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.546+0000 7efd7aa00640 1 --2- 172.21.15.196:0/2023852170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd640039e0 0x7efd64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.546+0000 7efd7aa00640 1 -- 172.21.15.196:0/2023852170 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd74056d90 con 0x7efd64004800 2024-04-28T13:31:30.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.547+0000 7efd7aa00640 1 --2- 172.21.15.196:0/2023852170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd64004800 0x7efd64004c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd7c06aa70 tx=0x7efd64005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d98ed6defb49dde server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.548+0000 7fa29f200640 1 Processor -- start 2024-04-28T13:31:30.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.547+0000 7efd7a000640 1 -- 172.21.15.196:0/2023852170 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0b4070 con 0x7efd64004800 2024-04-28T13:31:30.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.548+0000 7fa29f200640 1 -- start start 2024-04-28T13:31:30.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.547+0000 7efd7a000640 1 -- 172.21.15.196:0/2023852170 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c0a7920 con 0x7efd64004800 2024-04-28T13:31:30.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.547+0000 7efd7a000640 1 -- 172.21.15.196:0/2023852170 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0515b0 con 0x7efd64004800 2024-04-28T13:31:30.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.548+0000 7efd80a00640 1 -- 172.21.15.196:0/2023852170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd64004800 msgr2=0x7efd64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.548+0000 7efd80a00640 1 --2- 172.21.15.196:0/2023852170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd64004800 0x7efd64004c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd7c06aa70 tx=0x7efd64005830 comp rx=0 tx=0).stop 2024-04-28T13:31:30.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.549+0000 7efd7be00640 1 -- 172.21.15.196:0/2023852170 reap_dead start 2024-04-28T13:31:30.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.549+0000 7efd80a00640 1 -- 172.21.15.196:0/2023852170 shutdown_connections 2024-04-28T13:31:30.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.549+0000 7efd80a00640 1 -- 172.21.15.196:0/2023852170 >> 172.21.15.196:0/2023852170 conn(0x7efd740559e0 msgr2=0x7efd74055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:30.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.549+0000 7efd80a00640 1 -- 172.21.15.196:0/2023852170 shutdown_connections 2024-04-28T13:31:30.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.549+0000 7efd80a00640 1 -- 172.21.15.196:0/2023852170 wait complete. 2024-04-28T13:31:30.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.550+0000 7fa29f200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2840051c0 0x7fa2a014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.550+0000 7fa29f200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2a01e7630 0x7fa2a00b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.551+0000 7fa29f200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2a01e5150 0x7fa2a01f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.551+0000 7fa29de00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2840051c0 0x7fa2a014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.551+0000 7fa29de00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2840051c0 0x7fa2a014fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:44928/0 (socket says 172.21.15.196:44928) 2024-04-28T13:31:30.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.551+0000 7fa29de00640 1 -- 172.21.15.196:0/3654307534 learned_addr learned my addr 172.21.15.196:0/3654307534 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:30.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.551+0000 7f8ac9000640 1 Processor -- start 2024-04-28T13:31:30.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.551+0000 7fe6a8a00640 1 Processor -- start 2024-04-28T13:31:30.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.551+0000 7f8ac9000640 1 -- start start 2024-04-28T13:31:30.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.551+0000 7fa29d400640 1 --2- 172.21.15.196:0/3654307534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2a01e7630 0x7fa2a00b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.552+0000 7efd80a00640 1 Processor -- start 2024-04-28T13:31:30.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.552+0000 7efd80a00640 1 -- start start 2024-04-28T13:31:30.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.552+0000 7fa29f200640 1 -- 172.21.15.196:0/3654307534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a00b4c70 con 0x7fa2a01e5150 2024-04-28T13:31:30.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.552+0000 7fa29f200640 1 -- 172.21.15.196:0/3654307534 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a00b4af0 con 0x7fa2a01e7630 2024-04-28T13:31:30.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.552+0000 7fa29f200640 1 -- 172.21.15.196:0/3654307534 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a01524b0 con 0x7fa2840051c0 2024-04-28T13:31:30.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.552+0000 7fd8e2200640 1 Processor -- start 2024-04-28T13:31:30.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.553+0000 7fd8e2200640 1 -- start start 2024-04-28T13:31:30.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.553+0000 7fa29e800640 1 --2- 172.21.15.196:0/3654307534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2a01e5150 0x7fa2a01f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.552+0000 7fe6a8a00640 1 -- start start 2024-04-28T13:31:30.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7fe6a8a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a414c600 0x7fe6a40d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7fe6a8a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe6a4072730 0x7fe6a40b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7fe6a8a00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6a40ceef0 con 0x7fe6a40d1460 2024-04-28T13:31:30.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7fe6a8a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fe6a414c9e0 con 0x7fe6a4072730 2024-04-28T13:31:30.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7fe6a8a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a40cf1d0 con 0x7fe6a414c600 2024-04-28T13:31:30.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7efd80a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd7404d350 0x7efd7404ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7efd80a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd74050e00 0x7efd7404d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7f8ac9000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8ac415f4d0 0x7f8ac40b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7f8ac9000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac40d5810 0x7f8ac40b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7f8ac9000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ac40b2550 con 0x7f8ac40d2e20 2024-04-28T13:31:30.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7f8ac9000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8ac40b6180 con 0x7f8ac415f4d0 2024-04-28T13:31:30.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7f8ac9000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8ac40b2830 con 0x7f8ac40d5810 2024-04-28T13:31:30.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7fa29e800640 1 -- 172.21.15.196:0/3654307534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2840051c0 msgr2=0x7fa2a014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7efd80a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd7404eb90 0x7efd7404f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7fd8e2200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8dc1629b0 0x7fd8dc0d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7fd8e2200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd8dc0d19d0 0x7fd8dc0d1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7fd8e2200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8dc0cfed0 con 0x7fd8dc162d90 2024-04-28T13:31:30.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7fd8e2200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd8dc0cfca0 con 0x7fd8dc0d19d0 2024-04-28T13:31:30.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7fd8e2200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd8dc0d0ae0 con 0x7fd8dc1629b0 2024-04-28T13:31:30.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7fe6a1400640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fe6a40d1460 0x7fe6a4072020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.196:36494/0 (socket says 172.21.15.196:36494) 2024-04-28T13:31:30.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7efd80a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd74057350 con 0x7efd74050e00 2024-04-28T13:31:30.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7efd80a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efd740579a0 con 0x7efd7404eb90 2024-04-28T13:31:30.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7efd80a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efd74057630 con 0x7efd7404d350 2024-04-28T13:31:30.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7efd7aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd74050e00 0x7efd7404d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7fe6a2800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe6a4072730 0x7fe6a40b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.554+0000 7fa29e800640 1 --2- 172.21.15.196:0/3654307534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2840051c0 0x7fa2a014fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7fa29e800640 1 -- 172.21.15.196:0/3654307534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2a01e7630 msgr2=0x7fa2a00b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7fa29e800640 1 --2- 172.21.15.196:0/3654307534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2a01e7630 0x7fa2a00b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7fa29e800640 1 -- 172.21.15.196:0/3654307534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a0150350 con 0x7fa2a01e5150 2024-04-28T13:31:30.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7fe6a1e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a414c600 0x7fe6a40d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7fe6a2800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe6a4072730 0x7fe6a40b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:39844/0 (socket says 172.21.15.196:39844) 2024-04-28T13:31:30.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7fe6a2800640 1 -- 172.21.15.196:0/3475419862 learned_addr learned my addr 172.21.15.196:0/3475419862 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:30.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7fd8db400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8dc1629b0 0x7fd8dc0d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7fd8db400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8dc1629b0 0x7fd8dc0d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:44938/0 (socket says 172.21.15.196:44938) 2024-04-28T13:31:30.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7fd8db400640 1 -- 172.21.15.196:0/1271399537 learned_addr learned my addr 172.21.15.196:0/1271399537 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:30.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7f8ac1400640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f8ac40d2e20 0x7f8ac40d5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.196:36500/0 (socket says 172.21.15.196:36500) 2024-04-28T13:31:30.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7f8ac1400640 1 -- 172.21.15.196:0/1456109123 learned_addr learned my addr 172.21.15.196:0/1456109123 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:30.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7f8ac1e00640 1 --2- 172.21.15.196:0/1456109123 >> v2:172.21.15.196:3300/0 conn(0x7f8ac415f4d0 0x7f8ac40b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.556+0000 7efd7be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd7404eb90 0x7efd7404f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.556+0000 7efd7be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd7404eb90 0x7efd7404f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:39846/0 (socket says 172.21.15.196:39846) 2024-04-28T13:31:30.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.556+0000 7efd7be00640 1 -- 172.21.15.196:0/3026862417 learned_addr learned my addr 172.21.15.196:0/3026862417 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:30.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.556+0000 7fd8dbe00640 1 --2- 172.21.15.196:0/1271399537 >> v2:172.21.15.203:3300/0 conn(0x7fd8dc0d19d0 0x7fd8dc0d1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.556+0000 7f8ac2800640 1 --2- 172.21.15.196:0/1456109123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac40d5810 0x7f8ac40b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.556+0000 7f8ac0a00640 1 -- 172.21.15.196:0/1456109123 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2263071449 0 0) 0x7f8ac40b2550 con 0x7f8ac40d2e20 2024-04-28T13:31:30.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.556+0000 7f8ac0a00640 1 -- 172.21.15.196:0/1456109123 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8aa8003520 con 0x7f8ac40d2e20 2024-04-28T13:31:30.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.556+0000 7f8ac1e00640 1 -- 172.21.15.196:0/1456109123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac40d5810 msgr2=0x7f8ac40b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:30.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.556+0000 7f8ac1e00640 1 --2- 172.21.15.196:0/1456109123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac40d5810 0x7f8ac40b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.564 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.556+0000 7f8ac1e00640 1 -- 172.21.15.196:0/1456109123 >> v1:172.21.15.203:6789/0 conn(0x7f8ac40d2e20 legacy=0x7f8ac40d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.564 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.555+0000 7efd7b400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd7404d350 0x7efd7404ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.564 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.556+0000 7f8ac1e00640 1 -- 172.21.15.196:0/1456109123 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac40b2b10 con 0x7f8ac415f4d0 2024-04-28T13:31:30.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.556+0000 7fd8da000640 1 -- 172.21.15.196:0/1271399537 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 437349874 0 0) 0x7fd8dc0cfed0 con 0x7fd8dc162d90 2024-04-28T13:31:30.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.556+0000 7fd8da000640 1 -- 172.21.15.196:0/1271399537 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8c0003520 con 0x7fd8dc162d90 2024-04-28T13:31:30.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.557+0000 7fd8db400640 1 -- 172.21.15.196:0/1271399537 >> v2:172.21.15.203:3300/0 conn(0x7fd8dc0d19d0 msgr2=0x7fd8dc0d1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:30.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.557+0000 7fd8db400640 1 --2- 172.21.15.196:0/1271399537 >> v2:172.21.15.203:3300/0 conn(0x7fd8dc0d19d0 0x7fd8dc0d1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.557+0000 7fd8db400640 1 -- 172.21.15.196:0/1271399537 >> v1:172.21.15.203:6789/0 conn(0x7fd8dc162d90 legacy=0x7fd8dc0d58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.557+0000 7fd8db400640 1 -- 172.21.15.196:0/1271399537 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8dc0d01b0 con 0x7fd8dc1629b0 2024-04-28T13:31:30.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.556+0000 7fa29e800640 1 --2- 172.21.15.196:0/3654307534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2a01e5150 0x7fa2a01f5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa2a015fde0 tx=0x7fa28c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.557+0000 7efd7be00640 1 -- 172.21.15.196:0/3026862417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd7404d350 msgr2=0x7efd7404ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.557+0000 7efd7be00640 1 --2- 172.21.15.196:0/3026862417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd7404d350 0x7efd7404ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.557+0000 7efd7be00640 1 -- 172.21.15.196:0/3026862417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd74050e00 msgr2=0x7efd7404d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.557+0000 7efd7be00640 1 --2- 172.21.15.196:0/3026862417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd74050e00 0x7efd7404d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.567 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.557+0000 7efd7be00640 1 -- 172.21.15.196:0/3026862417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd74051e40 con 0x7efd7404eb90 2024-04-28T13:31:30.567 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.557+0000 7efd7aa00640 1 --2- 172.21.15.196:0/3026862417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd74050e00 0x7efd7404d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:31:30.567 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.557+0000 7efd7b400640 1 --2- 172.21.15.196:0/3026862417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd7404d350 0x7efd7404ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:30.567 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.558+0000 7fa296000640 1 -- 172.21.15.196:0/3654307534 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c007d50 con 0x7fa2a01e5150 2024-04-28T13:31:30.567 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.558+0000 7fd8dbe00640 1 --2- 172.21.15.196:0/1271399537 >> v2:172.21.15.203:3300/0 conn(0x7fd8dc0d19d0 0x7fd8dc0d1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:31:30.568 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.558+0000 7efd7be00640 1 --2- 172.21.15.196:0/3026862417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd7404eb90 0x7efd7404f020 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd74059f00 tx=0x7efd6800e800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.568 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.556+0000 7fe6a2800640 1 -- 172.21.15.196:0/3475419862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a414c600 msgr2=0x7fe6a40d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:30.568 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.557+0000 7fe6a2800640 1 --2- 172.21.15.196:0/3475419862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a414c600 0x7fe6a40d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.568 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.557+0000 7fe6a2800640 1 -- 172.21.15.196:0/3475419862 >> v1:172.21.15.203:6789/0 conn(0x7fe6a40d1460 legacy=0x7fe6a4072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.568 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.557+0000 7fe6a2800640 1 -- 172.21.15.196:0/3475419862 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a40cf4b0 con 0x7fe6a4072730 2024-04-28T13:31:30.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.558+0000 7fa296000640 1 -- 172.21.15.196:0/3654307534 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa28c0075b0 con 0x7fa2a01e5150 2024-04-28T13:31:30.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.558+0000 7fa296000640 1 -- 172.21.15.196:0/3654307534 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c03d630 con 0x7fa2a01e5150 2024-04-28T13:31:30.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.558+0000 7efd6f400640 1 -- 172.21.15.196:0/3026862417 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6801a070 con 0x7efd7404eb90 2024-04-28T13:31:30.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.559+0000 7efd6f400640 1 -- 172.21.15.196:0/3026862417 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd68004250 con 0x7efd7404eb90 2024-04-28T13:31:30.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.558+0000 7fd8db400640 1 --2- 172.21.15.196:0/1271399537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8dc1629b0 0x7fd8dc0d0f70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009870 tx=0x7fd8cc02a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57c5c08b90b26b74 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.558+0000 7f8ac1e00640 1 --2- 172.21.15.196:0/1456109123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac415f4d0 0x7f8ac40b1ee0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009d90 tx=0x7f8abc02a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c231e487aa835bf server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.559+0000 7efd6f400640 1 -- 172.21.15.196:0/3026862417 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6800f040 con 0x7efd7404eb90 2024-04-28T13:31:30.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.558+0000 7fa29f200640 1 -- 172.21.15.196:0/3654307534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a0150600 con 0x7fa2a01e5150 2024-04-28T13:31:30.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.559+0000 7fa29f200640 1 -- 172.21.15.196:0/3654307534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a0150bc0 con 0x7fa2a01e5150 2024-04-28T13:31:30.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.559+0000 7efd80a00640 1 -- 172.21.15.196:0/3026862417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7404dc70 con 0x7efd7404eb90 2024-04-28T13:31:30.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.559+0000 7efd80a00640 1 -- 172.21.15.196:0/3026862417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd740515a0 con 0x7efd7404eb90 2024-04-28T13:31:30.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.559+0000 7fd8da000640 1 -- 172.21.15.196:0/1271399537 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc037070 con 0x7fd8dc1629b0 2024-04-28T13:31:30.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.559+0000 7fd8da000640 1 -- 172.21.15.196:0/1271399537 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8cc032440 con 0x7fd8dc1629b0 2024-04-28T13:31:30.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.559+0000 7fd8da000640 1 -- 172.21.15.196:0/1271399537 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc032830 con 0x7fd8dc1629b0 2024-04-28T13:31:30.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.560+0000 7f8ac0a00640 1 -- 172.21.15.196:0/1456109123 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc007740 con 0x7f8ac415f4d0 2024-04-28T13:31:30.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.560+0000 7f8ac0a00640 1 -- 172.21.15.196:0/1456109123 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8abc0078e0 con 0x7f8ac415f4d0 2024-04-28T13:31:30.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.560+0000 7f8ac0a00640 1 -- 172.21.15.196:0/1456109123 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc007c90 con 0x7f8ac415f4d0 2024-04-28T13:31:30.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.560+0000 7fe6a2800640 1 --2- 172.21.15.196:0/3475419862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a4072730 0x7fe6a40b3de0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe690009870 tx=0x7fe69002a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c54937737ea8ed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.560+0000 7fd8e2200640 1 -- 172.21.15.196:0/1271399537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8dc1629b0 msgr2=0x7fd8dc0d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.560+0000 7fa29f200640 1 -- 172.21.15.196:0/3654307534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a006a290 con 0x7fa2a01e5150 2024-04-28T13:31:30.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.560+0000 7fd8e2200640 1 --2- 172.21.15.196:0/1271399537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8dc1629b0 0x7fd8dc0d0f70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009870 tx=0x7fd8cc02a290 comp rx=0 tx=0).stop 2024-04-28T13:31:30.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.561+0000 7fd8e2200640 1 -- 172.21.15.196:0/1271399537 shutdown_connections 2024-04-28T13:31:30.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.561+0000 7fd8e2200640 1 --2- 172.21.15.196:0/1271399537 >> v2:172.21.15.203:3300/0 conn(0x7fd8dc0d19d0 0x7fd8dc0d1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.561+0000 7fd8e2200640 1 --2- 172.21.15.196:0/1271399537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8dc1629b0 0x7fd8dc0d0f70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.561+0000 7fd8e2200640 1 -- 172.21.15.196:0/1271399537 >> 172.21.15.196:0/1271399537 conn(0x7fd8dc09e4d0 msgr2=0x7fd8dc0d5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:30.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.561+0000 7f8ac9000640 1 -- 172.21.15.196:0/1456109123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac415f4d0 msgr2=0x7f8ac40b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.561+0000 7f8ac9000640 1 --2- 172.21.15.196:0/1456109123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac415f4d0 0x7f8ac40b1ee0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009d90 tx=0x7f8abc02a590 comp rx=0 tx=0).stop 2024-04-28T13:31:30.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.561+0000 7f8ac9000640 1 -- 172.21.15.196:0/1456109123 shutdown_connections 2024-04-28T13:31:30.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.561+0000 7f8ac9000640 1 --2- 172.21.15.196:0/1456109123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac40d5810 0x7f8ac40b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.561+0000 7f8ac9000640 1 --2- 172.21.15.196:0/1456109123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac415f4d0 0x7f8ac40b1ee0 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009d90 tx=0x7f8abc02a590 comp rx=0 tx=0).stop 2024-04-28T13:31:30.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.561+0000 7f8ac9000640 1 -- 172.21.15.196:0/1456109123 >> 172.21.15.196:0/1456109123 conn(0x7f8ac408ebf0 msgr2=0x7f8ac409e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:30.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.562+0000 7fe6a0a00640 1 -- 172.21.15.196:0/3475419862 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690037070 con 0x7fe6a4072730 2024-04-28T13:31:30.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.563+0000 7fe6a0a00640 1 -- 172.21.15.196:0/3475419862 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe690032440 con 0x7fe6a4072730 2024-04-28T13:31:30.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.563+0000 7fd8e2200640 1 -- 172.21.15.196:0/1271399537 shutdown_connections 2024-04-28T13:31:30.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.563+0000 7fe6a0a00640 1 -- 172.21.15.196:0/3475419862 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690032830 con 0x7fe6a4072730 2024-04-28T13:31:30.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.563+0000 7efd80a00640 1 -- 172.21.15.196:0/3026862417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd58005290 con 0x7efd7404eb90 2024-04-28T13:31:30.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.564+0000 7fd8e2200640 1 -- 172.21.15.196:0/1271399537 wait complete. 2024-04-28T13:31:30.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.564+0000 7fe6a8a00640 1 -- 172.21.15.196:0/3475419862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a4072730 msgr2=0x7fe6a40b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.564+0000 7fe6a8a00640 1 --2- 172.21.15.196:0/3475419862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a4072730 0x7fe6a40b3de0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe690009870 tx=0x7fe69002a290 comp rx=0 tx=0).stop 2024-04-28T13:31:30.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.565+0000 7fe6a8a00640 1 -- 172.21.15.196:0/3475419862 shutdown_connections 2024-04-28T13:31:30.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.565+0000 7fe6a8a00640 1 --2- 172.21.15.196:0/3475419862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a4072730 0x7fe6a40b3de0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.565+0000 7fe6a8a00640 1 --2- 172.21.15.196:0/3475419862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a414c600 0x7fe6a40d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.565+0000 7fe6a8a00640 1 -- 172.21.15.196:0/3475419862 >> 172.21.15.196:0/3475419862 conn(0x7fe6a40cd830 msgr2=0x7fe6a40cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:30.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.564+0000 7fa296000640 1 -- 172.21.15.196:0/3654307534 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa28c03b040 con 0x7fa2a01e5150 2024-04-28T13:31:30.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.565+0000 7efd6f400640 1 -- 172.21.15.196:0/3026862417 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd6801d070 con 0x7efd7404eb90 2024-04-28T13:31:30.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.565+0000 7fe6a8a00640 1 -- 172.21.15.196:0/3475419862 shutdown_connections 2024-04-28T13:31:30.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.565+0000 7fa296000640 1 --2- 172.21.15.196:0/3654307534 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa28006c960 0x7fa28006ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.565+0000 7fa296000640 1 -- 172.21.15.196:0/3654307534 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa28c0ac860 con 0x7fa2a01e5150 2024-04-28T13:31:30.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.565+0000 7f8ac9000640 1 -- 172.21.15.196:0/1456109123 shutdown_connections 2024-04-28T13:31:30.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.566+0000 7fa29de00640 1 --2- 172.21.15.196:0/3654307534 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa28006c960 0x7fa28006ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.566+0000 7fd8e2200640 1 Processor -- start 2024-04-28T13:31:30.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.567+0000 7fd8e2200640 1 -- start start 2024-04-28T13:31:30.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.567+0000 7efd6f400640 1 --2- 172.21.15.196:0/3026862417 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efd5c06c920 0x7efd5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.567+0000 7f8ac9000640 1 -- 172.21.15.196:0/1456109123 wait complete. 2024-04-28T13:31:30.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.567+0000 7efd6f400640 1 -- 172.21.15.196:0/3026862417 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efd6808a800 con 0x7efd7404eb90 2024-04-28T13:31:30.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.567+0000 7efd7b400640 1 --2- 172.21.15.196:0/3026862417 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efd5c06c920 0x7efd5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.568+0000 7fd8e2200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8dc0d19d0 0x7fd8dc0c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.568+0000 7fd8e2200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8dc1629b0 0x7fd8dc15c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.568+0000 7fd8e2200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8dc162d90 0x7fd8dc0ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.568+0000 7fd8e2200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc0d5fb0 con 0x7fd8dc162d90 2024-04-28T13:31:30.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.568+0000 7fd8e2200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc0d0770 con 0x7fd8dc1629b0 2024-04-28T13:31:30.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.568+0000 7fd8e2200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd8dc0d3fe0 con 0x7fd8dc0d19d0 2024-04-28T13:31:30.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.565+0000 7fe6a8a00640 1 -- 172.21.15.196:0/3475419862 wait complete. 2024-04-28T13:31:30.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.568+0000 7fd8dbe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8dc162d90 0x7fd8dc0ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.567+0000 7fe6a8a00640 1 Processor -- start 2024-04-28T13:31:30.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.568+0000 7fa29de00640 1 --2- 172.21.15.196:0/3654307534 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa28006c960 0x7fa28006ee20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa2980074a0 tx=0x7fa29800e040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.568+0000 7fd8dbe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8dc162d90 0x7fd8dc0ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35896/0 (socket says 172.21.15.196:35896) 2024-04-28T13:31:30.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.568+0000 7fd8dbe00640 1 -- 172.21.15.196:0/986599602 learned_addr learned my addr 172.21.15.196:0/986599602 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:30.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.569+0000 7fd8dbe00640 1 -- 172.21.15.196:0/986599602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8dc0d19d0 msgr2=0x7fd8dc0c9660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T13:31:30.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.569+0000 7fd8dbe00640 1 --2- 172.21.15.196:0/986599602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8dc0d19d0 0x7fd8dc0c9660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.569+0000 7fd8dbe00640 1 -- 172.21.15.196:0/986599602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8dc1629b0 msgr2=0x7fd8dc15c7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T13:31:30.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.569+0000 7fd8dbe00640 1 --2- 172.21.15.196:0/986599602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8dc1629b0 0x7fd8dc15c7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.569+0000 7fd8dbe00640 1 -- 172.21.15.196:0/986599602 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8dc0cadd0 con 0x7fd8dc162d90 2024-04-28T13:31:30.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.569+0000 7fd8dbe00640 1 --2- 172.21.15.196:0/986599602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8dc162d90 0x7fd8dc0ca410 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400d1b0 tx=0x7fd8d400d670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.570+0000 7fe6a8a00640 1 -- start start 2024-04-28T13:31:30.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.570+0000 7f8ac9000640 1 Processor -- start 2024-04-28T13:31:30.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.571+0000 7f8ac9000640 1 -- start start 2024-04-28T13:31:30.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.572+0000 7fe6a8a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a4072730 0x7fe6a4137280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.572+0000 7fe6a8a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a40d1460 0x7fe6a41625a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.587 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.572+0000 7f8ac9000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ac40d2e20 0x7f8ac4150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.587 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.573+0000 7fe6a8a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a414c600 0x7fe6a4162ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.572+0000 7f8ac9000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac40d5810 0x7f8ac4150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.572+0000 7f8ac9000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac4151510 0x7f8ac406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.572+0000 7f8ac9000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac40b3b30 con 0x7f8ac40d5810 2024-04-28T13:31:30.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.572+0000 7f8ac9000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac40b30d0 con 0x7f8ac40d2e20 2024-04-28T13:31:30.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.572+0000 7f8ac9000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8ac40b3cb0 con 0x7f8ac4151510 2024-04-28T13:31:30.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.573+0000 7f8ac2800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac4151510 0x7f8ac406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.573+0000 7f8ac1400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac40d5810 0x7f8ac4150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.573+0000 7fe6a8a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a40cff90 con 0x7fe6a4072730 2024-04-28T13:31:30.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.573+0000 7f8ac1e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ac40d2e20 0x7f8ac4150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.573+0000 7f8ac1e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ac40d2e20 0x7f8ac4150710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:39870/0 (socket says 172.21.15.196:39870) 2024-04-28T13:31:30.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.573+0000 7f8ac1e00640 1 -- 172.21.15.196:0/3653862757 learned_addr learned my addr 172.21.15.196:0/3653862757 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:30.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.573+0000 7f8ac2800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac4151510 0x7f8ac406d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:44970/0 (socket says 172.21.15.196:44970) 2024-04-28T13:31:30.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.573+0000 7fe6a1e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a4072730 0x7fe6a4137280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.573+0000 7fe6a2800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a414c600 0x7fe6a4162ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.573+0000 7fe6a8a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a40cfa70 con 0x7fe6a414c600 2024-04-28T13:31:30.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.573+0000 7fe6a8a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a40ceb50 con 0x7fe6a40d1460 2024-04-28T13:31:30.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.573+0000 7fe6a1400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a40d1460 0x7fe6a41625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.574+0000 7fe6a1400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a40d1460 0x7fe6a41625a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:44974/0 (socket says 172.21.15.196:44974) 2024-04-28T13:31:30.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.574+0000 7fe6a1400640 1 -- 172.21.15.196:0/644869295 learned_addr learned my addr 172.21.15.196:0/644869295 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:30.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.573+0000 7fe6a1e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a4072730 0x7fe6a4137280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35904/0 (socket says 172.21.15.196:35904) 2024-04-28T13:31:30.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.573+0000 7fe6a2800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a414c600 0x7fe6a4162ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:39874/0 (socket says 172.21.15.196:39874) 2024-04-28T13:31:30.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.574+0000 7efd7b400640 1 --2- 172.21.15.196:0/3026862417 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efd5c06c920 0x7efd5c06ede0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efd70007d00 tx=0x7efd70009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.574+0000 7fd8d3400640 1 -- 172.21.15.196:0/986599602 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d401a070 con 0x7fd8dc162d90 2024-04-28T13:31:30.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.574+0000 7fd8d3400640 1 -- 172.21.15.196:0/986599602 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8d4003990 con 0x7fd8dc162d90 2024-04-28T13:31:30.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.574+0000 7fd8d3400640 1 -- 172.21.15.196:0/986599602 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4003d40 con 0x7fd8dc162d90 2024-04-28T13:31:30.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.574+0000 7fe6a1400640 1 -- 172.21.15.196:0/644869295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a414c600 msgr2=0x7fe6a4162ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.574+0000 7fe6a1400640 1 --2- 172.21.15.196:0/644869295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a414c600 0x7fe6a4162ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.574+0000 7fe6a1400640 1 -- 172.21.15.196:0/644869295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a4072730 msgr2=0x7fe6a4137280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.575+0000 7fd8e2200640 1 -- 172.21.15.196:0/986599602 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8dc0cb0b0 con 0x7fd8dc162d90 2024-04-28T13:31:30.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.575+0000 7fd8e2200640 1 -- 172.21.15.196:0/986599602 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8dc0c8e60 con 0x7fd8dc162d90 2024-04-28T13:31:30.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.575+0000 7fe6a1400640 1 --2- 172.21.15.196:0/644869295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a4072730 0x7fe6a4137280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.575+0000 7fe6a1400640 1 -- 172.21.15.196:0/644869295 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a4137c00 con 0x7fe6a40d1460 2024-04-28T13:31:30.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.575+0000 7fe6a1e00640 1 --2- 172.21.15.196:0/644869295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a4072730 0x7fe6a4137280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:31:30.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.575+0000 7f8ac2800640 1 -- 172.21.15.196:0/3653862757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ac40d2e20 msgr2=0x7f8ac4150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.575+0000 7f8ac2800640 1 --2- 172.21.15.196:0/3653862757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ac40d2e20 0x7f8ac4150710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.576+0000 7fe6a1400640 1 --2- 172.21.15.196:0/644869295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a40d1460 0x7fe6a41625a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00c8a0 tx=0x7fe69c00cd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.575+0000 7f8ac2800640 1 -- 172.21.15.196:0/3653862757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac40d5810 msgr2=0x7f8ac4150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.575+0000 7f8ac2800640 1 --2- 172.21.15.196:0/3653862757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac40d5810 0x7f8ac4150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.576+0000 7fa296000640 1 -- 172.21.15.196:0/3654307534 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa28c07a6b0 con 0x7fa2a01e5150 2024-04-28T13:31:30.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.575+0000 7f8ac2800640 1 -- 172.21.15.196:0/3653862757 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac41523a0 con 0x7f8ac4151510 2024-04-28T13:31:30.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.576+0000 7f8ac1400640 1 --2- 172.21.15.196:0/3653862757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac40d5810 0x7f8ac4150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:31:30.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.576+0000 7fe69a000640 1 -- 172.21.15.196:0/644869295 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c017070 con 0x7fe6a40d1460 2024-04-28T13:31:30.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.577+0000 7f8ac2800640 1 --2- 172.21.15.196:0/3653862757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac4151510 0x7f8ac406d350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000e7f0 tx=0x7f8ab000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.576+0000 7fe69a000640 1 -- 172.21.15.196:0/644869295 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe69c00a640 con 0x7fe6a40d1460 2024-04-28T13:31:30.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.576+0000 7fe69a000640 1 -- 172.21.15.196:0/644869295 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c00a9f0 con 0x7fe6a40d1460 2024-04-28T13:31:30.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.578+0000 7f8aba000640 1 -- 172.21.15.196:0/3653862757 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0017070 con 0x7f8ac4151510 2024-04-28T13:31:30.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.578+0000 7f8aba000640 1 -- 172.21.15.196:0/3653862757 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab000c6d0 con 0x7f8ac4151510 2024-04-28T13:31:30.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.578+0000 7f8aba000640 1 -- 172.21.15.196:0/3653862757 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab000cb00 con 0x7f8ac4151510 2024-04-28T13:31:30.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.578+0000 7f8ac9000640 1 -- 172.21.15.196:0/3653862757 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac4151a90 con 0x7f8ac4151510 2024-04-28T13:31:30.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.578+0000 7f8ac9000640 1 -- 172.21.15.196:0/3653862757 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac4152050 con 0x7f8ac4151510 2024-04-28T13:31:30.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.579+0000 7fd8e2200640 1 -- 172.21.15.196:0/986599602 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8dc0693d0 con 0x7fd8dc162d90 2024-04-28T13:31:30.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.577+0000 7fe6a8a00640 1 -- 172.21.15.196:0/644869295 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a4137ee0 con 0x7fe6a40d1460 2024-04-28T13:31:30.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.577+0000 7fe6a8a00640 1 -- 172.21.15.196:0/644869295 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a40b4b80 con 0x7fe6a40d1460 2024-04-28T13:31:30.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.579+0000 7fe6a8a00640 1 -- 172.21.15.196:0/644869295 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a406a290 con 0x7fe6a40d1460 2024-04-28T13:31:30.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.581+0000 7f8ac9000640 1 -- 172.21.15.196:0/3653862757 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a9c005290 con 0x7f8ac4151510 2024-04-28T13:31:30.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.579+0000 7fd8d3400640 1 -- 172.21.15.196:0/986599602 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8d401d070 con 0x7fd8dc162d90 2024-04-28T13:31:30.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.581+0000 7fd8d3400640 1 --2- 172.21.15.196:0/986599602 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd8bc06c940 0x7fd8bc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.581+0000 7fd8d3400640 1 -- 172.21.15.196:0/986599602 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd8d408bcd0 con 0x7fd8dc162d90 2024-04-28T13:31:30.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.582+0000 7fd8db400640 1 --2- 172.21.15.196:0/986599602 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd8bc06c940 0x7fd8bc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.583+0000 7fd8db400640 1 --2- 172.21.15.196:0/986599602 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd8bc06c940 0x7fd8bc06ee00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc034000 tx=0x7fd8cc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.582+0000 7fe69a000640 1 -- 172.21.15.196:0/644869295 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe69c01a070 con 0x7fe6a40d1460 2024-04-28T13:31:30.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.584+0000 7fe69a000640 1 --2- 172.21.15.196:0/644869295 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe68406c9c0 0x7fe68406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.584+0000 7fe69a000640 1 -- 172.21.15.196:0/644869295 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe69c08b330 con 0x7fe6a40d1460 2024-04-28T13:31:30.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.584+0000 7f8aba000640 1 -- 172.21.15.196:0/3653862757 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ab001a070 con 0x7f8ac4151510 2024-04-28T13:31:30.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.584+0000 7fe6a1e00640 1 --2- 172.21.15.196:0/644869295 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe68406c9c0 0x7fe68406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.584+0000 7efd6f400640 1 -- 172.21.15.196:0/3026862417 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd68051c00 con 0x7efd7404eb90 2024-04-28T13:31:30.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.585+0000 7fe6a1e00640 1 --2- 172.21.15.196:0/644869295 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe68406c9c0 0x7fe68406ee80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe694006fd0 tx=0x7fe694008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.586+0000 7f8aba000640 1 --2- 172.21.15.196:0/3653862757 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8aa406c940 0x7f8aa406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.586+0000 7f8aba000640 1 -- 172.21.15.196:0/3653862757 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8ab008bac0 con 0x7f8ac4151510 2024-04-28T13:31:30.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.587+0000 7f8ac1e00640 1 --2- 172.21.15.196:0/3653862757 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8aa406c940 0x7f8aa406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.588+0000 7f8ac1e00640 1 --2- 172.21.15.196:0/3653862757 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8aa406c940 0x7f8aa406ee00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8abc02a5d0 tx=0x7f8abc02f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.590+0000 7fd8d3400640 1 -- 172.21.15.196:0/986599602 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8d4010070 con 0x7fd8dc162d90 2024-04-28T13:31:30.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.594+0000 7fe69a000640 1 -- 172.21.15.196:0/644869295 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe69c0590f0 con 0x7fe6a40d1460 2024-04-28T13:31:30.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.600+0000 7f9050000640 1 Processor -- start 2024-04-28T13:31:30.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.601+0000 7f9050000640 1 -- start start 2024-04-28T13:31:30.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.602+0000 7f9050000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f904814c600 0x7f90480d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.602+0000 7f9050000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f90480d1460 0x7f9048072020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.602+0000 7f9050000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90480ceef0 con 0x7f9048072560 2024-04-28T13:31:30.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.602+0000 7f9050000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f904814c9e0 con 0x7f90480d1460 2024-04-28T13:31:30.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.602+0000 7f9050000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f90480b4df0 con 0x7f904814c600 2024-04-28T13:31:30.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.603+0000 7f904e000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f9048072560 0x7f904806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56136/0 (socket says 172.21.15.196:56136) 2024-04-28T13:31:30.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.603+0000 7f904d600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f904814c600 0x7f90480d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.603+0000 7f904e000640 1 -- 172.21.15.196:0/2366066030 learned_addr learned my addr 172.21.15.196:0/2366066030 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:30.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.603+0000 7f904cc00640 1 --2- 172.21.15.196:0/2366066030 >> v2:172.21.15.203:3300/0 conn(0x7f90480d1460 0x7f9048072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.604+0000 7f904d600640 1 -- 172.21.15.196:0/2366066030 >> v2:172.21.15.203:3300/0 conn(0x7f90480d1460 msgr2=0x7f9048072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:30.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.604+0000 7f904d600640 1 --2- 172.21.15.196:0/2366066030 >> v2:172.21.15.203:3300/0 conn(0x7f90480d1460 0x7f9048072020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.604+0000 7f904d600640 1 -- 172.21.15.196:0/2366066030 >> v1:172.21.15.196:6789/0 conn(0x7f9048072560 legacy=0x7f904806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.604+0000 7f904d600640 1 -- 172.21.15.196:0/2366066030 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90480cf1d0 con 0x7f904814c600 2024-04-28T13:31:30.606 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.605+0000 7f904d600640 1 --2- 172.21.15.196:0/2366066030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f904814c600 0x7f90480d0f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f90340097a0 tx=0x7f903402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee6103968ce4d3a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.607+0000 7f9047e00640 1 -- 172.21.15.196:0/2366066030 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034037070 con 0x7f904814c600 2024-04-28T13:31:30.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.607+0000 7f9047e00640 1 -- 172.21.15.196:0/2366066030 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9034032440 con 0x7f904814c600 2024-04-28T13:31:30.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.607+0000 7f9047e00640 1 -- 172.21.15.196:0/2366066030 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034032810 con 0x7f904814c600 2024-04-28T13:31:30.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.608+0000 7f8aba000640 1 -- 172.21.15.196:0/3653862757 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8ab0059980 con 0x7f8ac4151510 2024-04-28T13:31:30.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.608+0000 7f9050000640 1 -- 172.21.15.196:0/2366066030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f904814c600 msgr2=0x7f90480d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.608+0000 7f9050000640 1 --2- 172.21.15.196:0/2366066030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f904814c600 0x7f90480d0f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f90340097a0 tx=0x7f903402a180 comp rx=0 tx=0).stop 2024-04-28T13:31:30.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.608+0000 7f9050000640 1 -- 172.21.15.196:0/2366066030 shutdown_connections 2024-04-28T13:31:30.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.608+0000 7f9050000640 1 --2- 172.21.15.196:0/2366066030 >> v2:172.21.15.203:3300/0 conn(0x7f90480d1460 0x7f9048072020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.608+0000 7f9050000640 1 --2- 172.21.15.196:0/2366066030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f904814c600 0x7f90480d0f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.608+0000 7f9050000640 1 -- 172.21.15.196:0/2366066030 >> 172.21.15.196:0/2366066030 conn(0x7f90480cd830 msgr2=0x7f90480cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:30.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.610+0000 7f9050000640 1 -- 172.21.15.196:0/2366066030 shutdown_connections 2024-04-28T13:31:30.612 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.611+0000 7f9050000640 1 -- 172.21.15.196:0/2366066030 wait complete. 2024-04-28T13:31:30.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.613+0000 7f0a57000640 1 Processor -- start 2024-04-28T13:31:30.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.613+0000 7f9050000640 1 Processor -- start 2024-04-28T13:31:30.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.613+0000 7f9050000640 1 -- start start 2024-04-28T13:31:30.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.613+0000 7f0a57000640 1 -- start start 2024-04-28T13:31:30.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.615+0000 7f9050000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9048072560 0x7f90481443f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.616+0000 7f9050000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90480d1460 0x7f90481ed190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.616+0000 7f9050000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f904814c600 0x7f9048146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.616+0000 7f9050000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f90480cfa70 con 0x7f90480d1460 2024-04-28T13:31:30.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.616+0000 7f9050000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f90480cf790 con 0x7f9048072560 2024-04-28T13:31:30.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.616+0000 7f9050000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f90480cff90 con 0x7f904814c600 2024-04-28T13:31:30.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.617+0000 7f904d600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9048072560 0x7f90481443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.616+0000 7f0a57000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0a5014c600 0x7f0a500d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.617+0000 7f0a57000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a50072730 0x7f0a500b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.617+0000 7f0a57000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a500ceef0 con 0x7f0a500d1460 2024-04-28T13:31:30.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.617+0000 7f0a57000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0a5014c9e0 con 0x7f0a5014c600 2024-04-28T13:31:30.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.617+0000 7f0a57000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0a500cf1d0 con 0x7f0a50072730 2024-04-28T13:31:30.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.617+0000 7f0a4f400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f0a500d1460 0x7f0a50072020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56150/0 (socket says 172.21.15.196:56150) 2024-04-28T13:31:30.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.617+0000 7f0a4f400640 1 -- 172.21.15.196:0/2944538226 learned_addr learned my addr 172.21.15.196:0/2944538226 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:30.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.617+0000 7f904d600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9048072560 0x7f90481443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:39892/0 (socket says 172.21.15.196:39892) 2024-04-28T13:31:30.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.617+0000 7f904d600640 1 -- 172.21.15.196:0/2450869996 learned_addr learned my addr 172.21.15.196:0/2450869996 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:30.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.617+0000 7f904e000640 1 --2- 172.21.15.196:0/2450869996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f904814c600 0x7f9048146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.617+0000 7f904cc00640 1 --2- 172.21.15.196:0/2450869996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90480d1460 0x7f90481ed190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.617+0000 7f0a4fe00640 1 --2- 172.21.15.196:0/2944538226 >> v2:172.21.15.196:3300/0 conn(0x7f0a5014c600 0x7f0a500d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.618+0000 7f904e000640 1 -- 172.21.15.196:0/2450869996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9048072560 msgr2=0x7f90481443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.618+0000 7f904e000640 1 --2- 172.21.15.196:0/2450869996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9048072560 0x7f90481443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.618+0000 7f904e000640 1 -- 172.21.15.196:0/2450869996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90480d1460 msgr2=0x7f90481ed190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.618+0000 7f904e000640 1 --2- 172.21.15.196:0/2450869996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90480d1460 0x7f90481ed190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.618+0000 7f904e000640 1 -- 172.21.15.196:0/2450869996 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90481fa790 con 0x7f904814c600 2024-04-28T13:31:30.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.618+0000 7f904cc00640 1 --2- 172.21.15.196:0/2450869996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90480d1460 0x7f90481ed190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:30.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.618+0000 7f0a4fe00640 1 -- 172.21.15.196:0/2944538226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a50072730 msgr2=0x7f0a500b3de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-28T13:31:30.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.618+0000 7f0a4fe00640 1 --2- 172.21.15.196:0/2944538226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a50072730 0x7f0a500b3de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.618+0000 7f0a4fe00640 1 -- 172.21.15.196:0/2944538226 >> v1:172.21.15.196:6789/0 conn(0x7f0a500d1460 legacy=0x7f0a50072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.618+0000 7f0a4fe00640 1 -- 172.21.15.196:0/2944538226 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a500cf4b0 con 0x7f0a5014c600 2024-04-28T13:31:30.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.619+0000 7f904e000640 1 --2- 172.21.15.196:0/2450869996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f904814c600 0x7f9048146f10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f904000ca70 tx=0x7f904000cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.620+0000 7f9045600640 1 -- 172.21.15.196:0/2450869996 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040017070 con 0x7f904814c600 2024-04-28T13:31:30.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.620+0000 7f9045600640 1 -- 172.21.15.196:0/2450869996 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f904000aab0 con 0x7f904814c600 2024-04-28T13:31:30.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.620+0000 7f9045600640 1 -- 172.21.15.196:0/2450869996 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040012400 con 0x7f904814c600 2024-04-28T13:31:30.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.620+0000 7f0a4fe00640 1 --2- 172.21.15.196:0/2944538226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a5014c600 0x7f0a500d0f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a400097a0 tx=0x7f0a4002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401766258009ffae server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.621+0000 7f0a4ea00640 1 -- 172.21.15.196:0/2944538226 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40037070 con 0x7f0a5014c600 2024-04-28T13:31:30.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.621+0000 7f0a4ea00640 1 -- 172.21.15.196:0/2944538226 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a40032440 con 0x7f0a5014c600 2024-04-28T13:31:30.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.621+0000 7f0a4ea00640 1 -- 172.21.15.196:0/2944538226 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40037070 con 0x7f0a5014c600 2024-04-28T13:31:30.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.622+0000 7f0a57000640 1 -- 172.21.15.196:0/2944538226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a5014c600 msgr2=0x7f0a500d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.622+0000 7f0a57000640 1 --2- 172.21.15.196:0/2944538226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a5014c600 0x7f0a500d0f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a400097a0 tx=0x7f0a4002a1a0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.622+0000 7f9050000640 1 -- 172.21.15.196:0/2450869996 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90481faa70 con 0x7f904814c600 2024-04-28T13:31:30.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.622+0000 7f9050000640 1 -- 172.21.15.196:0/2450869996 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f90481fb030 con 0x7f904814c600 2024-04-28T13:31:30.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.623+0000 7f0a57000640 1 -- 172.21.15.196:0/2944538226 shutdown_connections 2024-04-28T13:31:30.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.623+0000 7f0a57000640 1 --2- 172.21.15.196:0/2944538226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a50072730 0x7f0a500b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.623+0000 7f0a57000640 1 --2- 172.21.15.196:0/2944538226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a5014c600 0x7f0a500d0f20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.623+0000 7f0a57000640 1 -- 172.21.15.196:0/2944538226 >> 172.21.15.196:0/2944538226 conn(0x7f0a500cd830 msgr2=0x7f0a500cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:30.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.623+0000 7f0a57000640 1 -- 172.21.15.196:0/2944538226 shutdown_connections 2024-04-28T13:31:30.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.623+0000 7f0a57000640 1 -- 172.21.15.196:0/2944538226 wait complete. 2024-04-28T13:31:30.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.626+0000 7f0a57000640 1 Processor -- start 2024-04-28T13:31:30.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.626+0000 7f0a57000640 1 -- start start 2024-04-28T13:31:30.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.628+0000 7f0a57000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a50072730 0x7f0a500b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.626+0000 7f9045600640 1 -- 172.21.15.196:0/2450869996 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f904001a070 con 0x7f904814c600 2024-04-28T13:31:30.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.628+0000 7f9045600640 1 --2- 172.21.15.196:0/2450869996 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f902806c8f0 0x7f902806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.628+0000 7f9045600640 1 -- 172.21.15.196:0/2450869996 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f904008b740 con 0x7f904814c600 2024-04-28T13:31:30.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.628+0000 7f9050000640 1 -- 172.21.15.196:0/2450869996 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f904806a290 con 0x7f904814c600 2024-04-28T13:31:30.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.628+0000 7f0a57000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a500d1460 0x7f0a50201c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.628+0000 7f0a57000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a5014c600 0x7f0a50202180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.628+0000 7f0a57000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0a500cff90 con 0x7f0a50072730 2024-04-28T13:31:30.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.628+0000 7f0a57000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0a500cfa70 con 0x7f0a5014c600 2024-04-28T13:31:30.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.628+0000 7f0a57000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0a500ceb50 con 0x7f0a500d1460 2024-04-28T13:31:30.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.629+0000 7f0a4fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a50072730 0x7f0a500b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.629+0000 7f904d600640 1 --2- 172.21.15.196:0/2450869996 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f902806c8f0 0x7f902806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.629+0000 7f0a55000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a5014c600 0x7f0a50202180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.629+0000 7f0a4f400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a500d1460 0x7f0a50201c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.629+0000 7f0a55000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a5014c600 0x7f0a50202180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:39908/0 (socket says 172.21.15.196:39908) 2024-04-28T13:31:30.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.629+0000 7f0a55000640 1 -- 172.21.15.196:0/1702690857 learned_addr learned my addr 172.21.15.196:0/1702690857 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:30.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.629+0000 7f0a4f400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a500d1460 0x7f0a50201c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:45008/0 (socket says 172.21.15.196:45008) 2024-04-28T13:31:30.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.630+0000 7f0a55000640 1 -- 172.21.15.196:0/1702690857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a500d1460 msgr2=0x7f0a50201c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.630+0000 7f0a55000640 1 --2- 172.21.15.196:0/1702690857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a500d1460 0x7f0a50201c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.630+0000 7f0a55000640 1 -- 172.21.15.196:0/1702690857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a50072730 msgr2=0x7f0a500b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.630+0000 7f0a55000640 1 --2- 172.21.15.196:0/1702690857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a50072730 0x7f0a500b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.630+0000 7f0a55000640 1 -- 172.21.15.196:0/1702690857 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a502029b0 con 0x7f0a5014c600 2024-04-28T13:31:30.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.631+0000 7f0a55000640 1 --2- 172.21.15.196:0/1702690857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a5014c600 0x7f0a50202180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800b9e0 tx=0x7f0a4800bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.632+0000 7f904d600640 1 --2- 172.21.15.196:0/2450869996 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f902806c8f0 0x7f902806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f903402a1c0 tx=0x7f903402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.632+0000 7f0a47e00640 1 -- 172.21.15.196:0/1702690857 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48019070 con 0x7f0a5014c600 2024-04-28T13:31:30.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.632+0000 7f0a57000640 1 -- 172.21.15.196:0/1702690857 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a50143560 con 0x7f0a5014c600 2024-04-28T13:31:30.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.632+0000 7f0a57000640 1 -- 172.21.15.196:0/1702690857 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a50143b20 con 0x7f0a5014c600 2024-04-28T13:31:30.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.632+0000 7f0a47e00640 1 -- 172.21.15.196:0/1702690857 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4800ca40 con 0x7f0a5014c600 2024-04-28T13:31:30.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.632+0000 7f0a47e00640 1 -- 172.21.15.196:0/1702690857 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48014440 con 0x7f0a5014c600 2024-04-28T13:31:30.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.634+0000 7f0a57000640 1 -- 172.21.15.196:0/1702690857 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a28005290 con 0x7f0a5014c600 2024-04-28T13:31:30.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.637+0000 7f0a47e00640 1 -- 172.21.15.196:0/1702690857 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a4801c070 con 0x7f0a5014c600 2024-04-28T13:31:30.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.640+0000 7f0a47e00640 1 --2- 172.21.15.196:0/1702690857 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0a3006c8f0 0x7f0a3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:30.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.640+0000 7f0a47e00640 1 -- 172.21.15.196:0/1702690857 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0a4808bb10 con 0x7f0a5014c600 2024-04-28T13:31:30.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.640+0000 7f0a4fe00640 1 --2- 172.21.15.196:0/1702690857 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0a3006c8f0 0x7f0a3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:30.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.642+0000 7f0a4fe00640 1 --2- 172.21.15.196:0/1702690857 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0a3006c8f0 0x7f0a3006edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a40034000 tx=0x7f0a4002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:30.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.645+0000 7f9045600640 1 -- 172.21.15.196:0/2450869996 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9040059480 con 0x7f904814c600 2024-04-28T13:31:30.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.650+0000 7f0a47e00640 1 -- 172.21.15.196:0/1702690857 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0a48023020 con 0x7f0a5014c600 2024-04-28T13:31:30.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.969+0000 7fe7c7000640 1 -- 172.21.15.196:0/2235858608 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe7b8056be0 con 0x7fe7b8053ca0 2024-04-28T13:31:30.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.970+0000 7fe7b3e00640 1 -- 172.21.15.196:0/2235858608 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe7ac05cf20 con 0x7fe7b8053ca0 2024-04-28T13:31:30.971 INFO:teuthology.orchestra.run.smithi196.stdout:55834574854 2024-04-28T13:31:30.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.974+0000 7fe7c7000640 1 -- 172.21.15.196:0/2235858608 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe7a006c940 msgr2=0x7fe7a006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.974+0000 7fe7c7000640 1 --2- 172.21.15.196:0/2235858608 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe7a006c940 0x7fe7a006ee00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4000f60 tx=0x7fe7b4007e10 comp rx=0 tx=0).stop 2024-04-28T13:31:30.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.974+0000 7fe7c7000640 1 -- 172.21.15.196:0/2235858608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7b8053ca0 msgr2=0x7fe7b816cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:30.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.974+0000 7fe7c7000640 1 --2- 172.21.15.196:0/2235858608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7b8053ca0 0x7fe7b816cad0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00b420 tx=0x7fe7ac00b8e0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.975+0000 7fe7c7000640 1 -- 172.21.15.196:0/2235858608 shutdown_connections 2024-04-28T13:31:30.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.975+0000 7fe7c7000640 1 --2- 172.21.15.196:0/2235858608 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe7a006c940 0x7fe7a006ee00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.975+0000 7fe7c7000640 1 --2- 172.21.15.196:0/2235858608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7b80e3150 0x7fe7b816d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.975+0000 7fe7c7000640 1 --2- 172.21.15.196:0/2235858608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7b8053ca0 0x7fe7b816cad0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.975+0000 7fe7c7000640 1 --2- 172.21.15.196:0/2235858608 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7b803c090 0x7fe7b804ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:30.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.975+0000 7fe7c7000640 1 -- 172.21.15.196:0/2235858608 >> 172.21.15.196:0/2235858608 conn(0x7fe7b8058ed0 msgr2=0x7fe7b80e6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:30.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.976+0000 7fe7c7000640 1 -- 172.21.15.196:0/2235858608 shutdown_connections 2024-04-28T13:31:30.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:30.976+0000 7fe7c7000640 1 -- 172.21.15.196:0/2235858608 wait complete. 2024-04-28T13:31:31.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-04-28T13:31:31.006 DEBUG:teuthology.parallel:result is None 2024-04-28T13:31:31.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.006+0000 7fd8e2200640 1 -- 172.21.15.196:0/986599602 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd8dc1621b0 con 0x7fd8dc162d90 2024-04-28T13:31:31.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.006+0000 7fd8d3400640 1 -- 172.21.15.196:0/986599602 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd8d4059b20 con 0x7fd8dc162d90 2024-04-28T13:31:31.008 INFO:teuthology.orchestra.run.smithi196.stdout:55834574855 2024-04-28T13:31:31.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.007+0000 7fa29f200640 1 -- 172.21.15.196:0/3654307534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa2a00b14b0 con 0x7fa2a01e5150 2024-04-28T13:31:31.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.008+0000 7fa296000640 1 -- 172.21.15.196:0/3654307534 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa28c07e9b0 con 0x7fa2a01e5150 2024-04-28T13:31:31.009 INFO:teuthology.orchestra.run.smithi196.stdout:55834574854 2024-04-28T13:31:31.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.010+0000 7fa29f200640 1 -- 172.21.15.196:0/3654307534 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa28006c960 msgr2=0x7fa28006ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.010+0000 7fa29f200640 1 --2- 172.21.15.196:0/3654307534 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa28006c960 0x7fa28006ee20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa2980074a0 tx=0x7fa29800e040 comp rx=0 tx=0).stop 2024-04-28T13:31:31.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.010+0000 7fa29f200640 1 -- 172.21.15.196:0/3654307534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2a01e5150 msgr2=0x7fa2a01f5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.010+0000 7fa29f200640 1 --2- 172.21.15.196:0/3654307534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2a01e5150 0x7fa2a01f5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa2a015fde0 tx=0x7fa28c001970 comp rx=0 tx=0).stop 2024-04-28T13:31:31.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.010+0000 7fd8e2200640 1 -- 172.21.15.196:0/986599602 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd8bc06c940 msgr2=0x7fd8bc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.011+0000 7fd8e2200640 1 --2- 172.21.15.196:0/986599602 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd8bc06c940 0x7fd8bc06ee00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc034000 tx=0x7fd8cc02e000 comp rx=0 tx=0).stop 2024-04-28T13:31:31.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.010+0000 7fa29f200640 1 -- 172.21.15.196:0/3654307534 shutdown_connections 2024-04-28T13:31:31.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.010+0000 7fa29f200640 1 --2- 172.21.15.196:0/3654307534 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa28006c960 0x7fa28006ee20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.010+0000 7fa29f200640 1 --2- 172.21.15.196:0/3654307534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2a01e5150 0x7fa2a01f5700 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.010+0000 7fa29f200640 1 --2- 172.21.15.196:0/3654307534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2a01e7630 0x7fa2a00b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.010+0000 7fa29f200640 1 --2- 172.21.15.196:0/3654307534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2840051c0 0x7fa2a014fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.010+0000 7fa29f200640 1 -- 172.21.15.196:0/3654307534 >> 172.21.15.196:0/3654307534 conn(0x7fa2a00b2990 msgr2=0x7fa2a015c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:31.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.011+0000 7fd8e2200640 1 -- 172.21.15.196:0/986599602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8dc162d90 msgr2=0x7fd8dc0ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.011+0000 7fd8e2200640 1 --2- 172.21.15.196:0/986599602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8dc162d90 0x7fd8dc0ca410 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400d1b0 tx=0x7fd8d400d670 comp rx=0 tx=0).stop 2024-04-28T13:31:31.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.011+0000 7fa29f200640 1 -- 172.21.15.196:0/3654307534 shutdown_connections 2024-04-28T13:31:31.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.011+0000 7fa29f200640 1 -- 172.21.15.196:0/3654307534 wait complete. 2024-04-28T13:31:31.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.011+0000 7fd8e2200640 1 -- 172.21.15.196:0/986599602 shutdown_connections 2024-04-28T13:31:31.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.011+0000 7fd8e2200640 1 --2- 172.21.15.196:0/986599602 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd8bc06c940 0x7fd8bc06ee00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.011+0000 7fd8e2200640 1 --2- 172.21.15.196:0/986599602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8dc162d90 0x7fd8dc0ca410 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.011+0000 7fd8e2200640 1 --2- 172.21.15.196:0/986599602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8dc1629b0 0x7fd8dc15c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.011+0000 7fd8e2200640 1 --2- 172.21.15.196:0/986599602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8dc0d19d0 0x7fd8dc0c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.011+0000 7fd8e2200640 1 -- 172.21.15.196:0/986599602 >> 172.21.15.196:0/986599602 conn(0x7fd8dc09e4d0 msgr2=0x7fd8dc161aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:31.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.011+0000 7fd8e2200640 1 -- 172.21.15.196:0/986599602 shutdown_connections 2024-04-28T13:31:31.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.012+0000 7fd8e2200640 1 -- 172.21.15.196:0/986599602 wait complete. 2024-04-28T13:31:31.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.025+0000 7fe6a8a00640 1 -- 172.21.15.196:0/644869295 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe6a4073ff0 con 0x7fe6a40d1460 2024-04-28T13:31:31.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.026+0000 7fe69a000640 1 -- 172.21.15.196:0/644869295 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe69c05d3f0 con 0x7fe6a40d1460 2024-04-28T13:31:31.028 INFO:teuthology.orchestra.run.smithi196.stdout:55834574853 2024-04-28T13:31:31.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-04-28T13:31:31.030 DEBUG:teuthology.parallel:result is None 2024-04-28T13:31:31.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.030+0000 7fe6a8a00640 1 -- 172.21.15.196:0/644869295 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe68406c9c0 msgr2=0x7fe68406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.030+0000 7fe6a8a00640 1 --2- 172.21.15.196:0/644869295 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe68406c9c0 0x7fe68406ee80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe694006fd0 tx=0x7fe694008040 comp rx=0 tx=0).stop 2024-04-28T13:31:31.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.030+0000 7fe6a8a00640 1 -- 172.21.15.196:0/644869295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a40d1460 msgr2=0x7fe6a41625a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.030+0000 7fe6a8a00640 1 --2- 172.21.15.196:0/644869295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a40d1460 0x7fe6a41625a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00c8a0 tx=0x7fe69c00cd60 comp rx=0 tx=0).stop 2024-04-28T13:31:31.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.030+0000 7fe6a8a00640 1 -- 172.21.15.196:0/644869295 shutdown_connections 2024-04-28T13:31:31.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.030+0000 7fe6a8a00640 1 --2- 172.21.15.196:0/644869295 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe68406c9c0 0x7fe68406ee80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.030+0000 7fe6a8a00640 1 --2- 172.21.15.196:0/644869295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a414c600 0x7fe6a4162ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.030+0000 7fe6a8a00640 1 --2- 172.21.15.196:0/644869295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a40d1460 0x7fe6a41625a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.030+0000 7fe6a8a00640 1 --2- 172.21.15.196:0/644869295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a4072730 0x7fe6a4137280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.030+0000 7fe6a8a00640 1 -- 172.21.15.196:0/644869295 >> 172.21.15.196:0/644869295 conn(0x7fe6a40cd830 msgr2=0x7fe6a414ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:31.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.030+0000 7fe6a8a00640 1 -- 172.21.15.196:0/644869295 shutdown_connections 2024-04-28T13:31:31.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.030+0000 7fe6a8a00640 1 -- 172.21.15.196:0/644869295 wait complete. 2024-04-28T13:31:31.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.032+0000 7efd80a00640 1 -- 172.21.15.196:0/3026862417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7efd58005580 con 0x7efd7404eb90 2024-04-28T13:31:31.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.034+0000 7efd6f400640 1 -- 172.21.15.196:0/3026862417 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efd68058c20 con 0x7efd7404eb90 2024-04-28T13:31:31.035 INFO:teuthology.orchestra.run.smithi196.stdout:55834574854 2024-04-28T13:31:31.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.035+0000 7f8ac9000640 1 -- 172.21.15.196:0/3653862757 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f8a9c005850 con 0x7f8ac4151510 2024-04-28T13:31:31.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.036+0000 7f8aba000640 1 -- 172.21.15.196:0/3653862757 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8ab005dc80 con 0x7f8ac4151510 2024-04-28T13:31:31.037 INFO:teuthology.orchestra.run.smithi196.stdout:55834574856 2024-04-28T13:31:31.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-04-28T13:31:31.038 DEBUG:teuthology.parallel:result is None 2024-04-28T13:31:31.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.038+0000 7efd80a00640 1 -- 172.21.15.196:0/3026862417 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efd5c06c920 msgr2=0x7efd5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.038+0000 7efd80a00640 1 --2- 172.21.15.196:0/3026862417 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efd5c06c920 0x7efd5c06ede0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efd70007d00 tx=0x7efd70009040 comp rx=0 tx=0).stop 2024-04-28T13:31:31.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.038+0000 7efd80a00640 1 -- 172.21.15.196:0/3026862417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd7404eb90 msgr2=0x7efd7404f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.038+0000 7efd80a00640 1 --2- 172.21.15.196:0/3026862417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd7404eb90 0x7efd7404f020 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd74059f00 tx=0x7efd6800e800 comp rx=0 tx=0).stop 2024-04-28T13:31:31.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.038+0000 7efd80a00640 1 -- 172.21.15.196:0/3026862417 shutdown_connections 2024-04-28T13:31:31.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.038+0000 7efd80a00640 1 --2- 172.21.15.196:0/3026862417 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efd5c06c920 0x7efd5c06ede0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.038+0000 7efd80a00640 1 --2- 172.21.15.196:0/3026862417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd7404eb90 0x7efd7404f020 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.038+0000 7efd80a00640 1 --2- 172.21.15.196:0/3026862417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd74050e00 0x7efd7404d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.038+0000 7efd80a00640 1 --2- 172.21.15.196:0/3026862417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd7404d350 0x7efd7404ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.038+0000 7efd80a00640 1 -- 172.21.15.196:0/3026862417 >> 172.21.15.196:0/3026862417 conn(0x7efd740559e0 msgr2=0x7efd7403b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:31.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.039+0000 7efd80a00640 1 -- 172.21.15.196:0/3026862417 shutdown_connections 2024-04-28T13:31:31.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.039+0000 7efd80a00640 1 -- 172.21.15.196:0/3026862417 wait complete. 2024-04-28T13:31:31.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.039+0000 7f8ac9000640 1 -- 172.21.15.196:0/3653862757 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8aa406c940 msgr2=0x7f8aa406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.039+0000 7f8ac9000640 1 --2- 172.21.15.196:0/3653862757 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8aa406c940 0x7f8aa406ee00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8abc02a5d0 tx=0x7f8abc02f000 comp rx=0 tx=0).stop 2024-04-28T13:31:31.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.039+0000 7f8ac9000640 1 -- 172.21.15.196:0/3653862757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac4151510 msgr2=0x7f8ac406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.039+0000 7f8ac9000640 1 --2- 172.21.15.196:0/3653862757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac4151510 0x7f8ac406d350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000e7f0 tx=0x7f8ab000ecb0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.039+0000 7f8ac9000640 1 -- 172.21.15.196:0/3653862757 shutdown_connections 2024-04-28T13:31:31.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.039+0000 7f8ac9000640 1 --2- 172.21.15.196:0/3653862757 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8aa406c940 0x7f8aa406ee00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.039+0000 7f8ac9000640 1 --2- 172.21.15.196:0/3653862757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac4151510 0x7f8ac406d350 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.039+0000 7f8ac9000640 1 --2- 172.21.15.196:0/3653862757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac40d5810 0x7f8ac4150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.039+0000 7f8ac9000640 1 --2- 172.21.15.196:0/3653862757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ac40d2e20 0x7f8ac4150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.039+0000 7f8ac9000640 1 -- 172.21.15.196:0/3653862757 >> 172.21.15.196:0/3653862757 conn(0x7f8ac408ebf0 msgr2=0x7f8ac40b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:31.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.040+0000 7f8ac9000640 1 -- 172.21.15.196:0/3653862757 shutdown_connections 2024-04-28T13:31:31.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.040+0000 7f8ac9000640 1 -- 172.21.15.196:0/3653862757 wait complete. 2024-04-28T13:31:31.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-04-28T13:31:31.053 DEBUG:teuthology.parallel:result is None 2024-04-28T13:31:31.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.055+0000 7f9050000640 1 -- 172.21.15.196:0/2450869996 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f90480b4700 con 0x7f904814c600 2024-04-28T13:31:31.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.056+0000 7f9045600640 1 -- 172.21.15.196:0/2450869996 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f904005d780 con 0x7f904814c600 2024-04-28T13:31:31.057 INFO:teuthology.orchestra.run.smithi196.stdout:55834574855 2024-04-28T13:31:31.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.058+0000 7f9050000640 1 -- 172.21.15.196:0/2450869996 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f902806c8f0 msgr2=0x7f902806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.058+0000 7f9050000640 1 --2- 172.21.15.196:0/2450869996 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f902806c8f0 0x7f902806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f903402a1c0 tx=0x7f903402e000 comp rx=0 tx=0).stop 2024-04-28T13:31:31.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.058+0000 7f9050000640 1 -- 172.21.15.196:0/2450869996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f904814c600 msgr2=0x7f9048146f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.060 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.058+0000 7f9050000640 1 --2- 172.21.15.196:0/2450869996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f904814c600 0x7f9048146f10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f904000ca70 tx=0x7f904000cf30 comp rx=0 tx=0).stop 2024-04-28T13:31:31.060 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.058+0000 7f9050000640 1 -- 172.21.15.196:0/2450869996 shutdown_connections 2024-04-28T13:31:31.060 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.058+0000 7f9050000640 1 --2- 172.21.15.196:0/2450869996 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f902806c8f0 0x7f902806edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.060 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.058+0000 7f9050000640 1 --2- 172.21.15.196:0/2450869996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f904814c600 0x7f9048146f10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.060 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.058+0000 7f9050000640 1 --2- 172.21.15.196:0/2450869996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90480d1460 0x7f90481ed190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.060 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.058+0000 7f9050000640 1 --2- 172.21.15.196:0/2450869996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9048072560 0x7f90481443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.060 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.058+0000 7f9050000640 1 -- 172.21.15.196:0/2450869996 >> 172.21.15.196:0/2450869996 conn(0x7f90480cd830 msgr2=0x7f90480b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:31.060 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.059+0000 7f9050000640 1 -- 172.21.15.196:0/2450869996 shutdown_connections 2024-04-28T13:31:31.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-04-28T13:31:31.061 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.059+0000 7f9050000640 1 -- 172.21.15.196:0/2450869996 wait complete. 2024-04-28T13:31:31.061 DEBUG:teuthology.parallel:result is None 2024-04-28T13:31:31.065 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-04-28T13:31:31.065 DEBUG:teuthology.parallel:result is None 2024-04-28T13:31:31.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.073+0000 7f0a57000640 1 -- 172.21.15.196:0/1702690857 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0a28005850 con 0x7f0a5014c600 2024-04-28T13:31:31.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.074+0000 7f0a47e00640 1 -- 172.21.15.196:0/1702690857 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0a48059960 con 0x7f0a5014c600 2024-04-28T13:31:31.075 INFO:teuthology.orchestra.run.smithi196.stdout:55834574855 2024-04-28T13:31:31.076 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-04-28T13:31:31.076 DEBUG:teuthology.parallel:result is None 2024-04-28T13:31:31.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.076+0000 7f0a57000640 1 -- 172.21.15.196:0/1702690857 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0a3006c8f0 msgr2=0x7f0a3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.076+0000 7f0a57000640 1 --2- 172.21.15.196:0/1702690857 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0a3006c8f0 0x7f0a3006edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a40034000 tx=0x7f0a4002e000 comp rx=0 tx=0).stop 2024-04-28T13:31:31.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.076+0000 7f0a57000640 1 -- 172.21.15.196:0/1702690857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a5014c600 msgr2=0x7f0a50202180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.076+0000 7f0a57000640 1 --2- 172.21.15.196:0/1702690857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a5014c600 0x7f0a50202180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800b9e0 tx=0x7f0a4800bea0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.076+0000 7f0a57000640 1 -- 172.21.15.196:0/1702690857 shutdown_connections 2024-04-28T13:31:31.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.076+0000 7f0a57000640 1 --2- 172.21.15.196:0/1702690857 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0a3006c8f0 0x7f0a3006edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.076+0000 7f0a57000640 1 --2- 172.21.15.196:0/1702690857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a5014c600 0x7f0a50202180 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.076+0000 7f0a57000640 1 --2- 172.21.15.196:0/1702690857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a500d1460 0x7f0a50201c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.076+0000 7f0a57000640 1 --2- 172.21.15.196:0/1702690857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a50072730 0x7f0a500b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.076+0000 7f0a57000640 1 -- 172.21.15.196:0/1702690857 >> 172.21.15.196:0/1702690857 conn(0x7f0a500cd830 msgr2=0x7f0a501487d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:31.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.076+0000 7f0a57000640 1 -- 172.21.15.196:0/1702690857 shutdown_connections 2024-04-28T13:31:31.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.076+0000 7f0a57000640 1 -- 172.21.15.196:0/1702690857 wait complete. 2024-04-28T13:31:31.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-04-28T13:31:31.093 DEBUG:teuthology.parallel:result is None 2024-04-28T13:31:31.094 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-28T13:31:31.094 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T13:31:31.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.321+0000 7f2191800640 1 Processor -- start 2024-04-28T13:31:31.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.321+0000 7f2191800640 1 -- start start 2024-04-28T13:31:31.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.322+0000 7f2191800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f218c0d1190 0x7f218c0b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.322+0000 7f2191800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f218c160f40 con 0x7f218c0d0d00 2024-04-28T13:31:31.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.322+0000 7f2191800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f218c161220 con 0x7f218c1469f0 2024-04-28T13:31:31.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.322+0000 7f2191800640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f218c0b5960 con 0x7f218c0d1190 2024-04-28T13:31:31.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.323+0000 7f218a000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f218c0d0d00 0x7f218c0b5dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56162/0 (socket says 172.21.15.196:56162) 2024-04-28T13:31:31.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.323+0000 7f218a000640 1 -- 172.21.15.196:0/2773912823 learned_addr learned my addr 172.21.15.196:0/2773912823 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:31.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.323+0000 7f218b400640 1 --2- 172.21.15.196:0/2773912823 >> v2:172.21.15.203:3300/0 conn(0x7f218c0d1190 0x7f218c0b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:31.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.323+0000 7f2189600640 1 -- 172.21.15.196:0/2773912823 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3421359927 0 0) 0x7f218c160f40 con 0x7f218c0d0d00 2024-04-28T13:31:31.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.323+0000 7f2189600640 1 -- 172.21.15.196:0/2773912823 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2170003520 con 0x7f218c0d0d00 2024-04-28T13:31:31.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.323+0000 7f2189600640 1 -- 172.21.15.196:0/2773912823 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1726262777 0 0) 0x7f2170003520 con 0x7f218c0d0d00 2024-04-28T13:31:31.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.323+0000 7f2189600640 1 -- 172.21.15.196:0/2773912823 >> v2:172.21.15.203:3300/0 conn(0x7f218c0d1190 msgr2=0x7f218c0b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:31.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.323+0000 7f2189600640 1 --2- 172.21.15.196:0/2773912823 >> v2:172.21.15.203:3300/0 conn(0x7f218c0d1190 0x7f218c0b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.323+0000 7f2189600640 1 -- 172.21.15.196:0/2773912823 >> v1:172.21.15.203:6789/0 conn(0x7f218c1469f0 legacy=0x7f218c0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.323+0000 7f2189600640 1 -- 172.21.15.196:0/2773912823 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218c161500 con 0x7f218c0d0d00 2024-04-28T13:31:31.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.323+0000 7f2189600640 1 -- 172.21.15.196:0/2773912823 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f2178001670 con 0x7f218c0d0d00 2024-04-28T13:31:31.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.323+0000 7f2189600640 1 -- 172.21.15.196:0/2773912823 >> v1:172.21.15.196:6789/0 conn(0x7f218c0d0d00 legacy=0x7f218c0b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.323+0000 7f2189600640 1 --2- 172.21.15.196:0/2773912823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21700039b0 0x7f2170003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.323+0000 7f2189600640 1 --2- 172.21.15.196:0/2773912823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2170004800 0x7f2170004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.324+0000 7f2189600640 1 --2- 172.21.15.196:0/2773912823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21700051c0 0x7f217000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.324+0000 7f2189600640 1 -- 172.21.15.196:0/2773912823 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f217000c030 con 0x7f21700039b0 2024-04-28T13:31:31.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.324+0000 7f2189600640 1 -- 172.21.15.196:0/2773912823 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f217000c1e0 con 0x7f2170004800 2024-04-28T13:31:31.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.324+0000 7f2189600640 1 -- 172.21.15.196:0/2773912823 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f217000c3e0 con 0x7f21700051c0 2024-04-28T13:31:31.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.324+0000 7f218aa00640 1 --2- 172.21.15.196:0/2773912823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21700039b0 0x7f2170003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:31.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.324+0000 7f218a000640 1 --2- 172.21.15.196:0/2773912823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2170004800 0x7f2170004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:31.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.324+0000 7f218aa00640 1 -- 172.21.15.196:0/2773912823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21700051c0 msgr2=0x7f217000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:31.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.324+0000 7f218aa00640 1 --2- 172.21.15.196:0/2773912823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21700051c0 0x7f217000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.324+0000 7f218aa00640 1 -- 172.21.15.196:0/2773912823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2170004800 msgr2=0x7f2170004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:31.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.324+0000 7f218aa00640 1 --2- 172.21.15.196:0/2773912823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2170004800 0x7f2170004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.324+0000 7f218aa00640 1 -- 172.21.15.196:0/2773912823 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2170003520 con 0x7f21700039b0 2024-04-28T13:31:31.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.324+0000 7f218b400640 1 -- 172.21.15.196:0/2773912823 reap_dead start 2024-04-28T13:31:31.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.325+0000 7f218aa00640 1 --2- 172.21.15.196:0/2773912823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21700039b0 0x7f2170003dd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f217c001750 tx=0x7f2170005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c9382de1eb8b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:31.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.325+0000 7f2189600640 1 -- 172.21.15.196:0/2773912823 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c001370 con 0x7f21700039b0 2024-04-28T13:31:31.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.325+0000 7f2189600640 1 -- 172.21.15.196:0/2773912823 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f217c0052d0 con 0x7f21700039b0 2024-04-28T13:31:31.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.325+0000 7f2189600640 1 -- 172.21.15.196:0/2773912823 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c02abe0 con 0x7f21700039b0 2024-04-28T13:31:31.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.326+0000 7f2191800640 1 -- 172.21.15.196:0/2773912823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21700039b0 msgr2=0x7f2170003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.326+0000 7f2191800640 1 --2- 172.21.15.196:0/2773912823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21700039b0 0x7f2170003dd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f217c001750 tx=0x7f2170005830 comp rx=0 tx=0).stop 2024-04-28T13:31:31.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.326+0000 7f2191800640 1 -- 172.21.15.196:0/2773912823 shutdown_connections 2024-04-28T13:31:31.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.326+0000 7f2191800640 1 --2- 172.21.15.196:0/2773912823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21700039b0 0x7f2170003dd0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.326+0000 7f2191800640 1 -- 172.21.15.196:0/2773912823 >> 172.21.15.196:0/2773912823 conn(0x7f218c0cd830 msgr2=0x7f218c0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:31.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.326+0000 7f2191800640 1 -- 172.21.15.196:0/2773912823 shutdown_connections 2024-04-28T13:31:31.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.326+0000 7f2191800640 1 -- 172.21.15.196:0/2773912823 wait complete. 2024-04-28T13:31:31.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.328+0000 7f2191800640 1 Processor -- start 2024-04-28T13:31:31.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.328+0000 7f2191800640 1 -- start start 2024-04-28T13:31:31.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.329+0000 7f2191800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21700039b0 0x7f218c0ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.329+0000 7f2191800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f218c15cd80 0x7f218c13c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.329+0000 7f2191800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f218c0ac470 0x7f218c0ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.329+0000 7f2191800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f218c072020 con 0x7f21700039b0 2024-04-28T13:31:31.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.329+0000 7f2191800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f218c161ac0 con 0x7f218c15cd80 2024-04-28T13:31:31.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.329+0000 7f2191800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f218c0d15c0 con 0x7f218c0ac470 2024-04-28T13:31:31.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.329+0000 7f218aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21700039b0 0x7f218c0ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:31.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.329+0000 7f218aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21700039b0 0x7f218c0ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35950/0 (socket says 172.21.15.196:35950) 2024-04-28T13:31:31.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.329+0000 7f218aa00640 1 -- 172.21.15.196:0/2307711541 learned_addr learned my addr 172.21.15.196:0/2307711541 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:31.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.329+0000 7f218aa00640 1 -- 172.21.15.196:0/2307711541 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f218c0ac470 msgr2=0x7f218c0ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.329+0000 7f218aa00640 1 --2- 172.21.15.196:0/2307711541 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f218c0ac470 0x7f218c0ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.329+0000 7f218aa00640 1 -- 172.21.15.196:0/2307711541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f218c15cd80 msgr2=0x7f218c13c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.329+0000 7f218aa00640 1 --2- 172.21.15.196:0/2307711541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f218c15cd80 0x7f218c13c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.329+0000 7f218aa00640 1 -- 172.21.15.196:0/2307711541 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218c0af790 con 0x7f21700039b0 2024-04-28T13:31:31.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.329+0000 7f218aa00640 1 --2- 172.21.15.196:0/2307711541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21700039b0 0x7f218c0ae6d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f217c005040 tx=0x7f217c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:31.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.330+0000 7f2182a00640 1 -- 172.21.15.196:0/2307711541 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c037040 con 0x7f21700039b0 2024-04-28T13:31:31.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.330+0000 7f2182a00640 1 -- 172.21.15.196:0/2307711541 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f217c040070 con 0x7f21700039b0 2024-04-28T13:31:31.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.330+0000 7f2182a00640 1 -- 172.21.15.196:0/2307711541 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c001370 con 0x7f21700039b0 2024-04-28T13:31:31.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.330+0000 7f2191800640 1 -- 172.21.15.196:0/2307711541 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f218c0afa70 con 0x7f21700039b0 2024-04-28T13:31:31.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.330+0000 7f2191800640 1 -- 172.21.15.196:0/2307711541 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f218c0b0030 con 0x7f21700039b0 2024-04-28T13:31:31.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.331+0000 7f2191800640 1 -- 172.21.15.196:0/2307711541 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f218c06a290 con 0x7f21700039b0 2024-04-28T13:31:31.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.332+0000 7f2182a00640 1 -- 172.21.15.196:0/2307711541 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f217c0052d0 con 0x7f21700039b0 2024-04-28T13:31:31.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.333+0000 7f2182a00640 1 --2- 172.21.15.196:0/2307711541 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f216c06c8f0 0x7f216c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.333+0000 7f2182a00640 1 -- 172.21.15.196:0/2307711541 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f217c0ab7f0 con 0x7f21700039b0 2024-04-28T13:31:31.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.333+0000 7f218a000640 1 --2- 172.21.15.196:0/2307711541 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f216c06c8f0 0x7f216c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:31.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.334+0000 7f218a000640 1 --2- 172.21.15.196:0/2307711541 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f216c06c8f0 0x7f216c06edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f21780017e0 tx=0x7f2178009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:31.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.337+0000 7f2182a00640 1 -- 172.21.15.196:0/2307711541 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f217c079b80 con 0x7f21700039b0 2024-04-28T13:31:31.520 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.519+0000 7f2191800640 1 -- 172.21.15.196:0/2307711541 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f218c06fd90 con 0x7f216c06c8f0 2024-04-28T13:31:31.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.538+0000 7f2182a00640 1 -- 172.21.15.196:0/2307711541 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f218c06fd90 con 0x7f216c06c8f0 2024-04-28T13:31:31.543 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:31.544 INFO:teuthology.orchestra.run.smithi196.stderr:dumped all 2024-04-28T13:31:31.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.543+0000 7f2191800640 1 -- 172.21.15.196:0/2307711541 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f216c06c8f0 msgr2=0x7f216c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.543+0000 7f2191800640 1 --2- 172.21.15.196:0/2307711541 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f216c06c8f0 0x7f216c06edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f21780017e0 tx=0x7f2178009040 comp rx=0 tx=0).stop 2024-04-28T13:31:31.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.543+0000 7f2191800640 1 -- 172.21.15.196:0/2307711541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21700039b0 msgr2=0x7f218c0ae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.543+0000 7f2191800640 1 --2- 172.21.15.196:0/2307711541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21700039b0 0x7f218c0ae6d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f217c005040 tx=0x7f217c001120 comp rx=0 tx=0).stop 2024-04-28T13:31:31.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.543+0000 7f2191800640 1 -- 172.21.15.196:0/2307711541 shutdown_connections 2024-04-28T13:31:31.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.543+0000 7f2191800640 1 --2- 172.21.15.196:0/2307711541 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f216c06c8f0 0x7f216c06edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.543+0000 7f2191800640 1 --2- 172.21.15.196:0/2307711541 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f218c0ac470 0x7f218c0ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.543+0000 7f2191800640 1 --2- 172.21.15.196:0/2307711541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f218c15cd80 0x7f218c13c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.543+0000 7f2191800640 1 --2- 172.21.15.196:0/2307711541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21700039b0 0x7f218c0ae6d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.543+0000 7f2191800640 1 -- 172.21.15.196:0/2307711541 >> 172.21.15.196:0/2307711541 conn(0x7f218c0cd830 msgr2=0x7f218c0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:31.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.543+0000 7f2191800640 1 -- 172.21.15.196:0/2307711541 shutdown_connections 2024-04-28T13:31:31.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.543+0000 7f2191800640 1 -- 172.21.15.196:0/2307711541 wait complete. 2024-04-28T13:31:31.561 INFO:teuthology.orchestra.run.smithi196.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-28T13:31:30.183131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.570523"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.232418+0000","last_change":"2024-04-28T13:31:21.653529+0000","last_active":"2024-04-28T13:31:24.232418+0000","last_peered":"2024-04-28T13:31:24.232418+0000","last_clean":"2024-04-28T13:31:24.232418+0000","last_became_active":"2024-04-28T13:31:21.653192+0000","last_became_peered":"2024-04-28T13:31:21.653192+0000","last_unstale":"2024-04-28T13:31:24.232418+0000","last_undegraded":"2024-04-28T13:31:24.232418+0000","last_fullsized":"2024-04-28T13:31:24.232418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:30:36.318507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640382+0000","last_change":"2024-04-28T13:31:21.636383+0000","last_active":"2024-04-28T13:31:22.640382+0000","last_peered":"2024-04-28T13:31:22.640382+0000","last_clean":"2024-04-28T13:31:22.640382+0000","last_became_active":"2024-04-28T13:31:21.635970+0000","last_became_peered":"2024-04-28T13:31:21.635970+0000","last_unstale":"2024-04-28T13:31:22.640382+0000","last_undegraded":"2024-04-28T13:31:22.640382+0000","last_fullsized":"2024-04-28T13:31:22.640382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:33.431077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.729998+0000","last_change":"2024-04-28T13:31:21.642396+0000","last_active":"2024-04-28T13:31:24.729998+0000","last_peered":"2024-04-28T13:31:24.729998+0000","last_clean":"2024-04-28T13:31:24.729998+0000","last_became_active":"2024-04-28T13:31:21.642062+0000","last_became_peered":"2024-04-28T13:31:21.642062+0000","last_unstale":"2024-04-28T13:31:24.729998+0000","last_undegraded":"2024-04-28T13:31:24.729998+0000","last_fullsized":"2024-04-28T13:31:24.729998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:43:30.133775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646626+0000","last_change":"2024-04-28T13:31:21.637265+0000","last_active":"2024-04-28T13:31:22.646626+0000","last_peered":"2024-04-28T13:31:22.646626+0000","last_clean":"2024-04-28T13:31:22.646626+0000","last_became_active":"2024-04-28T13:31:21.636679+0000","last_became_peered":"2024-04-28T13:31:21.636679+0000","last_unstale":"2024-04-28T13:31:22.646626+0000","last_undegraded":"2024-04-28T13:31:22.646626+0000","last_fullsized":"2024-04-28T13:31:22.646626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:25.059779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231103+0000","last_change":"2024-04-28T13:31:21.660088+0000","last_active":"2024-04-28T13:31:24.231103+0000","last_peered":"2024-04-28T13:31:24.231103+0000","last_clean":"2024-04-28T13:31:24.231103+0000","last_became_active":"2024-04-28T13:31:21.652264+0000","last_became_peered":"2024-04-28T13:31:21.652264+0000","last_unstale":"2024-04-28T13:31:24.231103+0000","last_undegraded":"2024-04-28T13:31:24.231103+0000","last_fullsized":"2024-04-28T13:31:24.231103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:28:41.694303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646580+0000","last_change":"2024-04-28T13:31:21.641292+0000","last_active":"2024-04-28T13:31:22.646580+0000","last_peered":"2024-04-28T13:31:22.646580+0000","last_clean":"2024-04-28T13:31:22.646580+0000","last_became_active":"2024-04-28T13:31:21.635397+0000","last_became_peered":"2024-04-28T13:31:21.635397+0000","last_unstale":"2024-04-28T13:31:22.646580+0000","last_undegraded":"2024-04-28T13:31:22.646580+0000","last_fullsized":"2024-04-28T13:31:22.646580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:16:56.574292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231153+0000","last_change":"2024-04-28T13:31:21.654211+0000","last_active":"2024-04-28T13:31:24.231153+0000","last_peered":"2024-04-28T13:31:24.231153+0000","last_clean":"2024-04-28T13:31:24.231153+0000","last_became_active":"2024-04-28T13:31:21.653516+0000","last_became_peered":"2024-04-28T13:31:21.653516+0000","last_unstale":"2024-04-28T13:31:24.231153+0000","last_undegraded":"2024-04-28T13:31:24.231153+0000","last_fullsized":"2024-04-28T13:31:24.231153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:26:26.344814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.729792+0000","last_change":"2024-04-28T13:31:21.641706+0000","last_active":"2024-04-28T13:31:24.729792+0000","last_peered":"2024-04-28T13:31:24.729792+0000","last_clean":"2024-04-28T13:31:24.729792+0000","last_became_active":"2024-04-28T13:31:21.639207+0000","last_became_peered":"2024-04-28T13:31:21.639207+0000","last_unstale":"2024-04-28T13:31:24.729792+0000","last_undegraded":"2024-04-28T13:31:24.729792+0000","last_fullsized":"2024-04-28T13:31:24.729792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:33:40.307112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.232568+0000","last_change":"2024-04-28T13:31:21.657597+0000","last_active":"2024-04-28T13:31:24.232568+0000","last_peered":"2024-04-28T13:31:24.232568+0000","last_clean":"2024-04-28T13:31:24.232568+0000","last_became_active":"2024-04-28T13:31:21.656871+0000","last_became_peered":"2024-04-28T13:31:21.656871+0000","last_unstale":"2024-04-28T13:31:24.232568+0000","last_undegraded":"2024-04-28T13:31:24.232568+0000","last_fullsized":"2024-04-28T13:31:24.232568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:23:11.970214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732465+0000","last_change":"2024-04-28T13:31:21.648874+0000","last_active":"2024-04-28T13:31:24.732465+0000","last_peered":"2024-04-28T13:31:24.732465+0000","last_clean":"2024-04-28T13:31:24.732465+0000","last_became_active":"2024-04-28T13:31:21.646033+0000","last_became_peered":"2024-04-28T13:31:21.646033+0000","last_unstale":"2024-04-28T13:31:24.732465+0000","last_undegraded":"2024-04-28T13:31:24.732465+0000","last_fullsized":"2024-04-28T13:31:24.732465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:33.431077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732151+0000","last_change":"2024-04-28T13:31:21.649391+0000","last_active":"2024-04-28T13:31:24.732151+0000","last_peered":"2024-04-28T13:31:24.732151+0000","last_clean":"2024-04-28T13:31:24.732151+0000","last_became_active":"2024-04-28T13:31:21.649078+0000","last_became_peered":"2024-04-28T13:31:21.649078+0000","last_unstale":"2024-04-28T13:31:24.732151+0000","last_undegraded":"2024-04-28T13:31:24.732151+0000","last_fullsized":"2024-04-28T13:31:24.732151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:32.352858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647834+0000","last_change":"2024-04-28T13:31:21.651903+0000","last_active":"2024-04-28T13:31:22.647834+0000","last_peered":"2024-04-28T13:31:22.647834+0000","last_clean":"2024-04-28T13:31:22.647834+0000","last_became_active":"2024-04-28T13:31:21.650905+0000","last_became_peered":"2024-04-28T13:31:21.650905+0000","last_unstale":"2024-04-28T13:31:22.647834+0000","last_undegraded":"2024-04-28T13:31:22.647834+0000","last_fullsized":"2024-04-28T13:31:22.647834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:01.137759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731025+0000","last_change":"2024-04-28T13:31:21.640551+0000","last_active":"2024-04-28T13:31:24.731025+0000","last_peered":"2024-04-28T13:31:24.731025+0000","last_clean":"2024-04-28T13:31:24.731025+0000","last_became_active":"2024-04-28T13:31:21.640275+0000","last_became_peered":"2024-04-28T13:31:21.640275+0000","last_unstale":"2024-04-28T13:31:24.731025+0000","last_undegraded":"2024-04-28T13:31:24.731025+0000","last_fullsized":"2024-04-28T13:31:24.731025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:43:16.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730356+0000","last_change":"2024-04-28T13:31:21.639830+0000","last_active":"2024-04-28T13:31:24.730356+0000","last_peered":"2024-04-28T13:31:24.730356+0000","last_clean":"2024-04-28T13:31:24.730356+0000","last_became_active":"2024-04-28T13:31:21.639502+0000","last_became_peered":"2024-04-28T13:31:21.639502+0000","last_unstale":"2024-04-28T13:31:24.730356+0000","last_undegraded":"2024-04-28T13:31:24.730356+0000","last_fullsized":"2024-04-28T13:31:24.730356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:30:04.305801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640325+0000","last_change":"2024-04-28T13:31:21.635217+0000","last_active":"2024-04-28T13:31:22.640325+0000","last_peered":"2024-04-28T13:31:22.640325+0000","last_clean":"2024-04-28T13:31:22.640325+0000","last_became_active":"2024-04-28T13:31:21.634165+0000","last_became_peered":"2024-04-28T13:31:21.634165+0000","last_unstale":"2024-04-28T13:31:22.640325+0000","last_undegraded":"2024-04-28T13:31:22.640325+0000","last_fullsized":"2024-04-28T13:31:22.640325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:32.352858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646029+0000","last_change":"2024-04-28T13:31:21.642988+0000","last_active":"2024-04-28T13:31:22.646029+0000","last_peered":"2024-04-28T13:31:22.646029+0000","last_clean":"2024-04-28T13:31:22.646029+0000","last_became_active":"2024-04-28T13:31:21.635586+0000","last_became_peered":"2024-04-28T13:31:21.635586+0000","last_unstale":"2024-04-28T13:31:22.646029+0000","last_undegraded":"2024-04-28T13:31:22.646029+0000","last_fullsized":"2024-04-28T13:31:22.646029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:33.431077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647955+0000","last_change":"2024-04-28T13:31:21.653816+0000","last_active":"2024-04-28T13:31:22.647955+0000","last_peered":"2024-04-28T13:31:22.647955+0000","last_clean":"2024-04-28T13:31:22.647955+0000","last_became_active":"2024-04-28T13:31:21.653606+0000","last_became_peered":"2024-04-28T13:31:21.653606+0000","last_unstale":"2024-04-28T13:31:22.647955+0000","last_undegraded":"2024-04-28T13:31:22.647955+0000","last_fullsized":"2024-04-28T13:31:22.647955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:53:05.667773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731713+0000","last_change":"2024-04-28T13:31:21.643582+0000","last_active":"2024-04-28T13:31:24.731713+0000","last_peered":"2024-04-28T13:31:24.731713+0000","last_clean":"2024-04-28T13:31:24.731713+0000","last_became_active":"2024-04-28T13:31:21.643318+0000","last_became_peered":"2024-04-28T13:31:21.643318+0000","last_unstale":"2024-04-28T13:31:24.731713+0000","last_undegraded":"2024-04-28T13:31:24.731713+0000","last_fullsized":"2024-04-28T13:31:24.731713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:45:35.324680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231673+0000","last_change":"2024-04-28T13:31:21.659912+0000","last_active":"2024-04-28T13:31:24.231673+0000","last_peered":"2024-04-28T13:31:24.231673+0000","last_clean":"2024-04-28T13:31:24.231673+0000","last_became_active":"2024-04-28T13:31:21.659577+0000","last_became_peered":"2024-04-28T13:31:21.659577+0000","last_unstale":"2024-04-28T13:31:24.231673+0000","last_undegraded":"2024-04-28T13:31:24.231673+0000","last_fullsized":"2024-04-28T13:31:24.231673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:04:16.728668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731562+0000","last_change":"2024-04-28T13:31:21.654570+0000","last_active":"2024-04-28T13:31:24.731562+0000","last_peered":"2024-04-28T13:31:24.731562+0000","last_clean":"2024-04-28T13:31:24.731562+0000","last_became_active":"2024-04-28T13:31:21.654238+0000","last_became_peered":"2024-04-28T13:31:21.654238+0000","last_unstale":"2024-04-28T13:31:24.731562+0000","last_undegraded":"2024-04-28T13:31:24.731562+0000","last_fullsized":"2024-04-28T13:31:24.731562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:33:40.307112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730818+0000","last_change":"2024-04-28T13:31:21.653783+0000","last_active":"2024-04-28T13:31:24.730818+0000","last_peered":"2024-04-28T13:31:24.730818+0000","last_clean":"2024-04-28T13:31:24.730818+0000","last_became_active":"2024-04-28T13:31:21.653493+0000","last_became_peered":"2024-04-28T13:31:21.653493+0000","last_unstale":"2024-04-28T13:31:24.730818+0000","last_undegraded":"2024-04-28T13:31:24.730818+0000","last_fullsized":"2024-04-28T13:31:24.730818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:52:32.051589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647048+0000","last_change":"2024-04-28T13:31:21.660686+0000","last_active":"2024-04-28T13:31:22.647048+0000","last_peered":"2024-04-28T13:31:22.647048+0000","last_clean":"2024-04-28T13:31:22.647048+0000","last_became_active":"2024-04-28T13:31:21.660320+0000","last_became_peered":"2024-04-28T13:31:21.660320+0000","last_unstale":"2024-04-28T13:31:22.647048+0000","last_undegraded":"2024-04-28T13:31:22.647048+0000","last_fullsized":"2024-04-28T13:31:22.647048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:09:03.049024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730352+0000","last_change":"2024-04-28T13:31:21.640131+0000","last_active":"2024-04-28T13:31:24.730352+0000","last_peered":"2024-04-28T13:31:24.730352+0000","last_clean":"2024-04-28T13:31:24.730352+0000","last_became_active":"2024-04-28T13:31:21.639821+0000","last_became_peered":"2024-04-28T13:31:21.639821+0000","last_unstale":"2024-04-28T13:31:24.730352+0000","last_undegraded":"2024-04-28T13:31:24.730352+0000","last_fullsized":"2024-04-28T13:31:24.730352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:39:23.345225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646449+0000","last_change":"2024-04-28T13:31:21.648922+0000","last_active":"2024-04-28T13:31:22.646449+0000","last_peered":"2024-04-28T13:31:22.646449+0000","last_clean":"2024-04-28T13:31:22.646449+0000","last_became_active":"2024-04-28T13:31:21.639876+0000","last_became_peered":"2024-04-28T13:31:21.639876+0000","last_unstale":"2024-04-28T13:31:22.646449+0000","last_undegraded":"2024-04-28T13:31:22.646449+0000","last_fullsized":"2024-04-28T13:31:22.646449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:43:16.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732198+0000","last_change":"2024-04-28T13:31:21.644835+0000","last_active":"2024-04-28T13:31:24.732198+0000","last_peered":"2024-04-28T13:31:24.732198+0000","last_clean":"2024-04-28T13:31:24.732198+0000","last_became_active":"2024-04-28T13:31:21.644564+0000","last_became_peered":"2024-04-28T13:31:21.644564+0000","last_unstale":"2024-04-28T13:31:24.732198+0000","last_undegraded":"2024-04-28T13:31:24.732198+0000","last_fullsized":"2024-04-28T13:31:24.732198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:09:03.049024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732493+0000","last_change":"2024-04-28T13:31:21.653502+0000","last_active":"2024-04-28T13:31:24.732493+0000","last_peered":"2024-04-28T13:31:24.732493+0000","last_clean":"2024-04-28T13:31:24.732493+0000","last_became_active":"2024-04-28T13:31:21.653235+0000","last_became_peered":"2024-04-28T13:31:21.653235+0000","last_unstale":"2024-04-28T13:31:24.732493+0000","last_undegraded":"2024-04-28T13:31:24.732493+0000","last_fullsized":"2024-04-28T13:31:24.732493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:43:16.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640377+0000","last_change":"2024-04-28T13:31:21.649404+0000","last_active":"2024-04-28T13:31:22.640377+0000","last_peered":"2024-04-28T13:31:22.640377+0000","last_clean":"2024-04-28T13:31:22.640377+0000","last_became_active":"2024-04-28T13:31:21.649097+0000","last_became_peered":"2024-04-28T13:31:21.649097+0000","last_unstale":"2024-04-28T13:31:22.640377+0000","last_undegraded":"2024-04-28T13:31:22.640377+0000","last_fullsized":"2024-04-28T13:31:22.640377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:43:16.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731939+0000","last_change":"2024-04-28T13:31:21.649378+0000","last_active":"2024-04-28T13:31:24.731939+0000","last_peered":"2024-04-28T13:31:24.731939+0000","last_clean":"2024-04-28T13:31:24.731939+0000","last_became_active":"2024-04-28T13:31:21.647397+0000","last_became_peered":"2024-04-28T13:31:21.647397+0000","last_unstale":"2024-04-28T13:31:24.731939+0000","last_undegraded":"2024-04-28T13:31:24.731939+0000","last_fullsized":"2024-04-28T13:31:24.731939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:25.059779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730846+0000","last_change":"2024-04-28T13:31:21.642005+0000","last_active":"2024-04-28T13:31:24.730846+0000","last_peered":"2024-04-28T13:31:24.730846+0000","last_clean":"2024-04-28T13:31:24.730846+0000","last_became_active":"2024-04-28T13:31:21.641662+0000","last_became_peered":"2024-04-28T13:31:21.641662+0000","last_unstale":"2024-04-28T13:31:24.730846+0000","last_undegraded":"2024-04-28T13:31:24.730846+0000","last_fullsized":"2024-04-28T13:31:24.730846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:32.352858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647442+0000","last_change":"2024-04-28T13:31:21.642673+0000","last_active":"2024-04-28T13:31:22.647442+0000","last_peered":"2024-04-28T13:31:22.647442+0000","last_clean":"2024-04-28T13:31:22.647442+0000","last_became_active":"2024-04-28T13:31:21.640799+0000","last_became_peered":"2024-04-28T13:31:21.640799+0000","last_unstale":"2024-04-28T13:31:22.647442+0000","last_undegraded":"2024-04-28T13:31:22.647442+0000","last_fullsized":"2024-04-28T13:31:22.647442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:04:16.728668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730785+0000","last_change":"2024-04-28T13:31:21.657008+0000","last_active":"2024-04-28T13:31:24.730785+0000","last_peered":"2024-04-28T13:31:24.730785+0000","last_clean":"2024-04-28T13:31:24.730785+0000","last_became_active":"2024-04-28T13:31:21.656390+0000","last_became_peered":"2024-04-28T13:31:21.656390+0000","last_unstale":"2024-04-28T13:31:24.730785+0000","last_undegraded":"2024-04-28T13:31:24.730785+0000","last_fullsized":"2024-04-28T13:31:24.730785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:01.137759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731902+0000","last_change":"2024-04-28T13:31:21.643888+0000","last_active":"2024-04-28T13:31:24.731902+0000","last_peered":"2024-04-28T13:31:24.731902+0000","last_clean":"2024-04-28T13:31:24.731902+0000","last_became_active":"2024-04-28T13:31:21.643153+0000","last_became_peered":"2024-04-28T13:31:21.643153+0000","last_unstale":"2024-04-28T13:31:24.731902+0000","last_undegraded":"2024-04-28T13:31:24.731902+0000","last_fullsized":"2024-04-28T13:31:24.731902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:52:32.051589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.645551+0000","last_change":"2024-04-28T13:31:21.640935+0000","last_active":"2024-04-28T13:31:22.645551+0000","last_peered":"2024-04-28T13:31:22.645551+0000","last_clean":"2024-04-28T13:31:22.645551+0000","last_became_active":"2024-04-28T13:31:21.640633+0000","last_became_peered":"2024-04-28T13:31:21.640633+0000","last_unstale":"2024-04-28T13:31:22.645551+0000","last_undegraded":"2024-04-28T13:31:22.645551+0000","last_fullsized":"2024-04-28T13:31:22.645551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:23.207789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731261+0000","last_change":"2024-04-28T13:31:20.624370+0000","last_active":"2024-04-28T13:31:24.731261+0000","last_peered":"2024-04-28T13:31:24.731261+0000","last_clean":"2024-04-28T13:31:24.731261+0000","last_became_active":"2024-04-28T13:31:20.624091+0000","last_became_peered":"2024-04-28T13:31:20.624091+0000","last_unstale":"2024-04-28T13:31:24.731261+0000","last_undegraded":"2024-04-28T13:31:24.731261+0000","last_fullsized":"2024-04-28T13:31:24.731261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:35.060920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731062+0000","last_change":"2024-04-28T13:31:21.626821+0000","last_active":"2024-04-28T13:31:24.731062+0000","last_peered":"2024-04-28T13:31:24.731062+0000","last_clean":"2024-04-28T13:31:24.731062+0000","last_became_active":"2024-04-28T13:31:21.625590+0000","last_became_peered":"2024-04-28T13:31:21.625590+0000","last_unstale":"2024-04-28T13:31:24.731062+0000","last_undegraded":"2024-04-28T13:31:24.731062+0000","last_fullsized":"2024-04-28T13:31:24.731062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:33.431077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731325+0000","last_change":"2024-04-28T13:31:20.629009+0000","last_active":"2024-04-28T13:31:24.731325+0000","last_peered":"2024-04-28T13:31:24.731325+0000","last_clean":"2024-04-28T13:31:24.731325+0000","last_became_active":"2024-04-28T13:31:20.627394+0000","last_became_peered":"2024-04-28T13:31:20.627394+0000","last_unstale":"2024-04-28T13:31:24.731325+0000","last_undegraded":"2024-04-28T13:31:24.731325+0000","last_fullsized":"2024-04-28T13:31:24.731325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:01.229713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730417+0000","last_change":"2024-04-28T13:31:20.624345+0000","last_active":"2024-04-28T13:31:24.730417+0000","last_peered":"2024-04-28T13:31:24.730417+0000","last_clean":"2024-04-28T13:31:24.730417+0000","last_became_active":"2024-04-28T13:31:20.624052+0000","last_became_peered":"2024-04-28T13:31:20.624052+0000","last_unstale":"2024-04-28T13:31:24.730417+0000","last_undegraded":"2024-04-28T13:31:24.730417+0000","last_fullsized":"2024-04-28T13:31:24.730417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:46.616532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.729933+0000","last_change":"2024-04-28T13:31:21.637599+0000","last_active":"2024-04-28T13:31:24.729933+0000","last_peered":"2024-04-28T13:31:24.729933+0000","last_clean":"2024-04-28T13:31:24.729933+0000","last_became_active":"2024-04-28T13:31:21.637366+0000","last_became_peered":"2024-04-28T13:31:21.637366+0000","last_unstale":"2024-04-28T13:31:24.729933+0000","last_undegraded":"2024-04-28T13:31:24.729933+0000","last_fullsized":"2024-04-28T13:31:24.729933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:14:19.057730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646642+0000","last_change":"2024-04-28T13:31:20.625321+0000","last_active":"2024-04-28T13:31:22.646642+0000","last_peered":"2024-04-28T13:31:22.646642+0000","last_clean":"2024-04-28T13:31:22.646642+0000","last_became_active":"2024-04-28T13:31:20.624034+0000","last_became_peered":"2024-04-28T13:31:20.624034+0000","last_unstale":"2024-04-28T13:31:22.646642+0000","last_undegraded":"2024-04-28T13:31:22.646642+0000","last_fullsized":"2024-04-28T13:31:22.646642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:22.193491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.639404+0000","last_change":"2024-04-28T13:31:21.650660+0000","last_active":"2024-04-28T13:31:22.639404+0000","last_peered":"2024-04-28T13:31:22.639404+0000","last_clean":"2024-04-28T13:31:22.639404+0000","last_became_active":"2024-04-28T13:31:21.650349+0000","last_became_peered":"2024-04-28T13:31:21.650349+0000","last_unstale":"2024-04-28T13:31:22.639404+0000","last_undegraded":"2024-04-28T13:31:22.639404+0000","last_fullsized":"2024-04-28T13:31:22.639404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:14:19.057730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732233+0000","last_change":"2024-04-28T13:31:20.632738+0000","last_active":"2024-04-28T13:31:24.732233+0000","last_peered":"2024-04-28T13:31:24.732233+0000","last_clean":"2024-04-28T13:31:24.732233+0000","last_became_active":"2024-04-28T13:31:20.631174+0000","last_became_peered":"2024-04-28T13:31:20.631174+0000","last_unstale":"2024-04-28T13:31:24.732233+0000","last_undegraded":"2024-04-28T13:31:24.732233+0000","last_fullsized":"2024-04-28T13:31:24.732233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:50.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647343+0000","last_change":"2024-04-28T13:31:21.639067+0000","last_active":"2024-04-28T13:31:22.647343+0000","last_peered":"2024-04-28T13:31:22.647343+0000","last_clean":"2024-04-28T13:31:22.647343+0000","last_became_active":"2024-04-28T13:31:21.638664+0000","last_became_peered":"2024-04-28T13:31:21.638664+0000","last_unstale":"2024-04-28T13:31:22.647343+0000","last_undegraded":"2024-04-28T13:31:22.647343+0000","last_fullsized":"2024-04-28T13:31:22.647343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:44:09.266587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647483+0000","last_change":"2024-04-28T13:31:20.631748+0000","last_active":"2024-04-28T13:31:22.647483+0000","last_peered":"2024-04-28T13:31:22.647483+0000","last_clean":"2024-04-28T13:31:22.647483+0000","last_became_active":"2024-04-28T13:31:20.631345+0000","last_became_peered":"2024-04-28T13:31:20.631345+0000","last_unstale":"2024-04-28T13:31:22.647483+0000","last_undegraded":"2024-04-28T13:31:22.647483+0000","last_fullsized":"2024-04-28T13:31:22.647483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:09:16.028560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730844+0000","last_change":"2024-04-28T13:31:21.636554+0000","last_active":"2024-04-28T13:31:24.730844+0000","last_peered":"2024-04-28T13:31:24.730844+0000","last_clean":"2024-04-28T13:31:24.730844+0000","last_became_active":"2024-04-28T13:31:21.636062+0000","last_became_peered":"2024-04-28T13:31:21.636062+0000","last_unstale":"2024-04-28T13:31:24.730844+0000","last_undegraded":"2024-04-28T13:31:24.730844+0000","last_fullsized":"2024-04-28T13:31:24.730844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:39:23.345225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640458+0000","last_change":"2024-04-28T13:31:20.627856+0000","last_active":"2024-04-28T13:31:22.640458+0000","last_peered":"2024-04-28T13:31:22.640458+0000","last_clean":"2024-04-28T13:31:22.640458+0000","last_became_active":"2024-04-28T13:31:20.627632+0000","last_became_peered":"2024-04-28T13:31:20.627632+0000","last_unstale":"2024-04-28T13:31:22.640458+0000","last_undegraded":"2024-04-28T13:31:22.640458+0000","last_fullsized":"2024-04-28T13:31:22.640458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:09:16.028560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.648191+0000","last_change":"2024-04-28T13:31:21.657348+0000","last_active":"2024-04-28T13:31:22.648191+0000","last_peered":"2024-04-28T13:31:22.648191+0000","last_clean":"2024-04-28T13:31:22.648191+0000","last_became_active":"2024-04-28T13:31:21.657055+0000","last_became_peered":"2024-04-28T13:31:21.657055+0000","last_unstale":"2024-04-28T13:31:22.648191+0000","last_undegraded":"2024-04-28T13:31:22.648191+0000","last_fullsized":"2024-04-28T13:31:22.648191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:45:35.324680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731568+0000","last_change":"2024-04-28T13:31:20.629687+0000","last_active":"2024-04-28T13:31:24.731568+0000","last_peered":"2024-04-28T13:31:24.731568+0000","last_clean":"2024-04-28T13:31:24.731568+0000","last_became_active":"2024-04-28T13:31:20.628070+0000","last_became_peered":"2024-04-28T13:31:20.628070+0000","last_unstale":"2024-04-28T13:31:24.731568+0000","last_undegraded":"2024-04-28T13:31:24.731568+0000","last_fullsized":"2024-04-28T13:31:24.731568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:09:16.028560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231080+0000","last_change":"2024-04-28T13:31:21.659218+0000","last_active":"2024-04-28T13:31:24.231080+0000","last_peered":"2024-04-28T13:31:24.231080+0000","last_clean":"2024-04-28T13:31:24.231080+0000","last_became_active":"2024-04-28T13:31:21.658927+0000","last_became_peered":"2024-04-28T13:31:21.658927+0000","last_unstale":"2024-04-28T13:31:24.231080+0000","last_undegraded":"2024-04-28T13:31:24.231080+0000","last_fullsized":"2024-04-28T13:31:24.231080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:09:03.049024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730927+0000","last_change":"2024-04-28T13:31:20.624425+0000","last_active":"2024-04-28T13:31:24.730927+0000","last_peered":"2024-04-28T13:31:24.730927+0000","last_clean":"2024-04-28T13:31:24.730927+0000","last_became_active":"2024-04-28T13:31:20.623737+0000","last_became_peered":"2024-04-28T13:31:20.623737+0000","last_unstale":"2024-04-28T13:31:24.730927+0000","last_undegraded":"2024-04-28T13:31:24.730927+0000","last_fullsized":"2024-04-28T13:31:24.730927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:09:09.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731087+0000","last_change":"2024-04-28T13:31:21.658397+0000","last_active":"2024-04-28T13:31:24.731087+0000","last_peered":"2024-04-28T13:31:24.731087+0000","last_clean":"2024-04-28T13:31:24.731087+0000","last_became_active":"2024-04-28T13:31:21.658082+0000","last_became_peered":"2024-04-28T13:31:21.658082+0000","last_unstale":"2024-04-28T13:31:24.731087+0000","last_undegraded":"2024-04-28T13:31:24.731087+0000","last_fullsized":"2024-04-28T13:31:24.731087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:43:16.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647574+0000","last_change":"2024-04-28T13:31:20.625371+0000","last_active":"2024-04-28T13:31:22.647574+0000","last_peered":"2024-04-28T13:31:22.647574+0000","last_clean":"2024-04-28T13:31:22.647574+0000","last_became_active":"2024-04-28T13:31:20.625028+0000","last_became_peered":"2024-04-28T13:31:20.625028+0000","last_unstale":"2024-04-28T13:31:22.647574+0000","last_undegraded":"2024-04-28T13:31:22.647574+0000","last_fullsized":"2024-04-28T13:31:22.647574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:01.229713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231541+0000","last_change":"2024-04-28T13:31:21.660871+0000","last_active":"2024-04-28T13:31:24.231541+0000","last_peered":"2024-04-28T13:31:24.231541+0000","last_clean":"2024-04-28T13:31:24.231541+0000","last_became_active":"2024-04-28T13:31:21.660649+0000","last_became_peered":"2024-04-28T13:31:21.660649+0000","last_unstale":"2024-04-28T13:31:24.231541+0000","last_undegraded":"2024-04-28T13:31:24.231541+0000","last_fullsized":"2024-04-28T13:31:24.231541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:44:09.266587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731265+0000","last_change":"2024-04-28T13:31:20.625903+0000","last_active":"2024-04-28T13:31:24.731265+0000","last_peered":"2024-04-28T13:31:24.731265+0000","last_clean":"2024-04-28T13:31:24.731265+0000","last_became_active":"2024-04-28T13:31:20.625350+0000","last_became_peered":"2024-04-28T13:31:20.625350+0000","last_unstale":"2024-04-28T13:31:24.731265+0000","last_undegraded":"2024-04-28T13:31:24.731265+0000","last_fullsized":"2024-04-28T13:31:24.731265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:35.060920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731578+0000","last_change":"2024-04-28T13:31:21.643517+0000","last_active":"2024-04-28T13:31:24.731578+0000","last_peered":"2024-04-28T13:31:24.731578+0000","last_clean":"2024-04-28T13:31:24.731578+0000","last_became_active":"2024-04-28T13:31:21.643128+0000","last_became_peered":"2024-04-28T13:31:21.643128+0000","last_unstale":"2024-04-28T13:31:24.731578+0000","last_undegraded":"2024-04-28T13:31:24.731578+0000","last_fullsized":"2024-04-28T13:31:24.731578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:14:19.057730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.729987+0000","last_change":"2024-04-28T13:31:20.609440+0000","last_active":"2024-04-28T13:31:24.729987+0000","last_peered":"2024-04-28T13:31:24.729987+0000","last_clean":"2024-04-28T13:31:24.729987+0000","last_became_active":"2024-04-28T13:31:20.609120+0000","last_became_peered":"2024-04-28T13:31:20.609120+0000","last_unstale":"2024-04-28T13:31:24.729987+0000","last_undegraded":"2024-04-28T13:31:24.729987+0000","last_fullsized":"2024-04-28T13:31:24.729987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:01.229713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.230626+0000","last_change":"2024-04-28T13:31:21.654431+0000","last_active":"2024-04-28T13:31:24.230626+0000","last_peered":"2024-04-28T13:31:24.230626+0000","last_clean":"2024-04-28T13:31:24.230626+0000","last_became_active":"2024-04-28T13:31:21.653884+0000","last_became_peered":"2024-04-28T13:31:21.653884+0000","last_unstale":"2024-04-28T13:31:24.230626+0000","last_undegraded":"2024-04-28T13:31:24.230626+0000","last_fullsized":"2024-04-28T13:31:24.230626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:41:07.707990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646224+0000","last_change":"2024-04-28T13:31:20.619411+0000","last_active":"2024-04-28T13:31:22.646224+0000","last_peered":"2024-04-28T13:31:22.646224+0000","last_clean":"2024-04-28T13:31:22.646224+0000","last_became_active":"2024-04-28T13:31:20.618632+0000","last_became_peered":"2024-04-28T13:31:20.618632+0000","last_unstale":"2024-04-28T13:31:22.646224+0000","last_undegraded":"2024-04-28T13:31:22.646224+0000","last_fullsized":"2024-04-28T13:31:22.646224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:09:09.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.232213+0000","last_change":"2024-04-28T13:31:21.656026+0000","last_active":"2024-04-28T13:31:24.232213+0000","last_peered":"2024-04-28T13:31:24.232213+0000","last_clean":"2024-04-28T13:31:24.232213+0000","last_became_active":"2024-04-28T13:31:21.655737+0000","last_became_peered":"2024-04-28T13:31:21.655737+0000","last_unstale":"2024-04-28T13:31:24.232213+0000","last_undegraded":"2024-04-28T13:31:24.232213+0000","last_fullsized":"2024-04-28T13:31:24.232213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:53:05.667773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731528+0000","last_change":"2024-04-28T13:31:20.616200+0000","last_active":"2024-04-28T13:31:24.731528+0000","last_peered":"2024-04-28T13:31:24.731528+0000","last_clean":"2024-04-28T13:31:24.731528+0000","last_became_active":"2024-04-28T13:31:20.615813+0000","last_became_peered":"2024-04-28T13:31:20.615813+0000","last_unstale":"2024-04-28T13:31:24.731528+0000","last_undegraded":"2024-04-28T13:31:24.731528+0000","last_fullsized":"2024-04-28T13:31:24.731528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:35.060920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.232625+0000","last_change":"2024-04-28T13:31:21.657900+0000","last_active":"2024-04-28T13:31:24.232625+0000","last_peered":"2024-04-28T13:31:24.232625+0000","last_clean":"2024-04-28T13:31:24.232625+0000","last_became_active":"2024-04-28T13:31:21.657276+0000","last_became_peered":"2024-04-28T13:31:21.657276+0000","last_unstale":"2024-04-28T13:31:24.232625+0000","last_undegraded":"2024-04-28T13:31:24.232625+0000","last_fullsized":"2024-04-28T13:31:24.232625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:15:21.164101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640518+0000","last_change":"2024-04-28T13:31:20.620725+0000","last_active":"2024-04-28T13:31:22.640518+0000","last_peered":"2024-04-28T13:31:22.640518+0000","last_clean":"2024-04-28T13:31:22.640518+0000","last_became_active":"2024-04-28T13:31:20.620392+0000","last_became_peered":"2024-04-28T13:31:20.620392+0000","last_unstale":"2024-04-28T13:31:22.640518+0000","last_undegraded":"2024-04-28T13:31:22.640518+0000","last_fullsized":"2024-04-28T13:31:22.640518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:50.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647204+0000","last_change":"2024-04-28T13:31:21.640722+0000","last_active":"2024-04-28T13:31:22.647204+0000","last_peered":"2024-04-28T13:31:22.647204+0000","last_clean":"2024-04-28T13:31:22.647204+0000","last_became_active":"2024-04-28T13:31:21.637069+0000","last_became_peered":"2024-04-28T13:31:21.637069+0000","last_unstale":"2024-04-28T13:31:22.647204+0000","last_undegraded":"2024-04-28T13:31:22.647204+0000","last_fullsized":"2024-04-28T13:31:22.647204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:26:26.344814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731464+0000","last_change":"2024-04-28T13:31:20.623333+0000","last_active":"2024-04-28T13:31:24.731464+0000","last_peered":"2024-04-28T13:31:24.731464+0000","last_clean":"2024-04-28T13:31:24.731464+0000","last_became_active":"2024-04-28T13:31:20.623063+0000","last_became_peered":"2024-04-28T13:31:20.623063+0000","last_unstale":"2024-04-28T13:31:24.731464+0000","last_undegraded":"2024-04-28T13:31:24.731464+0000","last_fullsized":"2024-04-28T13:31:24.731464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:50.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231274+0000","last_change":"2024-04-28T13:31:21.663106+0000","last_active":"2024-04-28T13:31:24.231274+0000","last_peered":"2024-04-28T13:31:24.231274+0000","last_clean":"2024-04-28T13:31:24.231274+0000","last_became_active":"2024-04-28T13:31:21.662627+0000","last_became_peered":"2024-04-28T13:31:21.662627+0000","last_unstale":"2024-04-28T13:31:24.231274+0000","last_undegraded":"2024-04-28T13:31:24.231274+0000","last_fullsized":"2024-04-28T13:31:24.231274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:04:59.462014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.230624+0000","last_change":"2024-04-28T13:31:11.831538+0000","last_active":"2024-04-28T13:31:24.230624+0000","last_peered":"2024-04-28T13:31:24.230624+0000","last_clean":"2024-04-28T13:31:24.230624+0000","last_became_active":"2024-04-28T13:31:11.828959+0000","last_became_peered":"2024-04-28T13:31:11.828959+0000","last_unstale":"2024-04-28T13:31:24.230624+0000","last_undegraded":"2024-04-28T13:31:24.230624+0000","last_fullsized":"2024-04-28T13:31:24.230624+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_clean_scrub_stamp":"2024-04-28T13:31:10.444388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:25:27.078911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.645900+0000","last_change":"2024-04-28T13:31:20.619170+0000","last_active":"2024-04-28T13:31:22.645900+0000","last_peered":"2024-04-28T13:31:22.645900+0000","last_clean":"2024-04-28T13:31:22.645900+0000","last_became_active":"2024-04-28T13:31:20.618861+0000","last_became_peered":"2024-04-28T13:31:20.618861+0000","last_unstale":"2024-04-28T13:31:22.645900+0000","last_undegraded":"2024-04-28T13:31:22.645900+0000","last_fullsized":"2024-04-28T13:31:22.645900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:35.060920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730744+0000","last_change":"2024-04-28T13:31:21.649271+0000","last_active":"2024-04-28T13:31:24.730744+0000","last_peered":"2024-04-28T13:31:24.730744+0000","last_clean":"2024-04-28T13:31:24.730744+0000","last_became_active":"2024-04-28T13:31:21.649009+0000","last_became_peered":"2024-04-28T13:31:21.649009+0000","last_unstale":"2024-04-28T13:31:24.730744+0000","last_undegraded":"2024-04-28T13:31:24.730744+0000","last_fullsized":"2024-04-28T13:31:24.730744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:32.352858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.230284+0000","last_change":"2024-04-28T13:31:20.628745+0000","last_active":"2024-04-28T13:31:24.230284+0000","last_peered":"2024-04-28T13:31:24.230284+0000","last_clean":"2024-04-28T13:31:24.230284+0000","last_became_active":"2024-04-28T13:31:20.628430+0000","last_became_peered":"2024-04-28T13:31:20.628430+0000","last_unstale":"2024-04-28T13:31:24.230284+0000","last_undegraded":"2024-04-28T13:31:24.230284+0000","last_fullsized":"2024-04-28T13:31:24.230284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:09:16.028560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.729791+0000","last_change":"2024-04-28T13:31:21.628901+0000","last_active":"2024-04-28T13:31:24.729791+0000","last_peered":"2024-04-28T13:31:24.729791+0000","last_clean":"2024-04-28T13:31:24.729791+0000","last_became_active":"2024-04-28T13:31:21.628483+0000","last_became_peered":"2024-04-28T13:31:21.628483+0000","last_unstale":"2024-04-28T13:31:24.729791+0000","last_undegraded":"2024-04-28T13:31:24.729791+0000","last_fullsized":"2024-04-28T13:31:24.729791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:06:20.265833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731104+0000","last_change":"2024-04-28T13:31:20.615910+0000","last_active":"2024-04-28T13:31:24.731104+0000","last_peered":"2024-04-28T13:31:24.731104+0000","last_clean":"2024-04-28T13:31:24.731104+0000","last_became_active":"2024-04-28T13:31:20.615610+0000","last_became_peered":"2024-04-28T13:31:20.615610+0000","last_unstale":"2024-04-28T13:31:24.731104+0000","last_undegraded":"2024-04-28T13:31:24.731104+0000","last_fullsized":"2024-04-28T13:31:24.731104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:50:14.806138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.639480+0000","last_change":"2024-04-28T13:31:21.648562+0000","last_active":"2024-04-28T13:31:22.639480+0000","last_peered":"2024-04-28T13:31:22.639480+0000","last_clean":"2024-04-28T13:31:22.639480+0000","last_became_active":"2024-04-28T13:31:21.648282+0000","last_became_peered":"2024-04-28T13:31:21.648282+0000","last_unstale":"2024-04-28T13:31:22.639480+0000","last_undegraded":"2024-04-28T13:31:22.639480+0000","last_fullsized":"2024-04-28T13:31:22.639480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:52:32.051589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647763+0000","last_change":"2024-04-28T13:31:20.623861+0000","last_active":"2024-04-28T13:31:22.647763+0000","last_peered":"2024-04-28T13:31:22.647763+0000","last_clean":"2024-04-28T13:31:22.647763+0000","last_became_active":"2024-04-28T13:31:20.623459+0000","last_became_peered":"2024-04-28T13:31:20.623459+0000","last_unstale":"2024-04-28T13:31:22.647763+0000","last_undegraded":"2024-04-28T13:31:22.647763+0000","last_fullsized":"2024-04-28T13:31:22.647763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:50.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.645431+0000","last_change":"2024-04-28T13:31:21.634320+0000","last_active":"2024-04-28T13:31:22.645431+0000","last_peered":"2024-04-28T13:31:22.645431+0000","last_clean":"2024-04-28T13:31:22.645431+0000","last_became_active":"2024-04-28T13:31:21.634045+0000","last_became_peered":"2024-04-28T13:31:21.634045+0000","last_unstale":"2024-04-28T13:31:22.645431+0000","last_undegraded":"2024-04-28T13:31:22.645431+0000","last_fullsized":"2024-04-28T13:31:22.645431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:51.731818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.639846+0000","last_change":"2024-04-28T13:31:20.612653+0000","last_active":"2024-04-28T13:31:22.639846+0000","last_peered":"2024-04-28T13:31:22.639846+0000","last_clean":"2024-04-28T13:31:22.639846+0000","last_became_active":"2024-04-28T13:31:20.612341+0000","last_became_peered":"2024-04-28T13:31:20.612341+0000","last_unstale":"2024-04-28T13:31:22.639846+0000","last_undegraded":"2024-04-28T13:31:22.639846+0000","last_fullsized":"2024-04-28T13:31:22.639846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:45:34.310382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647331+0000","last_change":"2024-04-28T13:31:21.642073+0000","last_active":"2024-04-28T13:31:22.647331+0000","last_peered":"2024-04-28T13:31:22.647331+0000","last_clean":"2024-04-28T13:31:22.647331+0000","last_became_active":"2024-04-28T13:31:21.641771+0000","last_became_peered":"2024-04-28T13:31:21.641771+0000","last_unstale":"2024-04-28T13:31:22.647331+0000","last_undegraded":"2024-04-28T13:31:22.647331+0000","last_fullsized":"2024-04-28T13:31:22.647331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:04:59.462014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231572+0000","last_change":"2024-04-28T13:31:20.634223+0000","last_active":"2024-04-28T13:31:24.231572+0000","last_peered":"2024-04-28T13:31:24.231572+0000","last_clean":"2024-04-28T13:31:24.231572+0000","last_became_active":"2024-04-28T13:31:20.633834+0000","last_became_peered":"2024-04-28T13:31:20.633834+0000","last_unstale":"2024-04-28T13:31:24.231572+0000","last_undegraded":"2024-04-28T13:31:24.231572+0000","last_fullsized":"2024-04-28T13:31:24.231572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:37:34.377933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.639712+0000","last_change":"2024-04-28T13:31:21.643483+0000","last_active":"2024-04-28T13:31:22.639712+0000","last_peered":"2024-04-28T13:31:22.639712+0000","last_clean":"2024-04-28T13:31:22.639712+0000","last_became_active":"2024-04-28T13:31:21.643157+0000","last_became_peered":"2024-04-28T13:31:21.643157+0000","last_unstale":"2024-04-28T13:31:22.639712+0000","last_undegraded":"2024-04-28T13:31:22.639712+0000","last_fullsized":"2024-04-28T13:31:22.639712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:39:23.345225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640516+0000","last_change":"2024-04-28T13:31:20.630471+0000","last_active":"2024-04-28T13:31:22.640516+0000","last_peered":"2024-04-28T13:31:22.640516+0000","last_clean":"2024-04-28T13:31:22.640516+0000","last_became_active":"2024-04-28T13:31:20.630161+0000","last_became_peered":"2024-04-28T13:31:20.630161+0000","last_unstale":"2024-04-28T13:31:22.640516+0000","last_undegraded":"2024-04-28T13:31:22.640516+0000","last_fullsized":"2024-04-28T13:31:22.640516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:22.193491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732154+0000","last_change":"2024-04-28T13:31:21.648853+0000","last_active":"2024-04-28T13:31:24.732154+0000","last_peered":"2024-04-28T13:31:24.732154+0000","last_clean":"2024-04-28T13:31:24.732154+0000","last_became_active":"2024-04-28T13:31:21.647783+0000","last_became_peered":"2024-04-28T13:31:21.647783+0000","last_unstale":"2024-04-28T13:31:24.732154+0000","last_undegraded":"2024-04-28T13:31:24.732154+0000","last_fullsized":"2024-04-28T13:31:24.732154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:41:07.707990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732342+0000","last_change":"2024-04-28T13:31:20.629409+0000","last_active":"2024-04-28T13:31:24.732342+0000","last_peered":"2024-04-28T13:31:24.732342+0000","last_clean":"2024-04-28T13:31:24.732342+0000","last_became_active":"2024-04-28T13:31:20.628182+0000","last_became_peered":"2024-04-28T13:31:20.628182+0000","last_unstale":"2024-04-28T13:31:24.732342+0000","last_undegraded":"2024-04-28T13:31:24.732342+0000","last_fullsized":"2024-04-28T13:31:24.732342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:22.193491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730384+0000","last_change":"2024-04-28T13:31:21.640225+0000","last_active":"2024-04-28T13:31:24.730384+0000","last_peered":"2024-04-28T13:31:24.730384+0000","last_clean":"2024-04-28T13:31:24.730384+0000","last_became_active":"2024-04-28T13:31:21.639921+0000","last_became_peered":"2024-04-28T13:31:21.639921+0000","last_unstale":"2024-04-28T13:31:24.730384+0000","last_undegraded":"2024-04-28T13:31:24.730384+0000","last_fullsized":"2024-04-28T13:31:24.730384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:33:40.307112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.230944+0000","last_change":"2024-04-28T13:31:20.633572+0000","last_active":"2024-04-28T13:31:24.230944+0000","last_peered":"2024-04-28T13:31:24.230944+0000","last_clean":"2024-04-28T13:31:24.230944+0000","last_became_active":"2024-04-28T13:31:20.630758+0000","last_became_peered":"2024-04-28T13:31:20.630758+0000","last_unstale":"2024-04-28T13:31:24.230944+0000","last_undegraded":"2024-04-28T13:31:24.230944+0000","last_fullsized":"2024-04-28T13:31:24.230944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:50.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.645842+0000","last_change":"2024-04-28T13:31:21.637178+0000","last_active":"2024-04-28T13:31:22.645842+0000","last_peered":"2024-04-28T13:31:22.645842+0000","last_clean":"2024-04-28T13:31:22.645842+0000","last_became_active":"2024-04-28T13:31:21.636898+0000","last_became_peered":"2024-04-28T13:31:21.636898+0000","last_unstale":"2024-04-28T13:31:22.645842+0000","last_undegraded":"2024-04-28T13:31:22.645842+0000","last_fullsized":"2024-04-28T13:31:22.645842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:09:17.042858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.639609+0000","last_change":"2024-04-28T13:31:20.624716+0000","last_active":"2024-04-28T13:31:22.639609+0000","last_peered":"2024-04-28T13:31:22.639609+0000","last_clean":"2024-04-28T13:31:22.639609+0000","last_became_active":"2024-04-28T13:31:20.623554+0000","last_became_peered":"2024-04-28T13:31:20.623554+0000","last_unstale":"2024-04-28T13:31:22.639609+0000","last_undegraded":"2024-04-28T13:31:22.639609+0000","last_fullsized":"2024-04-28T13:31:22.639609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:01.229713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730144+0000","last_change":"2024-04-28T13:31:21.646597+0000","last_active":"2024-04-28T13:31:24.730144+0000","last_peered":"2024-04-28T13:31:24.730144+0000","last_clean":"2024-04-28T13:31:24.730144+0000","last_became_active":"2024-04-28T13:31:21.646307+0000","last_became_peered":"2024-04-28T13:31:21.646307+0000","last_unstale":"2024-04-28T13:31:24.730144+0000","last_undegraded":"2024-04-28T13:31:24.730144+0000","last_fullsized":"2024-04-28T13:31:24.730144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:52:32.051589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640004+0000","last_change":"2024-04-28T13:31:20.617782+0000","last_active":"2024-04-28T13:31:22.640004+0000","last_peered":"2024-04-28T13:31:22.640004+0000","last_clean":"2024-04-28T13:31:22.640004+0000","last_became_active":"2024-04-28T13:31:20.617213+0000","last_became_peered":"2024-04-28T13:31:20.617213+0000","last_unstale":"2024-04-28T13:31:22.640004+0000","last_undegraded":"2024-04-28T13:31:22.640004+0000","last_fullsized":"2024-04-28T13:31:22.640004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:00.123461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731198+0000","last_change":"2024-04-28T13:31:21.637666+0000","last_active":"2024-04-28T13:31:24.731198+0000","last_peered":"2024-04-28T13:31:24.731198+0000","last_clean":"2024-04-28T13:31:24.731198+0000","last_became_active":"2024-04-28T13:31:21.637359+0000","last_became_peered":"2024-04-28T13:31:21.637359+0000","last_unstale":"2024-04-28T13:31:24.731198+0000","last_undegraded":"2024-04-28T13:31:24.731198+0000","last_fullsized":"2024-04-28T13:31:24.731198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:16:56.574292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231671+0000","last_change":"2024-04-28T13:31:20.632480+0000","last_active":"2024-04-28T13:31:24.231671+0000","last_peered":"2024-04-28T13:31:24.231671+0000","last_clean":"2024-04-28T13:31:24.231671+0000","last_became_active":"2024-04-28T13:31:20.632177+0000","last_became_peered":"2024-04-28T13:31:20.632177+0000","last_unstale":"2024-04-28T13:31:24.231671+0000","last_undegraded":"2024-04-28T13:31:24.231671+0000","last_fullsized":"2024-04-28T13:31:24.231671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:22.193491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732048+0000","last_change":"2024-04-28T13:31:21.645354+0000","last_active":"2024-04-28T13:31:24.732048+0000","last_peered":"2024-04-28T13:31:24.732048+0000","last_clean":"2024-04-28T13:31:24.732048+0000","last_became_active":"2024-04-28T13:31:21.644879+0000","last_became_peered":"2024-04-28T13:31:21.644879+0000","last_unstale":"2024-04-28T13:31:24.732048+0000","last_undegraded":"2024-04-28T13:31:24.732048+0000","last_fullsized":"2024-04-28T13:31:24.732048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:39:23.345225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647195+0000","last_change":"2024-04-28T13:31:21.636961+0000","last_active":"2024-04-28T13:31:22.647195+0000","last_peered":"2024-04-28T13:31:22.647195+0000","last_clean":"2024-04-28T13:31:22.647195+0000","last_became_active":"2024-04-28T13:31:21.636609+0000","last_became_peered":"2024-04-28T13:31:21.636609+0000","last_unstale":"2024-04-28T13:31:22.647195+0000","last_undegraded":"2024-04-28T13:31:22.647195+0000","last_fullsized":"2024-04-28T13:31:22.647195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:28:41.694303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.230631+0000","last_change":"2024-04-28T13:31:20.632395+0000","last_active":"2024-04-28T13:31:24.230631+0000","last_peered":"2024-04-28T13:31:24.230631+0000","last_clean":"2024-04-28T13:31:24.230631+0000","last_became_active":"2024-04-28T13:31:20.632083+0000","last_became_peered":"2024-04-28T13:31:20.632083+0000","last_unstale":"2024-04-28T13:31:24.230631+0000","last_undegraded":"2024-04-28T13:31:24.230631+0000","last_fullsized":"2024-04-28T13:31:24.230631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:00.123461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646732+0000","last_change":"2024-04-28T13:31:21.637466+0000","last_active":"2024-04-28T13:31:22.646732+0000","last_peered":"2024-04-28T13:31:22.646732+0000","last_clean":"2024-04-28T13:31:22.646732+0000","last_became_active":"2024-04-28T13:31:21.636945+0000","last_became_peered":"2024-04-28T13:31:21.636945+0000","last_unstale":"2024-04-28T13:31:22.646732+0000","last_undegraded":"2024-04-28T13:31:22.646732+0000","last_fullsized":"2024-04-28T13:31:22.646732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:41:07.707990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231240+0000","last_change":"2024-04-28T13:31:20.636621+0000","last_active":"2024-04-28T13:31:24.231240+0000","last_peered":"2024-04-28T13:31:24.231240+0000","last_clean":"2024-04-28T13:31:24.231240+0000","last_became_active":"2024-04-28T13:31:20.636363+0000","last_became_peered":"2024-04-28T13:31:20.636363+0000","last_unstale":"2024-04-28T13:31:24.231240+0000","last_undegraded":"2024-04-28T13:31:24.231240+0000","last_fullsized":"2024-04-28T13:31:24.231240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:45:34.310382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730662+0000","last_change":"2024-04-28T13:31:21.636329+0000","last_active":"2024-04-28T13:31:24.730662+0000","last_peered":"2024-04-28T13:31:24.730662+0000","last_clean":"2024-04-28T13:31:24.730662+0000","last_became_active":"2024-04-28T13:31:21.635635+0000","last_became_peered":"2024-04-28T13:31:21.635635+0000","last_unstale":"2024-04-28T13:31:24.730662+0000","last_undegraded":"2024-04-28T13:31:24.730662+0000","last_fullsized":"2024-04-28T13:31:24.730662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:14:19.057730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732045+0000","last_change":"2024-04-28T13:31:20.622197+0000","last_active":"2024-04-28T13:31:24.732045+0000","last_peered":"2024-04-28T13:31:24.732045+0000","last_clean":"2024-04-28T13:31:24.732045+0000","last_became_active":"2024-04-28T13:31:20.621973+0000","last_became_peered":"2024-04-28T13:31:20.621973+0000","last_unstale":"2024-04-28T13:31:24.732045+0000","last_undegraded":"2024-04-28T13:31:24.732045+0000","last_fullsized":"2024-04-28T13:31:24.732045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:50:14.806138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730760+0000","last_change":"2024-04-28T13:31:21.642200+0000","last_active":"2024-04-28T13:31:24.730760+0000","last_peered":"2024-04-28T13:31:24.730760+0000","last_clean":"2024-04-28T13:31:24.730760+0000","last_became_active":"2024-04-28T13:31:21.641925+0000","last_became_peered":"2024-04-28T13:31:21.641925+0000","last_unstale":"2024-04-28T13:31:24.730760+0000","last_undegraded":"2024-04-28T13:31:24.730760+0000","last_fullsized":"2024-04-28T13:31:24.730760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:33.431077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730908+0000","last_change":"2024-04-28T13:31:20.628724+0000","last_active":"2024-04-28T13:31:24.730908+0000","last_peered":"2024-04-28T13:31:24.730908+0000","last_clean":"2024-04-28T13:31:24.730908+0000","last_became_active":"2024-04-28T13:31:20.628363+0000","last_became_peered":"2024-04-28T13:31:20.628363+0000","last_unstale":"2024-04-28T13:31:24.730908+0000","last_undegraded":"2024-04-28T13:31:24.730908+0000","last_fullsized":"2024-04-28T13:31:24.730908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:19:25.770351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T13:31:31.565 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T13:31:31.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.797+0000 7f055f200640 1 Processor -- start 2024-04-28T13:31:31.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.797+0000 7f055f200640 1 -- start start 2024-04-28T13:31:31.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.799+0000 7f055f200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f05580d0540 0x7f05580af560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.799+0000 7f055f200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05580b3c20 0x7f05580b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.799+0000 7f055f200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05580afaa0 con 0x7f0558073370 2024-04-28T13:31:31.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.799+0000 7f055f200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f05580b47a0 con 0x7f05580d0540 2024-04-28T13:31:31.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.799+0000 7f055f200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f05580738e0 con 0x7f05580b3c20 2024-04-28T13:31:31.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.799+0000 7f055d400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f0558073370 0x7f0558072900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56164/0 (socket says 172.21.15.196:56164) 2024-04-28T13:31:31.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.799+0000 7f055d400640 1 -- 172.21.15.196:0/976056940 learned_addr learned my addr 172.21.15.196:0/976056940 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:31.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055de00640 1 --2- 172.21.15.196:0/976056940 >> v2:172.21.15.203:3300/0 conn(0x7f05580d0540 0x7f05580af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:31.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055e800640 1 --2- 172.21.15.196:0/976056940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05580b3c20 0x7f05580b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:31.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 -- 172.21.15.196:0/976056940 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2502771682 0 0) 0x7f05580afaa0 con 0x7f0558073370 2024-04-28T13:31:31.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 -- 172.21.15.196:0/976056940 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0540003520 con 0x7f0558073370 2024-04-28T13:31:31.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 -- 172.21.15.196:0/976056940 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1688388345 0 0) 0x7f0540003520 con 0x7f0558073370 2024-04-28T13:31:31.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 -- 172.21.15.196:0/976056940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05580b3c20 msgr2=0x7f05580b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:31.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 --2- 172.21.15.196:0/976056940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05580b3c20 0x7f05580b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 -- 172.21.15.196:0/976056940 >> v2:172.21.15.203:3300/0 conn(0x7f05580d0540 msgr2=0x7f05580af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:31.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 --2- 172.21.15.196:0/976056940 >> v2:172.21.15.203:3300/0 conn(0x7f05580d0540 0x7f05580af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 -- 172.21.15.196:0/976056940 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05580aea80 con 0x7f0558073370 2024-04-28T13:31:31.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 -- 172.21.15.196:0/976056940 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f0548001670 con 0x7f0558073370 2024-04-28T13:31:31.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 -- 172.21.15.196:0/976056940 >> v1:172.21.15.196:6789/0 conn(0x7f0558073370 legacy=0x7f0558072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 --2- 172.21.15.196:0/976056940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05400039b0 0x7f0540003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 --2- 172.21.15.196:0/976056940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05580b3c20 0x7f0540004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 --2- 172.21.15.196:0/976056940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004d40 0x7f05400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 -- 172.21.15.196:0/976056940 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05580b47a0 con 0x7f0540004d40 2024-04-28T13:31:31.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 -- 172.21.15.196:0/976056940 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f05580738e0 con 0x7f05400039b0 2024-04-28T13:31:31.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055ca00640 1 -- 172.21.15.196:0/976056940 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f054000ba40 con 0x7f05580b3c20 2024-04-28T13:31:31.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.800+0000 7f055de00640 1 --2- 172.21.15.196:0/976056940 >> v2:172.21.15.203:3300/0 conn(0x7f05580d0540 0x7f05580af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:31:31.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.801+0000 7f055e800640 1 --2- 172.21.15.196:0/976056940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004d40 0x7f05400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:31.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.801+0000 7f055e800640 1 -- 172.21.15.196:0/976056940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05580b3c20 msgr2=0x7f0540004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:31.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.801+0000 7f055e800640 1 --2- 172.21.15.196:0/976056940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05580b3c20 0x7f0540004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.801+0000 7f055e800640 1 -- 172.21.15.196:0/976056940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05400039b0 msgr2=0x7f0540003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:31.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.801+0000 7f055e800640 1 --2- 172.21.15.196:0/976056940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05400039b0 0x7f0540003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.801+0000 7f055e800640 1 -- 172.21.15.196:0/976056940 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0540003520 con 0x7f0540004d40 2024-04-28T13:31:31.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.802+0000 7f055e800640 1 --2- 172.21.15.196:0/976056940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004d40 0x7f05400051c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0554009fc0 tx=0x7f0540005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77bb043b4f636ff server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:31.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.802+0000 7f055ca00640 1 -- 172.21.15.196:0/976056940 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554009c10 con 0x7f0540004d40 2024-04-28T13:31:31.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.802+0000 7f055ca00640 1 -- 172.21.15.196:0/976056940 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0554036070 con 0x7f0540004d40 2024-04-28T13:31:31.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.802+0000 7f055ca00640 1 -- 172.21.15.196:0/976056940 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055402d070 con 0x7f0540004d40 2024-04-28T13:31:31.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.803+0000 7f055f200640 1 -- 172.21.15.196:0/976056940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004d40 msgr2=0x7f05400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.803+0000 7f055f200640 1 --2- 172.21.15.196:0/976056940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004d40 0x7f05400051c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0554009fc0 tx=0x7f0540005830 comp rx=0 tx=0).stop 2024-04-28T13:31:31.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.803+0000 7f055e800640 1 -- 172.21.15.196:0/976056940 reap_dead start 2024-04-28T13:31:31.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.803+0000 7f055f200640 1 -- 172.21.15.196:0/976056940 shutdown_connections 2024-04-28T13:31:31.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.803+0000 7f055f200640 1 -- 172.21.15.196:0/976056940 >> 172.21.15.196:0/976056940 conn(0x7f055806c6c0 msgr2=0x7f055806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:31.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.803+0000 7f055f200640 1 -- 172.21.15.196:0/976056940 shutdown_connections 2024-04-28T13:31:31.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.803+0000 7f055f200640 1 -- 172.21.15.196:0/976056940 wait complete. 2024-04-28T13:31:31.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.805+0000 7f055f200640 1 Processor -- start 2024-04-28T13:31:31.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.805+0000 7f055f200640 1 -- start start 2024-04-28T13:31:31.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.806+0000 7f055f200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004d40 0x7f0558144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.806+0000 7f055f200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0558158a40 0x7f05581473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.806+0000 7f055f200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f055815a0f0 0x7f055815a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.806+0000 7f055f200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05580d0920 con 0x7f0540004d40 2024-04-28T13:31:31.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.806+0000 7f055f200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f05580af040 con 0x7f0558158a40 2024-04-28T13:31:31.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.806+0000 7f055f200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0558147ca0 con 0x7f055815a0f0 2024-04-28T13:31:31.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.806+0000 7f055de00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004d40 0x7f0558144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:31.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.806+0000 7f055de00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004d40 0x7f0558144b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35964/0 (socket says 172.21.15.196:35964) 2024-04-28T13:31:31.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.806+0000 7f055de00640 1 -- 172.21.15.196:0/2757738341 learned_addr learned my addr 172.21.15.196:0/2757738341 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:31.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.806+0000 7f055de00640 1 -- 172.21.15.196:0/2757738341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f055815a0f0 msgr2=0x7f055815a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.806+0000 7f055de00640 1 --2- 172.21.15.196:0/2757738341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f055815a0f0 0x7f055815a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.806+0000 7f055de00640 1 -- 172.21.15.196:0/2757738341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0558158a40 msgr2=0x7f05581473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:31.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.806+0000 7f055de00640 1 --2- 172.21.15.196:0/2757738341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0558158a40 0x7f05581473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:31.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.806+0000 7f055de00640 1 -- 172.21.15.196:0/2757738341 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0558147f80 con 0x7f0540004d40 2024-04-28T13:31:31.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.807+0000 7f055de00640 1 --2- 172.21.15.196:0/2757738341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004d40 0x7f0558144b60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f054c00e760 tx=0x7f054c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:31.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.807+0000 7f0552000640 1 -- 172.21.15.196:0/2757738341 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c00f040 con 0x7f0540004d40 2024-04-28T13:31:31.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.807+0000 7f0552000640 1 -- 172.21.15.196:0/2757738341 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f054c004030 con 0x7f0540004d40 2024-04-28T13:31:31.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.807+0000 7f0552000640 1 -- 172.21.15.196:0/2757738341 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c004420 con 0x7f0540004d40 2024-04-28T13:31:31.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.807+0000 7f055f200640 1 -- 172.21.15.196:0/2757738341 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0558148260 con 0x7f0540004d40 2024-04-28T13:31:31.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.807+0000 7f055f200640 1 -- 172.21.15.196:0/2757738341 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05581478e0 con 0x7f0540004d40 2024-04-28T13:31:31.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.808+0000 7f055f200640 1 -- 172.21.15.196:0/2757738341 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f055806a290 con 0x7f0540004d40 2024-04-28T13:31:31.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.809+0000 7f0552000640 1 -- 172.21.15.196:0/2757738341 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f054c00c6d0 con 0x7f0540004d40 2024-04-28T13:31:31.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.810+0000 7f0552000640 1 --2- 172.21.15.196:0/2757738341 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f053c06c8f0 0x7f053c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:31.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.810+0000 7f0552000640 1 -- 172.21.15.196:0/2757738341 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f054c08b8a0 con 0x7f0540004d40 2024-04-28T13:31:31.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.810+0000 7f055d400640 1 --2- 172.21.15.196:0/2757738341 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f053c06c8f0 0x7f053c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:31.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.811+0000 7f055d400640 1 --2- 172.21.15.196:0/2757738341 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f053c06c8f0 0x7f053c06edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0548001050 tx=0x7f0548001540 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:31.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.814+0000 7f0552000640 1 -- 172.21.15.196:0/2757738341 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f054c017020 con 0x7f0540004d40 2024-04-28T13:31:31.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:31.998+0000 7f055f200640 1 -- 172.21.15.196:0/2757738341 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f055815f830 con 0x7f053c06c8f0 2024-04-28T13:31:32.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.016+0000 7f0552000640 1 -- 172.21.15.196:0/2757738341 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f055815f830 con 0x7f053c06c8f0 2024-04-28T13:31:32.021 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:32.022 INFO:teuthology.orchestra.run.smithi196.stderr:dumped all 2024-04-28T13:31:32.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.021+0000 7f055f200640 1 -- 172.21.15.196:0/2757738341 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f053c06c8f0 msgr2=0x7f053c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:32.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.021+0000 7f055f200640 1 --2- 172.21.15.196:0/2757738341 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f053c06c8f0 0x7f053c06edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0548001050 tx=0x7f0548001540 comp rx=0 tx=0).stop 2024-04-28T13:31:32.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.021+0000 7f055f200640 1 -- 172.21.15.196:0/2757738341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004d40 msgr2=0x7f0558144b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:32.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.021+0000 7f055f200640 1 --2- 172.21.15.196:0/2757738341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004d40 0x7f0558144b60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f054c00e760 tx=0x7f054c00ec20 comp rx=0 tx=0).stop 2024-04-28T13:31:32.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.021+0000 7f055f200640 1 -- 172.21.15.196:0/2757738341 shutdown_connections 2024-04-28T13:31:32.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.021+0000 7f055f200640 1 --2- 172.21.15.196:0/2757738341 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f053c06c8f0 0x7f053c06edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.021+0000 7f055f200640 1 --2- 172.21.15.196:0/2757738341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f055815a0f0 0x7f055815a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.021+0000 7f055f200640 1 --2- 172.21.15.196:0/2757738341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0558158a40 0x7f05581473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.021+0000 7f055f200640 1 --2- 172.21.15.196:0/2757738341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004d40 0x7f0558144b60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.021+0000 7f055f200640 1 -- 172.21.15.196:0/2757738341 >> 172.21.15.196:0/2757738341 conn(0x7f055806c6c0 msgr2=0x7f055806f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:32.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.022+0000 7f055f200640 1 -- 172.21.15.196:0/2757738341 shutdown_connections 2024-04-28T13:31:32.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.022+0000 7f055f200640 1 -- 172.21.15.196:0/2757738341 wait complete. 2024-04-28T13:31:32.039 INFO:teuthology.orchestra.run.smithi196.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-28T13:31:30.183131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.570523"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.232418+0000","last_change":"2024-04-28T13:31:21.653529+0000","last_active":"2024-04-28T13:31:24.232418+0000","last_peered":"2024-04-28T13:31:24.232418+0000","last_clean":"2024-04-28T13:31:24.232418+0000","last_became_active":"2024-04-28T13:31:21.653192+0000","last_became_peered":"2024-04-28T13:31:21.653192+0000","last_unstale":"2024-04-28T13:31:24.232418+0000","last_undegraded":"2024-04-28T13:31:24.232418+0000","last_fullsized":"2024-04-28T13:31:24.232418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:30:36.318507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640382+0000","last_change":"2024-04-28T13:31:21.636383+0000","last_active":"2024-04-28T13:31:22.640382+0000","last_peered":"2024-04-28T13:31:22.640382+0000","last_clean":"2024-04-28T13:31:22.640382+0000","last_became_active":"2024-04-28T13:31:21.635970+0000","last_became_peered":"2024-04-28T13:31:21.635970+0000","last_unstale":"2024-04-28T13:31:22.640382+0000","last_undegraded":"2024-04-28T13:31:22.640382+0000","last_fullsized":"2024-04-28T13:31:22.640382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:33.431077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.729998+0000","last_change":"2024-04-28T13:31:21.642396+0000","last_active":"2024-04-28T13:31:24.729998+0000","last_peered":"2024-04-28T13:31:24.729998+0000","last_clean":"2024-04-28T13:31:24.729998+0000","last_became_active":"2024-04-28T13:31:21.642062+0000","last_became_peered":"2024-04-28T13:31:21.642062+0000","last_unstale":"2024-04-28T13:31:24.729998+0000","last_undegraded":"2024-04-28T13:31:24.729998+0000","last_fullsized":"2024-04-28T13:31:24.729998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:43:30.133775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646626+0000","last_change":"2024-04-28T13:31:21.637265+0000","last_active":"2024-04-28T13:31:22.646626+0000","last_peered":"2024-04-28T13:31:22.646626+0000","last_clean":"2024-04-28T13:31:22.646626+0000","last_became_active":"2024-04-28T13:31:21.636679+0000","last_became_peered":"2024-04-28T13:31:21.636679+0000","last_unstale":"2024-04-28T13:31:22.646626+0000","last_undegraded":"2024-04-28T13:31:22.646626+0000","last_fullsized":"2024-04-28T13:31:22.646626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:25.059779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231103+0000","last_change":"2024-04-28T13:31:21.660088+0000","last_active":"2024-04-28T13:31:24.231103+0000","last_peered":"2024-04-28T13:31:24.231103+0000","last_clean":"2024-04-28T13:31:24.231103+0000","last_became_active":"2024-04-28T13:31:21.652264+0000","last_became_peered":"2024-04-28T13:31:21.652264+0000","last_unstale":"2024-04-28T13:31:24.231103+0000","last_undegraded":"2024-04-28T13:31:24.231103+0000","last_fullsized":"2024-04-28T13:31:24.231103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:28:41.694303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646580+0000","last_change":"2024-04-28T13:31:21.641292+0000","last_active":"2024-04-28T13:31:22.646580+0000","last_peered":"2024-04-28T13:31:22.646580+0000","last_clean":"2024-04-28T13:31:22.646580+0000","last_became_active":"2024-04-28T13:31:21.635397+0000","last_became_peered":"2024-04-28T13:31:21.635397+0000","last_unstale":"2024-04-28T13:31:22.646580+0000","last_undegraded":"2024-04-28T13:31:22.646580+0000","last_fullsized":"2024-04-28T13:31:22.646580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:16:56.574292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231153+0000","last_change":"2024-04-28T13:31:21.654211+0000","last_active":"2024-04-28T13:31:24.231153+0000","last_peered":"2024-04-28T13:31:24.231153+0000","last_clean":"2024-04-28T13:31:24.231153+0000","last_became_active":"2024-04-28T13:31:21.653516+0000","last_became_peered":"2024-04-28T13:31:21.653516+0000","last_unstale":"2024-04-28T13:31:24.231153+0000","last_undegraded":"2024-04-28T13:31:24.231153+0000","last_fullsized":"2024-04-28T13:31:24.231153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:26:26.344814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.729792+0000","last_change":"2024-04-28T13:31:21.641706+0000","last_active":"2024-04-28T13:31:24.729792+0000","last_peered":"2024-04-28T13:31:24.729792+0000","last_clean":"2024-04-28T13:31:24.729792+0000","last_became_active":"2024-04-28T13:31:21.639207+0000","last_became_peered":"2024-04-28T13:31:21.639207+0000","last_unstale":"2024-04-28T13:31:24.729792+0000","last_undegraded":"2024-04-28T13:31:24.729792+0000","last_fullsized":"2024-04-28T13:31:24.729792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:33:40.307112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.232568+0000","last_change":"2024-04-28T13:31:21.657597+0000","last_active":"2024-04-28T13:31:24.232568+0000","last_peered":"2024-04-28T13:31:24.232568+0000","last_clean":"2024-04-28T13:31:24.232568+0000","last_became_active":"2024-04-28T13:31:21.656871+0000","last_became_peered":"2024-04-28T13:31:21.656871+0000","last_unstale":"2024-04-28T13:31:24.232568+0000","last_undegraded":"2024-04-28T13:31:24.232568+0000","last_fullsized":"2024-04-28T13:31:24.232568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:23:11.970214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732465+0000","last_change":"2024-04-28T13:31:21.648874+0000","last_active":"2024-04-28T13:31:24.732465+0000","last_peered":"2024-04-28T13:31:24.732465+0000","last_clean":"2024-04-28T13:31:24.732465+0000","last_became_active":"2024-04-28T13:31:21.646033+0000","last_became_peered":"2024-04-28T13:31:21.646033+0000","last_unstale":"2024-04-28T13:31:24.732465+0000","last_undegraded":"2024-04-28T13:31:24.732465+0000","last_fullsized":"2024-04-28T13:31:24.732465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:33.431077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732151+0000","last_change":"2024-04-28T13:31:21.649391+0000","last_active":"2024-04-28T13:31:24.732151+0000","last_peered":"2024-04-28T13:31:24.732151+0000","last_clean":"2024-04-28T13:31:24.732151+0000","last_became_active":"2024-04-28T13:31:21.649078+0000","last_became_peered":"2024-04-28T13:31:21.649078+0000","last_unstale":"2024-04-28T13:31:24.732151+0000","last_undegraded":"2024-04-28T13:31:24.732151+0000","last_fullsized":"2024-04-28T13:31:24.732151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:32.352858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647834+0000","last_change":"2024-04-28T13:31:21.651903+0000","last_active":"2024-04-28T13:31:22.647834+0000","last_peered":"2024-04-28T13:31:22.647834+0000","last_clean":"2024-04-28T13:31:22.647834+0000","last_became_active":"2024-04-28T13:31:21.650905+0000","last_became_peered":"2024-04-28T13:31:21.650905+0000","last_unstale":"2024-04-28T13:31:22.647834+0000","last_undegraded":"2024-04-28T13:31:22.647834+0000","last_fullsized":"2024-04-28T13:31:22.647834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:01.137759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731025+0000","last_change":"2024-04-28T13:31:21.640551+0000","last_active":"2024-04-28T13:31:24.731025+0000","last_peered":"2024-04-28T13:31:24.731025+0000","last_clean":"2024-04-28T13:31:24.731025+0000","last_became_active":"2024-04-28T13:31:21.640275+0000","last_became_peered":"2024-04-28T13:31:21.640275+0000","last_unstale":"2024-04-28T13:31:24.731025+0000","last_undegraded":"2024-04-28T13:31:24.731025+0000","last_fullsized":"2024-04-28T13:31:24.731025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:43:16.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730356+0000","last_change":"2024-04-28T13:31:21.639830+0000","last_active":"2024-04-28T13:31:24.730356+0000","last_peered":"2024-04-28T13:31:24.730356+0000","last_clean":"2024-04-28T13:31:24.730356+0000","last_became_active":"2024-04-28T13:31:21.639502+0000","last_became_peered":"2024-04-28T13:31:21.639502+0000","last_unstale":"2024-04-28T13:31:24.730356+0000","last_undegraded":"2024-04-28T13:31:24.730356+0000","last_fullsized":"2024-04-28T13:31:24.730356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:30:04.305801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640325+0000","last_change":"2024-04-28T13:31:21.635217+0000","last_active":"2024-04-28T13:31:22.640325+0000","last_peered":"2024-04-28T13:31:22.640325+0000","last_clean":"2024-04-28T13:31:22.640325+0000","last_became_active":"2024-04-28T13:31:21.634165+0000","last_became_peered":"2024-04-28T13:31:21.634165+0000","last_unstale":"2024-04-28T13:31:22.640325+0000","last_undegraded":"2024-04-28T13:31:22.640325+0000","last_fullsized":"2024-04-28T13:31:22.640325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:32.352858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646029+0000","last_change":"2024-04-28T13:31:21.642988+0000","last_active":"2024-04-28T13:31:22.646029+0000","last_peered":"2024-04-28T13:31:22.646029+0000","last_clean":"2024-04-28T13:31:22.646029+0000","last_became_active":"2024-04-28T13:31:21.635586+0000","last_became_peered":"2024-04-28T13:31:21.635586+0000","last_unstale":"2024-04-28T13:31:22.646029+0000","last_undegraded":"2024-04-28T13:31:22.646029+0000","last_fullsized":"2024-04-28T13:31:22.646029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:33.431077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647955+0000","last_change":"2024-04-28T13:31:21.653816+0000","last_active":"2024-04-28T13:31:22.647955+0000","last_peered":"2024-04-28T13:31:22.647955+0000","last_clean":"2024-04-28T13:31:22.647955+0000","last_became_active":"2024-04-28T13:31:21.653606+0000","last_became_peered":"2024-04-28T13:31:21.653606+0000","last_unstale":"2024-04-28T13:31:22.647955+0000","last_undegraded":"2024-04-28T13:31:22.647955+0000","last_fullsized":"2024-04-28T13:31:22.647955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:53:05.667773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731713+0000","last_change":"2024-04-28T13:31:21.643582+0000","last_active":"2024-04-28T13:31:24.731713+0000","last_peered":"2024-04-28T13:31:24.731713+0000","last_clean":"2024-04-28T13:31:24.731713+0000","last_became_active":"2024-04-28T13:31:21.643318+0000","last_became_peered":"2024-04-28T13:31:21.643318+0000","last_unstale":"2024-04-28T13:31:24.731713+0000","last_undegraded":"2024-04-28T13:31:24.731713+0000","last_fullsized":"2024-04-28T13:31:24.731713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:45:35.324680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231673+0000","last_change":"2024-04-28T13:31:21.659912+0000","last_active":"2024-04-28T13:31:24.231673+0000","last_peered":"2024-04-28T13:31:24.231673+0000","last_clean":"2024-04-28T13:31:24.231673+0000","last_became_active":"2024-04-28T13:31:21.659577+0000","last_became_peered":"2024-04-28T13:31:21.659577+0000","last_unstale":"2024-04-28T13:31:24.231673+0000","last_undegraded":"2024-04-28T13:31:24.231673+0000","last_fullsized":"2024-04-28T13:31:24.231673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:04:16.728668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731562+0000","last_change":"2024-04-28T13:31:21.654570+0000","last_active":"2024-04-28T13:31:24.731562+0000","last_peered":"2024-04-28T13:31:24.731562+0000","last_clean":"2024-04-28T13:31:24.731562+0000","last_became_active":"2024-04-28T13:31:21.654238+0000","last_became_peered":"2024-04-28T13:31:21.654238+0000","last_unstale":"2024-04-28T13:31:24.731562+0000","last_undegraded":"2024-04-28T13:31:24.731562+0000","last_fullsized":"2024-04-28T13:31:24.731562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:33:40.307112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730818+0000","last_change":"2024-04-28T13:31:21.653783+0000","last_active":"2024-04-28T13:31:24.730818+0000","last_peered":"2024-04-28T13:31:24.730818+0000","last_clean":"2024-04-28T13:31:24.730818+0000","last_became_active":"2024-04-28T13:31:21.653493+0000","last_became_peered":"2024-04-28T13:31:21.653493+0000","last_unstale":"2024-04-28T13:31:24.730818+0000","last_undegraded":"2024-04-28T13:31:24.730818+0000","last_fullsized":"2024-04-28T13:31:24.730818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:52:32.051589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647048+0000","last_change":"2024-04-28T13:31:21.660686+0000","last_active":"2024-04-28T13:31:22.647048+0000","last_peered":"2024-04-28T13:31:22.647048+0000","last_clean":"2024-04-28T13:31:22.647048+0000","last_became_active":"2024-04-28T13:31:21.660320+0000","last_became_peered":"2024-04-28T13:31:21.660320+0000","last_unstale":"2024-04-28T13:31:22.647048+0000","last_undegraded":"2024-04-28T13:31:22.647048+0000","last_fullsized":"2024-04-28T13:31:22.647048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:09:03.049024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730352+0000","last_change":"2024-04-28T13:31:21.640131+0000","last_active":"2024-04-28T13:31:24.730352+0000","last_peered":"2024-04-28T13:31:24.730352+0000","last_clean":"2024-04-28T13:31:24.730352+0000","last_became_active":"2024-04-28T13:31:21.639821+0000","last_became_peered":"2024-04-28T13:31:21.639821+0000","last_unstale":"2024-04-28T13:31:24.730352+0000","last_undegraded":"2024-04-28T13:31:24.730352+0000","last_fullsized":"2024-04-28T13:31:24.730352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:39:23.345225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646449+0000","last_change":"2024-04-28T13:31:21.648922+0000","last_active":"2024-04-28T13:31:22.646449+0000","last_peered":"2024-04-28T13:31:22.646449+0000","last_clean":"2024-04-28T13:31:22.646449+0000","last_became_active":"2024-04-28T13:31:21.639876+0000","last_became_peered":"2024-04-28T13:31:21.639876+0000","last_unstale":"2024-04-28T13:31:22.646449+0000","last_undegraded":"2024-04-28T13:31:22.646449+0000","last_fullsized":"2024-04-28T13:31:22.646449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:43:16.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732198+0000","last_change":"2024-04-28T13:31:21.644835+0000","last_active":"2024-04-28T13:31:24.732198+0000","last_peered":"2024-04-28T13:31:24.732198+0000","last_clean":"2024-04-28T13:31:24.732198+0000","last_became_active":"2024-04-28T13:31:21.644564+0000","last_became_peered":"2024-04-28T13:31:21.644564+0000","last_unstale":"2024-04-28T13:31:24.732198+0000","last_undegraded":"2024-04-28T13:31:24.732198+0000","last_fullsized":"2024-04-28T13:31:24.732198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:09:03.049024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732493+0000","last_change":"2024-04-28T13:31:21.653502+0000","last_active":"2024-04-28T13:31:24.732493+0000","last_peered":"2024-04-28T13:31:24.732493+0000","last_clean":"2024-04-28T13:31:24.732493+0000","last_became_active":"2024-04-28T13:31:21.653235+0000","last_became_peered":"2024-04-28T13:31:21.653235+0000","last_unstale":"2024-04-28T13:31:24.732493+0000","last_undegraded":"2024-04-28T13:31:24.732493+0000","last_fullsized":"2024-04-28T13:31:24.732493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:43:16.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640377+0000","last_change":"2024-04-28T13:31:21.649404+0000","last_active":"2024-04-28T13:31:22.640377+0000","last_peered":"2024-04-28T13:31:22.640377+0000","last_clean":"2024-04-28T13:31:22.640377+0000","last_became_active":"2024-04-28T13:31:21.649097+0000","last_became_peered":"2024-04-28T13:31:21.649097+0000","last_unstale":"2024-04-28T13:31:22.640377+0000","last_undegraded":"2024-04-28T13:31:22.640377+0000","last_fullsized":"2024-04-28T13:31:22.640377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:43:16.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731939+0000","last_change":"2024-04-28T13:31:21.649378+0000","last_active":"2024-04-28T13:31:24.731939+0000","last_peered":"2024-04-28T13:31:24.731939+0000","last_clean":"2024-04-28T13:31:24.731939+0000","last_became_active":"2024-04-28T13:31:21.647397+0000","last_became_peered":"2024-04-28T13:31:21.647397+0000","last_unstale":"2024-04-28T13:31:24.731939+0000","last_undegraded":"2024-04-28T13:31:24.731939+0000","last_fullsized":"2024-04-28T13:31:24.731939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:25.059779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730846+0000","last_change":"2024-04-28T13:31:21.642005+0000","last_active":"2024-04-28T13:31:24.730846+0000","last_peered":"2024-04-28T13:31:24.730846+0000","last_clean":"2024-04-28T13:31:24.730846+0000","last_became_active":"2024-04-28T13:31:21.641662+0000","last_became_peered":"2024-04-28T13:31:21.641662+0000","last_unstale":"2024-04-28T13:31:24.730846+0000","last_undegraded":"2024-04-28T13:31:24.730846+0000","last_fullsized":"2024-04-28T13:31:24.730846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:32.352858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647442+0000","last_change":"2024-04-28T13:31:21.642673+0000","last_active":"2024-04-28T13:31:22.647442+0000","last_peered":"2024-04-28T13:31:22.647442+0000","last_clean":"2024-04-28T13:31:22.647442+0000","last_became_active":"2024-04-28T13:31:21.640799+0000","last_became_peered":"2024-04-28T13:31:21.640799+0000","last_unstale":"2024-04-28T13:31:22.647442+0000","last_undegraded":"2024-04-28T13:31:22.647442+0000","last_fullsized":"2024-04-28T13:31:22.647442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:04:16.728668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730785+0000","last_change":"2024-04-28T13:31:21.657008+0000","last_active":"2024-04-28T13:31:24.730785+0000","last_peered":"2024-04-28T13:31:24.730785+0000","last_clean":"2024-04-28T13:31:24.730785+0000","last_became_active":"2024-04-28T13:31:21.656390+0000","last_became_peered":"2024-04-28T13:31:21.656390+0000","last_unstale":"2024-04-28T13:31:24.730785+0000","last_undegraded":"2024-04-28T13:31:24.730785+0000","last_fullsized":"2024-04-28T13:31:24.730785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:01.137759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731902+0000","last_change":"2024-04-28T13:31:21.643888+0000","last_active":"2024-04-28T13:31:24.731902+0000","last_peered":"2024-04-28T13:31:24.731902+0000","last_clean":"2024-04-28T13:31:24.731902+0000","last_became_active":"2024-04-28T13:31:21.643153+0000","last_became_peered":"2024-04-28T13:31:21.643153+0000","last_unstale":"2024-04-28T13:31:24.731902+0000","last_undegraded":"2024-04-28T13:31:24.731902+0000","last_fullsized":"2024-04-28T13:31:24.731902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:52:32.051589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.645551+0000","last_change":"2024-04-28T13:31:21.640935+0000","last_active":"2024-04-28T13:31:22.645551+0000","last_peered":"2024-04-28T13:31:22.645551+0000","last_clean":"2024-04-28T13:31:22.645551+0000","last_became_active":"2024-04-28T13:31:21.640633+0000","last_became_peered":"2024-04-28T13:31:21.640633+0000","last_unstale":"2024-04-28T13:31:22.645551+0000","last_undegraded":"2024-04-28T13:31:22.645551+0000","last_fullsized":"2024-04-28T13:31:22.645551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:23.207789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731261+0000","last_change":"2024-04-28T13:31:20.624370+0000","last_active":"2024-04-28T13:31:24.731261+0000","last_peered":"2024-04-28T13:31:24.731261+0000","last_clean":"2024-04-28T13:31:24.731261+0000","last_became_active":"2024-04-28T13:31:20.624091+0000","last_became_peered":"2024-04-28T13:31:20.624091+0000","last_unstale":"2024-04-28T13:31:24.731261+0000","last_undegraded":"2024-04-28T13:31:24.731261+0000","last_fullsized":"2024-04-28T13:31:24.731261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:35.060920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731062+0000","last_change":"2024-04-28T13:31:21.626821+0000","last_active":"2024-04-28T13:31:24.731062+0000","last_peered":"2024-04-28T13:31:24.731062+0000","last_clean":"2024-04-28T13:31:24.731062+0000","last_became_active":"2024-04-28T13:31:21.625590+0000","last_became_peered":"2024-04-28T13:31:21.625590+0000","last_unstale":"2024-04-28T13:31:24.731062+0000","last_undegraded":"2024-04-28T13:31:24.731062+0000","last_fullsized":"2024-04-28T13:31:24.731062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:33.431077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731325+0000","last_change":"2024-04-28T13:31:20.629009+0000","last_active":"2024-04-28T13:31:24.731325+0000","last_peered":"2024-04-28T13:31:24.731325+0000","last_clean":"2024-04-28T13:31:24.731325+0000","last_became_active":"2024-04-28T13:31:20.627394+0000","last_became_peered":"2024-04-28T13:31:20.627394+0000","last_unstale":"2024-04-28T13:31:24.731325+0000","last_undegraded":"2024-04-28T13:31:24.731325+0000","last_fullsized":"2024-04-28T13:31:24.731325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:01.229713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730417+0000","last_change":"2024-04-28T13:31:20.624345+0000","last_active":"2024-04-28T13:31:24.730417+0000","last_peered":"2024-04-28T13:31:24.730417+0000","last_clean":"2024-04-28T13:31:24.730417+0000","last_became_active":"2024-04-28T13:31:20.624052+0000","last_became_peered":"2024-04-28T13:31:20.624052+0000","last_unstale":"2024-04-28T13:31:24.730417+0000","last_undegraded":"2024-04-28T13:31:24.730417+0000","last_fullsized":"2024-04-28T13:31:24.730417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:46.616532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.729933+0000","last_change":"2024-04-28T13:31:21.637599+0000","last_active":"2024-04-28T13:31:24.729933+0000","last_peered":"2024-04-28T13:31:24.729933+0000","last_clean":"2024-04-28T13:31:24.729933+0000","last_became_active":"2024-04-28T13:31:21.637366+0000","last_became_peered":"2024-04-28T13:31:21.637366+0000","last_unstale":"2024-04-28T13:31:24.729933+0000","last_undegraded":"2024-04-28T13:31:24.729933+0000","last_fullsized":"2024-04-28T13:31:24.729933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:14:19.057730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646642+0000","last_change":"2024-04-28T13:31:20.625321+0000","last_active":"2024-04-28T13:31:22.646642+0000","last_peered":"2024-04-28T13:31:22.646642+0000","last_clean":"2024-04-28T13:31:22.646642+0000","last_became_active":"2024-04-28T13:31:20.624034+0000","last_became_peered":"2024-04-28T13:31:20.624034+0000","last_unstale":"2024-04-28T13:31:22.646642+0000","last_undegraded":"2024-04-28T13:31:22.646642+0000","last_fullsized":"2024-04-28T13:31:22.646642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:22.193491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.639404+0000","last_change":"2024-04-28T13:31:21.650660+0000","last_active":"2024-04-28T13:31:22.639404+0000","last_peered":"2024-04-28T13:31:22.639404+0000","last_clean":"2024-04-28T13:31:22.639404+0000","last_became_active":"2024-04-28T13:31:21.650349+0000","last_became_peered":"2024-04-28T13:31:21.650349+0000","last_unstale":"2024-04-28T13:31:22.639404+0000","last_undegraded":"2024-04-28T13:31:22.639404+0000","last_fullsized":"2024-04-28T13:31:22.639404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:14:19.057730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732233+0000","last_change":"2024-04-28T13:31:20.632738+0000","last_active":"2024-04-28T13:31:24.732233+0000","last_peered":"2024-04-28T13:31:24.732233+0000","last_clean":"2024-04-28T13:31:24.732233+0000","last_became_active":"2024-04-28T13:31:20.631174+0000","last_became_peered":"2024-04-28T13:31:20.631174+0000","last_unstale":"2024-04-28T13:31:24.732233+0000","last_undegraded":"2024-04-28T13:31:24.732233+0000","last_fullsized":"2024-04-28T13:31:24.732233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:50.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647343+0000","last_change":"2024-04-28T13:31:21.639067+0000","last_active":"2024-04-28T13:31:22.647343+0000","last_peered":"2024-04-28T13:31:22.647343+0000","last_clean":"2024-04-28T13:31:22.647343+0000","last_became_active":"2024-04-28T13:31:21.638664+0000","last_became_peered":"2024-04-28T13:31:21.638664+0000","last_unstale":"2024-04-28T13:31:22.647343+0000","last_undegraded":"2024-04-28T13:31:22.647343+0000","last_fullsized":"2024-04-28T13:31:22.647343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:44:09.266587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647483+0000","last_change":"2024-04-28T13:31:20.631748+0000","last_active":"2024-04-28T13:31:22.647483+0000","last_peered":"2024-04-28T13:31:22.647483+0000","last_clean":"2024-04-28T13:31:22.647483+0000","last_became_active":"2024-04-28T13:31:20.631345+0000","last_became_peered":"2024-04-28T13:31:20.631345+0000","last_unstale":"2024-04-28T13:31:22.647483+0000","last_undegraded":"2024-04-28T13:31:22.647483+0000","last_fullsized":"2024-04-28T13:31:22.647483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:09:16.028560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730844+0000","last_change":"2024-04-28T13:31:21.636554+0000","last_active":"2024-04-28T13:31:24.730844+0000","last_peered":"2024-04-28T13:31:24.730844+0000","last_clean":"2024-04-28T13:31:24.730844+0000","last_became_active":"2024-04-28T13:31:21.636062+0000","last_became_peered":"2024-04-28T13:31:21.636062+0000","last_unstale":"2024-04-28T13:31:24.730844+0000","last_undegraded":"2024-04-28T13:31:24.730844+0000","last_fullsized":"2024-04-28T13:31:24.730844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:39:23.345225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640458+0000","last_change":"2024-04-28T13:31:20.627856+0000","last_active":"2024-04-28T13:31:22.640458+0000","last_peered":"2024-04-28T13:31:22.640458+0000","last_clean":"2024-04-28T13:31:22.640458+0000","last_became_active":"2024-04-28T13:31:20.627632+0000","last_became_peered":"2024-04-28T13:31:20.627632+0000","last_unstale":"2024-04-28T13:31:22.640458+0000","last_undegraded":"2024-04-28T13:31:22.640458+0000","last_fullsized":"2024-04-28T13:31:22.640458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:09:16.028560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.648191+0000","last_change":"2024-04-28T13:31:21.657348+0000","last_active":"2024-04-28T13:31:22.648191+0000","last_peered":"2024-04-28T13:31:22.648191+0000","last_clean":"2024-04-28T13:31:22.648191+0000","last_became_active":"2024-04-28T13:31:21.657055+0000","last_became_peered":"2024-04-28T13:31:21.657055+0000","last_unstale":"2024-04-28T13:31:22.648191+0000","last_undegraded":"2024-04-28T13:31:22.648191+0000","last_fullsized":"2024-04-28T13:31:22.648191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:45:35.324680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731568+0000","last_change":"2024-04-28T13:31:20.629687+0000","last_active":"2024-04-28T13:31:24.731568+0000","last_peered":"2024-04-28T13:31:24.731568+0000","last_clean":"2024-04-28T13:31:24.731568+0000","last_became_active":"2024-04-28T13:31:20.628070+0000","last_became_peered":"2024-04-28T13:31:20.628070+0000","last_unstale":"2024-04-28T13:31:24.731568+0000","last_undegraded":"2024-04-28T13:31:24.731568+0000","last_fullsized":"2024-04-28T13:31:24.731568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:09:16.028560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231080+0000","last_change":"2024-04-28T13:31:21.659218+0000","last_active":"2024-04-28T13:31:24.231080+0000","last_peered":"2024-04-28T13:31:24.231080+0000","last_clean":"2024-04-28T13:31:24.231080+0000","last_became_active":"2024-04-28T13:31:21.658927+0000","last_became_peered":"2024-04-28T13:31:21.658927+0000","last_unstale":"2024-04-28T13:31:24.231080+0000","last_undegraded":"2024-04-28T13:31:24.231080+0000","last_fullsized":"2024-04-28T13:31:24.231080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:09:03.049024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730927+0000","last_change":"2024-04-28T13:31:20.624425+0000","last_active":"2024-04-28T13:31:24.730927+0000","last_peered":"2024-04-28T13:31:24.730927+0000","last_clean":"2024-04-28T13:31:24.730927+0000","last_became_active":"2024-04-28T13:31:20.623737+0000","last_became_peered":"2024-04-28T13:31:20.623737+0000","last_unstale":"2024-04-28T13:31:24.730927+0000","last_undegraded":"2024-04-28T13:31:24.730927+0000","last_fullsized":"2024-04-28T13:31:24.730927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:09:09.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731087+0000","last_change":"2024-04-28T13:31:21.658397+0000","last_active":"2024-04-28T13:31:24.731087+0000","last_peered":"2024-04-28T13:31:24.731087+0000","last_clean":"2024-04-28T13:31:24.731087+0000","last_became_active":"2024-04-28T13:31:21.658082+0000","last_became_peered":"2024-04-28T13:31:21.658082+0000","last_unstale":"2024-04-28T13:31:24.731087+0000","last_undegraded":"2024-04-28T13:31:24.731087+0000","last_fullsized":"2024-04-28T13:31:24.731087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:43:16.835525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647574+0000","last_change":"2024-04-28T13:31:20.625371+0000","last_active":"2024-04-28T13:31:22.647574+0000","last_peered":"2024-04-28T13:31:22.647574+0000","last_clean":"2024-04-28T13:31:22.647574+0000","last_became_active":"2024-04-28T13:31:20.625028+0000","last_became_peered":"2024-04-28T13:31:20.625028+0000","last_unstale":"2024-04-28T13:31:22.647574+0000","last_undegraded":"2024-04-28T13:31:22.647574+0000","last_fullsized":"2024-04-28T13:31:22.647574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:01.229713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231541+0000","last_change":"2024-04-28T13:31:21.660871+0000","last_active":"2024-04-28T13:31:24.231541+0000","last_peered":"2024-04-28T13:31:24.231541+0000","last_clean":"2024-04-28T13:31:24.231541+0000","last_became_active":"2024-04-28T13:31:21.660649+0000","last_became_peered":"2024-04-28T13:31:21.660649+0000","last_unstale":"2024-04-28T13:31:24.231541+0000","last_undegraded":"2024-04-28T13:31:24.231541+0000","last_fullsized":"2024-04-28T13:31:24.231541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:44:09.266587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731265+0000","last_change":"2024-04-28T13:31:20.625903+0000","last_active":"2024-04-28T13:31:24.731265+0000","last_peered":"2024-04-28T13:31:24.731265+0000","last_clean":"2024-04-28T13:31:24.731265+0000","last_became_active":"2024-04-28T13:31:20.625350+0000","last_became_peered":"2024-04-28T13:31:20.625350+0000","last_unstale":"2024-04-28T13:31:24.731265+0000","last_undegraded":"2024-04-28T13:31:24.731265+0000","last_fullsized":"2024-04-28T13:31:24.731265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:35.060920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731578+0000","last_change":"2024-04-28T13:31:21.643517+0000","last_active":"2024-04-28T13:31:24.731578+0000","last_peered":"2024-04-28T13:31:24.731578+0000","last_clean":"2024-04-28T13:31:24.731578+0000","last_became_active":"2024-04-28T13:31:21.643128+0000","last_became_peered":"2024-04-28T13:31:21.643128+0000","last_unstale":"2024-04-28T13:31:24.731578+0000","last_undegraded":"2024-04-28T13:31:24.731578+0000","last_fullsized":"2024-04-28T13:31:24.731578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:14:19.057730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.729987+0000","last_change":"2024-04-28T13:31:20.609440+0000","last_active":"2024-04-28T13:31:24.729987+0000","last_peered":"2024-04-28T13:31:24.729987+0000","last_clean":"2024-04-28T13:31:24.729987+0000","last_became_active":"2024-04-28T13:31:20.609120+0000","last_became_peered":"2024-04-28T13:31:20.609120+0000","last_unstale":"2024-04-28T13:31:24.729987+0000","last_undegraded":"2024-04-28T13:31:24.729987+0000","last_fullsized":"2024-04-28T13:31:24.729987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:01.229713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.230626+0000","last_change":"2024-04-28T13:31:21.654431+0000","last_active":"2024-04-28T13:31:24.230626+0000","last_peered":"2024-04-28T13:31:24.230626+0000","last_clean":"2024-04-28T13:31:24.230626+0000","last_became_active":"2024-04-28T13:31:21.653884+0000","last_became_peered":"2024-04-28T13:31:21.653884+0000","last_unstale":"2024-04-28T13:31:24.230626+0000","last_undegraded":"2024-04-28T13:31:24.230626+0000","last_fullsized":"2024-04-28T13:31:24.230626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:41:07.707990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646224+0000","last_change":"2024-04-28T13:31:20.619411+0000","last_active":"2024-04-28T13:31:22.646224+0000","last_peered":"2024-04-28T13:31:22.646224+0000","last_clean":"2024-04-28T13:31:22.646224+0000","last_became_active":"2024-04-28T13:31:20.618632+0000","last_became_peered":"2024-04-28T13:31:20.618632+0000","last_unstale":"2024-04-28T13:31:22.646224+0000","last_undegraded":"2024-04-28T13:31:22.646224+0000","last_fullsized":"2024-04-28T13:31:22.646224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:09:09.948446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.232213+0000","last_change":"2024-04-28T13:31:21.656026+0000","last_active":"2024-04-28T13:31:24.232213+0000","last_peered":"2024-04-28T13:31:24.232213+0000","last_clean":"2024-04-28T13:31:24.232213+0000","last_became_active":"2024-04-28T13:31:21.655737+0000","last_became_peered":"2024-04-28T13:31:21.655737+0000","last_unstale":"2024-04-28T13:31:24.232213+0000","last_undegraded":"2024-04-28T13:31:24.232213+0000","last_fullsized":"2024-04-28T13:31:24.232213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:53:05.667773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731528+0000","last_change":"2024-04-28T13:31:20.616200+0000","last_active":"2024-04-28T13:31:24.731528+0000","last_peered":"2024-04-28T13:31:24.731528+0000","last_clean":"2024-04-28T13:31:24.731528+0000","last_became_active":"2024-04-28T13:31:20.615813+0000","last_became_peered":"2024-04-28T13:31:20.615813+0000","last_unstale":"2024-04-28T13:31:24.731528+0000","last_undegraded":"2024-04-28T13:31:24.731528+0000","last_fullsized":"2024-04-28T13:31:24.731528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:35.060920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.232625+0000","last_change":"2024-04-28T13:31:21.657900+0000","last_active":"2024-04-28T13:31:24.232625+0000","last_peered":"2024-04-28T13:31:24.232625+0000","last_clean":"2024-04-28T13:31:24.232625+0000","last_became_active":"2024-04-28T13:31:21.657276+0000","last_became_peered":"2024-04-28T13:31:21.657276+0000","last_unstale":"2024-04-28T13:31:24.232625+0000","last_undegraded":"2024-04-28T13:31:24.232625+0000","last_fullsized":"2024-04-28T13:31:24.232625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:15:21.164101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640518+0000","last_change":"2024-04-28T13:31:20.620725+0000","last_active":"2024-04-28T13:31:22.640518+0000","last_peered":"2024-04-28T13:31:22.640518+0000","last_clean":"2024-04-28T13:31:22.640518+0000","last_became_active":"2024-04-28T13:31:20.620392+0000","last_became_peered":"2024-04-28T13:31:20.620392+0000","last_unstale":"2024-04-28T13:31:22.640518+0000","last_undegraded":"2024-04-28T13:31:22.640518+0000","last_fullsized":"2024-04-28T13:31:22.640518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:50.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647204+0000","last_change":"2024-04-28T13:31:21.640722+0000","last_active":"2024-04-28T13:31:22.647204+0000","last_peered":"2024-04-28T13:31:22.647204+0000","last_clean":"2024-04-28T13:31:22.647204+0000","last_became_active":"2024-04-28T13:31:21.637069+0000","last_became_peered":"2024-04-28T13:31:21.637069+0000","last_unstale":"2024-04-28T13:31:22.647204+0000","last_undegraded":"2024-04-28T13:31:22.647204+0000","last_fullsized":"2024-04-28T13:31:22.647204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:26:26.344814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731464+0000","last_change":"2024-04-28T13:31:20.623333+0000","last_active":"2024-04-28T13:31:24.731464+0000","last_peered":"2024-04-28T13:31:24.731464+0000","last_clean":"2024-04-28T13:31:24.731464+0000","last_became_active":"2024-04-28T13:31:20.623063+0000","last_became_peered":"2024-04-28T13:31:20.623063+0000","last_unstale":"2024-04-28T13:31:24.731464+0000","last_undegraded":"2024-04-28T13:31:24.731464+0000","last_fullsized":"2024-04-28T13:31:24.731464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:50.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231274+0000","last_change":"2024-04-28T13:31:21.663106+0000","last_active":"2024-04-28T13:31:24.231274+0000","last_peered":"2024-04-28T13:31:24.231274+0000","last_clean":"2024-04-28T13:31:24.231274+0000","last_became_active":"2024-04-28T13:31:21.662627+0000","last_became_peered":"2024-04-28T13:31:21.662627+0000","last_unstale":"2024-04-28T13:31:24.231274+0000","last_undegraded":"2024-04-28T13:31:24.231274+0000","last_fullsized":"2024-04-28T13:31:24.231274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:04:59.462014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.230624+0000","last_change":"2024-04-28T13:31:11.831538+0000","last_active":"2024-04-28T13:31:24.230624+0000","last_peered":"2024-04-28T13:31:24.230624+0000","last_clean":"2024-04-28T13:31:24.230624+0000","last_became_active":"2024-04-28T13:31:11.828959+0000","last_became_peered":"2024-04-28T13:31:11.828959+0000","last_unstale":"2024-04-28T13:31:24.230624+0000","last_undegraded":"2024-04-28T13:31:24.230624+0000","last_fullsized":"2024-04-28T13:31:24.230624+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_clean_scrub_stamp":"2024-04-28T13:31:10.444388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:25:27.078911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.645900+0000","last_change":"2024-04-28T13:31:20.619170+0000","last_active":"2024-04-28T13:31:22.645900+0000","last_peered":"2024-04-28T13:31:22.645900+0000","last_clean":"2024-04-28T13:31:22.645900+0000","last_became_active":"2024-04-28T13:31:20.618861+0000","last_became_peered":"2024-04-28T13:31:20.618861+0000","last_unstale":"2024-04-28T13:31:22.645900+0000","last_undegraded":"2024-04-28T13:31:22.645900+0000","last_fullsized":"2024-04-28T13:31:22.645900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:35.060920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730744+0000","last_change":"2024-04-28T13:31:21.649271+0000","last_active":"2024-04-28T13:31:24.730744+0000","last_peered":"2024-04-28T13:31:24.730744+0000","last_clean":"2024-04-28T13:31:24.730744+0000","last_became_active":"2024-04-28T13:31:21.649009+0000","last_became_peered":"2024-04-28T13:31:21.649009+0000","last_unstale":"2024-04-28T13:31:24.730744+0000","last_undegraded":"2024-04-28T13:31:24.730744+0000","last_fullsized":"2024-04-28T13:31:24.730744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:32.352858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.230284+0000","last_change":"2024-04-28T13:31:20.628745+0000","last_active":"2024-04-28T13:31:24.230284+0000","last_peered":"2024-04-28T13:31:24.230284+0000","last_clean":"2024-04-28T13:31:24.230284+0000","last_became_active":"2024-04-28T13:31:20.628430+0000","last_became_peered":"2024-04-28T13:31:20.628430+0000","last_unstale":"2024-04-28T13:31:24.230284+0000","last_undegraded":"2024-04-28T13:31:24.230284+0000","last_fullsized":"2024-04-28T13:31:24.230284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:09:16.028560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.729791+0000","last_change":"2024-04-28T13:31:21.628901+0000","last_active":"2024-04-28T13:31:24.729791+0000","last_peered":"2024-04-28T13:31:24.729791+0000","last_clean":"2024-04-28T13:31:24.729791+0000","last_became_active":"2024-04-28T13:31:21.628483+0000","last_became_peered":"2024-04-28T13:31:21.628483+0000","last_unstale":"2024-04-28T13:31:24.729791+0000","last_undegraded":"2024-04-28T13:31:24.729791+0000","last_fullsized":"2024-04-28T13:31:24.729791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:06:20.265833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731104+0000","last_change":"2024-04-28T13:31:20.615910+0000","last_active":"2024-04-28T13:31:24.731104+0000","last_peered":"2024-04-28T13:31:24.731104+0000","last_clean":"2024-04-28T13:31:24.731104+0000","last_became_active":"2024-04-28T13:31:20.615610+0000","last_became_peered":"2024-04-28T13:31:20.615610+0000","last_unstale":"2024-04-28T13:31:24.731104+0000","last_undegraded":"2024-04-28T13:31:24.731104+0000","last_fullsized":"2024-04-28T13:31:24.731104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:50:14.806138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.639480+0000","last_change":"2024-04-28T13:31:21.648562+0000","last_active":"2024-04-28T13:31:22.639480+0000","last_peered":"2024-04-28T13:31:22.639480+0000","last_clean":"2024-04-28T13:31:22.639480+0000","last_became_active":"2024-04-28T13:31:21.648282+0000","last_became_peered":"2024-04-28T13:31:21.648282+0000","last_unstale":"2024-04-28T13:31:22.639480+0000","last_undegraded":"2024-04-28T13:31:22.639480+0000","last_fullsized":"2024-04-28T13:31:22.639480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:52:32.051589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647763+0000","last_change":"2024-04-28T13:31:20.623861+0000","last_active":"2024-04-28T13:31:22.647763+0000","last_peered":"2024-04-28T13:31:22.647763+0000","last_clean":"2024-04-28T13:31:22.647763+0000","last_became_active":"2024-04-28T13:31:20.623459+0000","last_became_peered":"2024-04-28T13:31:20.623459+0000","last_unstale":"2024-04-28T13:31:22.647763+0000","last_undegraded":"2024-04-28T13:31:22.647763+0000","last_fullsized":"2024-04-28T13:31:22.647763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:50.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.645431+0000","last_change":"2024-04-28T13:31:21.634320+0000","last_active":"2024-04-28T13:31:22.645431+0000","last_peered":"2024-04-28T13:31:22.645431+0000","last_clean":"2024-04-28T13:31:22.645431+0000","last_became_active":"2024-04-28T13:31:21.634045+0000","last_became_peered":"2024-04-28T13:31:21.634045+0000","last_unstale":"2024-04-28T13:31:22.645431+0000","last_undegraded":"2024-04-28T13:31:22.645431+0000","last_fullsized":"2024-04-28T13:31:22.645431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:51.731818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.639846+0000","last_change":"2024-04-28T13:31:20.612653+0000","last_active":"2024-04-28T13:31:22.639846+0000","last_peered":"2024-04-28T13:31:22.639846+0000","last_clean":"2024-04-28T13:31:22.639846+0000","last_became_active":"2024-04-28T13:31:20.612341+0000","last_became_peered":"2024-04-28T13:31:20.612341+0000","last_unstale":"2024-04-28T13:31:22.639846+0000","last_undegraded":"2024-04-28T13:31:22.639846+0000","last_fullsized":"2024-04-28T13:31:22.639846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:45:34.310382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647331+0000","last_change":"2024-04-28T13:31:21.642073+0000","last_active":"2024-04-28T13:31:22.647331+0000","last_peered":"2024-04-28T13:31:22.647331+0000","last_clean":"2024-04-28T13:31:22.647331+0000","last_became_active":"2024-04-28T13:31:21.641771+0000","last_became_peered":"2024-04-28T13:31:21.641771+0000","last_unstale":"2024-04-28T13:31:22.647331+0000","last_undegraded":"2024-04-28T13:31:22.647331+0000","last_fullsized":"2024-04-28T13:31:22.647331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:04:59.462014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231572+0000","last_change":"2024-04-28T13:31:20.634223+0000","last_active":"2024-04-28T13:31:24.231572+0000","last_peered":"2024-04-28T13:31:24.231572+0000","last_clean":"2024-04-28T13:31:24.231572+0000","last_became_active":"2024-04-28T13:31:20.633834+0000","last_became_peered":"2024-04-28T13:31:20.633834+0000","last_unstale":"2024-04-28T13:31:24.231572+0000","last_undegraded":"2024-04-28T13:31:24.231572+0000","last_fullsized":"2024-04-28T13:31:24.231572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:37:34.377933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.639712+0000","last_change":"2024-04-28T13:31:21.643483+0000","last_active":"2024-04-28T13:31:22.639712+0000","last_peered":"2024-04-28T13:31:22.639712+0000","last_clean":"2024-04-28T13:31:22.639712+0000","last_became_active":"2024-04-28T13:31:21.643157+0000","last_became_peered":"2024-04-28T13:31:21.643157+0000","last_unstale":"2024-04-28T13:31:22.639712+0000","last_undegraded":"2024-04-28T13:31:22.639712+0000","last_fullsized":"2024-04-28T13:31:22.639712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:39:23.345225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640516+0000","last_change":"2024-04-28T13:31:20.630471+0000","last_active":"2024-04-28T13:31:22.640516+0000","last_peered":"2024-04-28T13:31:22.640516+0000","last_clean":"2024-04-28T13:31:22.640516+0000","last_became_active":"2024-04-28T13:31:20.630161+0000","last_became_peered":"2024-04-28T13:31:20.630161+0000","last_unstale":"2024-04-28T13:31:22.640516+0000","last_undegraded":"2024-04-28T13:31:22.640516+0000","last_fullsized":"2024-04-28T13:31:22.640516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:22.193491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732154+0000","last_change":"2024-04-28T13:31:21.648853+0000","last_active":"2024-04-28T13:31:24.732154+0000","last_peered":"2024-04-28T13:31:24.732154+0000","last_clean":"2024-04-28T13:31:24.732154+0000","last_became_active":"2024-04-28T13:31:21.647783+0000","last_became_peered":"2024-04-28T13:31:21.647783+0000","last_unstale":"2024-04-28T13:31:24.732154+0000","last_undegraded":"2024-04-28T13:31:24.732154+0000","last_fullsized":"2024-04-28T13:31:24.732154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:41:07.707990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732342+0000","last_change":"2024-04-28T13:31:20.629409+0000","last_active":"2024-04-28T13:31:24.732342+0000","last_peered":"2024-04-28T13:31:24.732342+0000","last_clean":"2024-04-28T13:31:24.732342+0000","last_became_active":"2024-04-28T13:31:20.628182+0000","last_became_peered":"2024-04-28T13:31:20.628182+0000","last_unstale":"2024-04-28T13:31:24.732342+0000","last_undegraded":"2024-04-28T13:31:24.732342+0000","last_fullsized":"2024-04-28T13:31:24.732342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:22.193491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730384+0000","last_change":"2024-04-28T13:31:21.640225+0000","last_active":"2024-04-28T13:31:24.730384+0000","last_peered":"2024-04-28T13:31:24.730384+0000","last_clean":"2024-04-28T13:31:24.730384+0000","last_became_active":"2024-04-28T13:31:21.639921+0000","last_became_peered":"2024-04-28T13:31:21.639921+0000","last_unstale":"2024-04-28T13:31:24.730384+0000","last_undegraded":"2024-04-28T13:31:24.730384+0000","last_fullsized":"2024-04-28T13:31:24.730384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:33:40.307112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.230944+0000","last_change":"2024-04-28T13:31:20.633572+0000","last_active":"2024-04-28T13:31:24.230944+0000","last_peered":"2024-04-28T13:31:24.230944+0000","last_clean":"2024-04-28T13:31:24.230944+0000","last_became_active":"2024-04-28T13:31:20.630758+0000","last_became_peered":"2024-04-28T13:31:20.630758+0000","last_unstale":"2024-04-28T13:31:24.230944+0000","last_undegraded":"2024-04-28T13:31:24.230944+0000","last_fullsized":"2024-04-28T13:31:24.230944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:50.717520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.645842+0000","last_change":"2024-04-28T13:31:21.637178+0000","last_active":"2024-04-28T13:31:22.645842+0000","last_peered":"2024-04-28T13:31:22.645842+0000","last_clean":"2024-04-28T13:31:22.645842+0000","last_became_active":"2024-04-28T13:31:21.636898+0000","last_became_peered":"2024-04-28T13:31:21.636898+0000","last_unstale":"2024-04-28T13:31:22.645842+0000","last_undegraded":"2024-04-28T13:31:22.645842+0000","last_fullsized":"2024-04-28T13:31:22.645842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:09:17.042858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.639609+0000","last_change":"2024-04-28T13:31:20.624716+0000","last_active":"2024-04-28T13:31:22.639609+0000","last_peered":"2024-04-28T13:31:22.639609+0000","last_clean":"2024-04-28T13:31:22.639609+0000","last_became_active":"2024-04-28T13:31:20.623554+0000","last_became_peered":"2024-04-28T13:31:20.623554+0000","last_unstale":"2024-04-28T13:31:22.639609+0000","last_undegraded":"2024-04-28T13:31:22.639609+0000","last_fullsized":"2024-04-28T13:31:22.639609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:34:01.229713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730144+0000","last_change":"2024-04-28T13:31:21.646597+0000","last_active":"2024-04-28T13:31:24.730144+0000","last_peered":"2024-04-28T13:31:24.730144+0000","last_clean":"2024-04-28T13:31:24.730144+0000","last_became_active":"2024-04-28T13:31:21.646307+0000","last_became_peered":"2024-04-28T13:31:21.646307+0000","last_unstale":"2024-04-28T13:31:24.730144+0000","last_undegraded":"2024-04-28T13:31:24.730144+0000","last_fullsized":"2024-04-28T13:31:24.730144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:52:32.051589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.640004+0000","last_change":"2024-04-28T13:31:20.617782+0000","last_active":"2024-04-28T13:31:22.640004+0000","last_peered":"2024-04-28T13:31:22.640004+0000","last_clean":"2024-04-28T13:31:22.640004+0000","last_became_active":"2024-04-28T13:31:20.617213+0000","last_became_peered":"2024-04-28T13:31:20.617213+0000","last_unstale":"2024-04-28T13:31:22.640004+0000","last_undegraded":"2024-04-28T13:31:22.640004+0000","last_fullsized":"2024-04-28T13:31:22.640004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:00.123461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.731198+0000","last_change":"2024-04-28T13:31:21.637666+0000","last_active":"2024-04-28T13:31:24.731198+0000","last_peered":"2024-04-28T13:31:24.731198+0000","last_clean":"2024-04-28T13:31:24.731198+0000","last_became_active":"2024-04-28T13:31:21.637359+0000","last_became_peered":"2024-04-28T13:31:21.637359+0000","last_unstale":"2024-04-28T13:31:24.731198+0000","last_undegraded":"2024-04-28T13:31:24.731198+0000","last_fullsized":"2024-04-28T13:31:24.731198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:16:56.574292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231671+0000","last_change":"2024-04-28T13:31:20.632480+0000","last_active":"2024-04-28T13:31:24.231671+0000","last_peered":"2024-04-28T13:31:24.231671+0000","last_clean":"2024-04-28T13:31:24.231671+0000","last_became_active":"2024-04-28T13:31:20.632177+0000","last_became_peered":"2024-04-28T13:31:20.632177+0000","last_unstale":"2024-04-28T13:31:24.231671+0000","last_undegraded":"2024-04-28T13:31:24.231671+0000","last_fullsized":"2024-04-28T13:31:24.231671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:22.193491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732048+0000","last_change":"2024-04-28T13:31:21.645354+0000","last_active":"2024-04-28T13:31:24.732048+0000","last_peered":"2024-04-28T13:31:24.732048+0000","last_clean":"2024-04-28T13:31:24.732048+0000","last_became_active":"2024-04-28T13:31:21.644879+0000","last_became_peered":"2024-04-28T13:31:21.644879+0000","last_unstale":"2024-04-28T13:31:24.732048+0000","last_undegraded":"2024-04-28T13:31:24.732048+0000","last_fullsized":"2024-04-28T13:31:24.732048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:39:23.345225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.647195+0000","last_change":"2024-04-28T13:31:21.636961+0000","last_active":"2024-04-28T13:31:22.647195+0000","last_peered":"2024-04-28T13:31:22.647195+0000","last_clean":"2024-04-28T13:31:22.647195+0000","last_became_active":"2024-04-28T13:31:21.636609+0000","last_became_peered":"2024-04-28T13:31:21.636609+0000","last_unstale":"2024-04-28T13:31:22.647195+0000","last_undegraded":"2024-04-28T13:31:22.647195+0000","last_fullsized":"2024-04-28T13:31:22.647195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:28:41.694303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.230631+0000","last_change":"2024-04-28T13:31:20.632395+0000","last_active":"2024-04-28T13:31:24.230631+0000","last_peered":"2024-04-28T13:31:24.230631+0000","last_clean":"2024-04-28T13:31:24.230631+0000","last_became_active":"2024-04-28T13:31:20.632083+0000","last_became_peered":"2024-04-28T13:31:20.632083+0000","last_unstale":"2024-04-28T13:31:24.230631+0000","last_undegraded":"2024-04-28T13:31:24.230631+0000","last_fullsized":"2024-04-28T13:31:24.230631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:02:00.123461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:22.646732+0000","last_change":"2024-04-28T13:31:21.637466+0000","last_active":"2024-04-28T13:31:22.646732+0000","last_peered":"2024-04-28T13:31:22.646732+0000","last_clean":"2024-04-28T13:31:22.646732+0000","last_became_active":"2024-04-28T13:31:21.636945+0000","last_became_peered":"2024-04-28T13:31:21.636945+0000","last_unstale":"2024-04-28T13:31:22.646732+0000","last_undegraded":"2024-04-28T13:31:22.646732+0000","last_fullsized":"2024-04-28T13:31:22.646732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:41:07.707990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.231240+0000","last_change":"2024-04-28T13:31:20.636621+0000","last_active":"2024-04-28T13:31:24.231240+0000","last_peered":"2024-04-28T13:31:24.231240+0000","last_clean":"2024-04-28T13:31:24.231240+0000","last_became_active":"2024-04-28T13:31:20.636363+0000","last_became_peered":"2024-04-28T13:31:20.636363+0000","last_unstale":"2024-04-28T13:31:24.231240+0000","last_undegraded":"2024-04-28T13:31:24.231240+0000","last_fullsized":"2024-04-28T13:31:24.231240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:45:34.310382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730662+0000","last_change":"2024-04-28T13:31:21.636329+0000","last_active":"2024-04-28T13:31:24.730662+0000","last_peered":"2024-04-28T13:31:24.730662+0000","last_clean":"2024-04-28T13:31:24.730662+0000","last_became_active":"2024-04-28T13:31:21.635635+0000","last_became_peered":"2024-04-28T13:31:21.635635+0000","last_unstale":"2024-04-28T13:31:24.730662+0000","last_undegraded":"2024-04-28T13:31:24.730662+0000","last_fullsized":"2024-04-28T13:31:24.730662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:14:19.057730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.732045+0000","last_change":"2024-04-28T13:31:20.622197+0000","last_active":"2024-04-28T13:31:24.732045+0000","last_peered":"2024-04-28T13:31:24.732045+0000","last_clean":"2024-04-28T13:31:24.732045+0000","last_became_active":"2024-04-28T13:31:20.621973+0000","last_became_peered":"2024-04-28T13:31:20.621973+0000","last_unstale":"2024-04-28T13:31:24.732045+0000","last_undegraded":"2024-04-28T13:31:24.732045+0000","last_fullsized":"2024-04-28T13:31:24.732045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:50:14.806138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730760+0000","last_change":"2024-04-28T13:31:21.642200+0000","last_active":"2024-04-28T13:31:24.730760+0000","last_peered":"2024-04-28T13:31:24.730760+0000","last_clean":"2024-04-28T13:31:24.730760+0000","last_became_active":"2024-04-28T13:31:21.641925+0000","last_became_peered":"2024-04-28T13:31:21.641925+0000","last_unstale":"2024-04-28T13:31:24.730760+0000","last_undegraded":"2024-04-28T13:31:24.730760+0000","last_fullsized":"2024-04-28T13:31:24.730760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:20.578890+0000","last_clean_scrub_stamp":"2024-04-28T13:31:20.578890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:33.431077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-28T13:31:24.730908+0000","last_change":"2024-04-28T13:31:20.628724+0000","last_active":"2024-04-28T13:31:24.730908+0000","last_peered":"2024-04-28T13:31:24.730908+0000","last_clean":"2024-04-28T13:31:24.730908+0000","last_became_active":"2024-04-28T13:31:20.628363+0000","last_became_peered":"2024-04-28T13:31:20.628363+0000","last_unstale":"2024-04-28T13:31:24.730908+0000","last_undegraded":"2024-04-28T13:31:24.730908+0000","last_fullsized":"2024-04-28T13:31:24.730908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:19.564592+0000","last_clean_scrub_stamp":"2024-04-28T13:31:19.564592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:19:25.770351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T13:31:32.043 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-28T13:31:32.043 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-28T13:31:32.043 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-28T13:31:32.043 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-28T13:31:32.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.275+0000 7f0f79000640 1 Processor -- start 2024-04-28T13:31:32.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.276+0000 7f0f79000640 1 -- start start 2024-04-28T13:31:32.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.277+0000 7f0f79000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0f7415f390 0x7f0f74163870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:32.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.277+0000 7f0f79000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f740d1b90 0x7f0f740d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:32.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.277+0000 7f0f79000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0f740d3200 0x7f0f740d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:32.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.277+0000 7f0f79000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0f740d2530 con 0x7f0f740d3200 2024-04-28T13:31:32.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.277+0000 7f0f79000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f0f7415f770 con 0x7f0f7415f390 2024-04-28T13:31:32.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.277+0000 7f0f79000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0f74163db0 con 0x7f0f740d1b90 2024-04-28T13:31:32.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.277+0000 7f0f73e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0f740d3200 0x7f0f740d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:32.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.277+0000 7f0f73e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0f740d3200 0x7f0f740d3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56308/0 (socket says 172.21.15.196:56308) 2024-04-28T13:31:32.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.277+0000 7f0f73e00640 1 -- 172.21.15.196:0/2440296662 learned_addr learned my addr 172.21.15.196:0/2440296662 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:32.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.278+0000 7f0f72a00640 1 --2- 172.21.15.196:0/2440296662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f740d1b90 0x7f0f740d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:32.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.278+0000 7f0f73e00640 1 -- 172.21.15.196:0/2440296662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f740d1b90 msgr2=0x7f0f740d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:32.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.278+0000 7f0f73400640 1 --2- 172.21.15.196:0/2440296662 >> v2:172.21.15.203:3300/0 conn(0x7f0f7415f390 0x7f0f74163870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:32.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.278+0000 7f0f73e00640 1 --2- 172.21.15.196:0/2440296662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f740d1b90 0x7f0f740d1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.278+0000 7f0f73e00640 1 -- 172.21.15.196:0/2440296662 >> v2:172.21.15.203:3300/0 conn(0x7f0f7415f390 msgr2=0x7f0f74163870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:32.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.278+0000 7f0f73e00640 1 --2- 172.21.15.196:0/2440296662 >> v2:172.21.15.203:3300/0 conn(0x7f0f7415f390 0x7f0f74163870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.278+0000 7f0f73e00640 1 -- 172.21.15.196:0/2440296662 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f740d0070 con 0x7f0f740d3200 2024-04-28T13:31:32.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.279+0000 7f0f73e00640 1 --2- 172.21.15.196:0/2440296662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f740d3200 0x7f0f740d3670 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0f7408f810 tx=0x7f0f6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9609d3ca681b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:32.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.279+0000 7f0f72000640 1 -- 172.21.15.196:0/2440296662 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c037070 con 0x7f0f740d3200 2024-04-28T13:31:32.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.279+0000 7f0f72000640 1 -- 172.21.15.196:0/2440296662 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6c032440 con 0x7f0f740d3200 2024-04-28T13:31:32.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.279+0000 7f0f72000640 1 -- 172.21.15.196:0/2440296662 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c032810 con 0x7f0f740d3200 2024-04-28T13:31:32.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.279+0000 7f0f79000640 1 -- 172.21.15.196:0/2440296662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f740d3200 msgr2=0x7f0f740d3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:32.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.279+0000 7f0f79000640 1 --2- 172.21.15.196:0/2440296662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f740d3200 0x7f0f740d3670 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0f7408f810 tx=0x7f0f6c02a1e0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.280+0000 7f0f79000640 1 -- 172.21.15.196:0/2440296662 shutdown_connections 2024-04-28T13:31:32.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.280+0000 7f0f79000640 1 --2- 172.21.15.196:0/2440296662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f740d3200 0x7f0f740d3670 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.280+0000 7f0f79000640 1 --2- 172.21.15.196:0/2440296662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f740d1b90 0x7f0f740d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.280+0000 7f0f79000640 1 --2- 172.21.15.196:0/2440296662 >> v2:172.21.15.203:3300/0 conn(0x7f0f7415f390 0x7f0f74163870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.280+0000 7f0f79000640 1 -- 172.21.15.196:0/2440296662 >> 172.21.15.196:0/2440296662 conn(0x7f0f7409e4b0 msgr2=0x7f0f740d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:32.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.280+0000 7f0f79000640 1 -- 172.21.15.196:0/2440296662 shutdown_connections 2024-04-28T13:31:32.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.280+0000 7f0f79000640 1 -- 172.21.15.196:0/2440296662 wait complete. 2024-04-28T13:31:32.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.281+0000 7f0f79000640 1 Processor -- start 2024-04-28T13:31:32.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.281+0000 7f0f79000640 1 -- start start 2024-04-28T13:31:32.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.282+0000 7f0f79000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f740d1b90 0x7f0f74161d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:32.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.282+0000 7f0f79000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f740d3200 0x7f0f740cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:32.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.282+0000 7f0f79000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f7415f390 0x7f0f740cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:32.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.282+0000 7f0f79000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f740d0910 con 0x7f0f740d1b90 2024-04-28T13:31:32.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.282+0000 7f0f79000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0f740d0630 con 0x7f0f740d3200 2024-04-28T13:31:32.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.282+0000 7f0f79000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0f740d0bf0 con 0x7f0f7415f390 2024-04-28T13:31:32.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.282+0000 7f0f73400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f740d1b90 0x7f0f74161d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:32.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.283+0000 7f0f73400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f740d1b90 0x7f0f74161d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56318/0 (socket says 172.21.15.196:56318) 2024-04-28T13:31:32.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.283+0000 7f0f73400640 1 -- 172.21.15.196:0/4264732619 learned_addr learned my addr 172.21.15.196:0/4264732619 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:32.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.283+0000 7f0f73e00640 1 --2- 172.21.15.196:0/4264732619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f7415f390 0x7f0f740cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:32.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.283+0000 7f0f72a00640 1 --2- 172.21.15.196:0/4264732619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f740d3200 0x7f0f740cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:32.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.283+0000 7f0f73400640 1 -- 172.21.15.196:0/4264732619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f7415f390 msgr2=0x7f0f740cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:32.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.283+0000 7f0f73400640 1 --2- 172.21.15.196:0/4264732619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f7415f390 0x7f0f740cd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.283+0000 7f0f73400640 1 -- 172.21.15.196:0/4264732619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f740d3200 msgr2=0x7f0f740cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:32.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.283+0000 7f0f73400640 1 --2- 172.21.15.196:0/4264732619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f740d3200 0x7f0f740cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.283+0000 7f0f73400640 1 -- 172.21.15.196:0/4264732619 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f740cda40 con 0x7f0f740d1b90 2024-04-28T13:31:32.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.283+0000 7f0f73400640 1 --2- 172.21.15.196:0/4264732619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f740d1b90 0x7f0f74161d30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400c820 tx=0x7f0f6400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:32.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.283+0000 7f0f6b400640 1 -- 172.21.15.196:0/4264732619 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64015070 con 0x7f0f740d1b90 2024-04-28T13:31:32.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.283+0000 7f0f6b400640 1 -- 172.21.15.196:0/4264732619 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f64004020 con 0x7f0f740d1b90 2024-04-28T13:31:32.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.283+0000 7f0f6b400640 1 -- 172.21.15.196:0/4264732619 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f640043f0 con 0x7f0f740d1b90 2024-04-28T13:31:32.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.283+0000 7f0f79000640 1 -- 172.21.15.196:0/4264732619 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f740cdd20 con 0x7f0f740d1b90 2024-04-28T13:31:32.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.284+0000 7f0f79000640 1 -- 172.21.15.196:0/4264732619 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f740ce2e0 con 0x7f0f740d1b90 2024-04-28T13:31:32.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.284+0000 7f0f79000640 1 -- 172.21.15.196:0/4264732619 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f740a3190 con 0x7f0f740d1b90 2024-04-28T13:31:32.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.285+0000 7f0f6b400640 1 -- 172.21.15.196:0/4264732619 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f64007830 con 0x7f0f740d1b90 2024-04-28T13:31:32.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.286+0000 7f0f6b400640 1 --2- 172.21.15.196:0/4264732619 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0f5006c8f0 0x7f0f5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:32.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.286+0000 7f0f6b400640 1 -- 172.21.15.196:0/4264732619 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0f6408b8c0 con 0x7f0f740d1b90 2024-04-28T13:31:32.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.287+0000 7f0f72a00640 1 --2- 172.21.15.196:0/4264732619 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0f5006c8f0 0x7f0f5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:32.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.288+0000 7f0f72a00640 1 --2- 172.21.15.196:0/4264732619 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0f5006c8f0 0x7f0f5006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0f74162890 tx=0x7f0f5c008440 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:32.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.291+0000 7f0f6b400640 1 -- 172.21.15.196:0/4264732619 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0f64059ce0 con 0x7f0f740d1b90 2024-04-28T13:31:32.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.516+0000 7f0f79000640 1 -- 172.21.15.196:0/4264732619 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0f740507a0 con 0x7f0f740d1b90 2024-04-28T13:31:32.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.517+0000 7f0f6b400640 1 -- 172.21.15.196:0/4264732619 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0f640595e0 con 0x7f0f740d1b90 2024-04-28T13:31:32.518 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:32.518 INFO:teuthology.orchestra.run.smithi196.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-28T13:31:32.520 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.519+0000 7f0f79000640 1 -- 172.21.15.196:0/4264732619 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0f5006c8f0 msgr2=0x7f0f5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:32.520 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.519+0000 7f0f79000640 1 --2- 172.21.15.196:0/4264732619 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0f5006c8f0 0x7f0f5006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0f74162890 tx=0x7f0f5c008440 comp rx=0 tx=0).stop 2024-04-28T13:31:32.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.519+0000 7f0f79000640 1 -- 172.21.15.196:0/4264732619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f740d1b90 msgr2=0x7f0f74161d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:32.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.519+0000 7f0f79000640 1 --2- 172.21.15.196:0/4264732619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f740d1b90 0x7f0f74161d30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400c820 tx=0x7f0f6400cce0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.519+0000 7f0f79000640 1 -- 172.21.15.196:0/4264732619 shutdown_connections 2024-04-28T13:31:32.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.519+0000 7f0f79000640 1 --2- 172.21.15.196:0/4264732619 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f0f5006c8f0 0x7f0f5006edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.519+0000 7f0f79000640 1 --2- 172.21.15.196:0/4264732619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f7415f390 0x7f0f740cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.519+0000 7f0f79000640 1 --2- 172.21.15.196:0/4264732619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f740d3200 0x7f0f740cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.519+0000 7f0f79000640 1 --2- 172.21.15.196:0/4264732619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f740d1b90 0x7f0f74161d30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.519+0000 7f0f79000640 1 -- 172.21.15.196:0/4264732619 >> 172.21.15.196:0/4264732619 conn(0x7f0f7409e4b0 msgr2=0x7f0f740b6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:32.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.520+0000 7f0f79000640 1 -- 172.21.15.196:0/4264732619 shutdown_connections 2024-04-28T13:31:32.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.520+0000 7f0f79000640 1 -- 172.21.15.196:0/4264732619 wait complete. 2024-04-28T13:31:32.538 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-28T13:31:32.538 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:32.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.737+0000 7f3823e00640 1 Processor -- start 2024-04-28T13:31:32.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.737+0000 7f3823e00640 1 -- start start 2024-04-28T13:31:32.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.739+0000 7f3823e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f38240d16a0 0x7f3824160330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:32.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.739+0000 7f3823e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3824070eb0 con 0x7f382406c2a0 2024-04-28T13:31:32.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.739+0000 7f3823e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38240b6d80 con 0x7f38240b6850 2024-04-28T13:31:32.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.739+0000 7f3823e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f382415fea0 con 0x7f38240d16a0 2024-04-28T13:31:32.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.739+0000 7f3822a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f38240d16a0 0x7f3824160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:32.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.739+0000 7f3822a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f38240d16a0 0x7f3824160330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56330/0 (socket says 172.21.15.196:56330) 2024-04-28T13:31:32.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.739+0000 7f3822a00640 1 -- 172.21.15.196:0/2302883152 learned_addr learned my addr 172.21.15.196:0/2302883152 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:32.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.739+0000 7f3823400640 1 --1- 172.21.15.196:0/2302883152 >> v1:172.21.15.196:6789/0 conn(0x7f382406c2a0 0x7f3824159010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:59176/0 (socket says 172.21.15.196:59176) 2024-04-28T13:31:32.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.739+0000 7f3822a00640 1 -- 172.21.15.196:0/2302883152 >> v1:172.21.15.203:6789/0 conn(0x7f38240b6850 legacy=0x7f38240b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:32.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.739+0000 7f3822a00640 1 -- 172.21.15.196:0/2302883152 >> v1:172.21.15.196:6789/0 conn(0x7f382406c2a0 legacy=0x7f3824159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:32.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.739+0000 7f3822a00640 1 -- 172.21.15.196:0/2302883152 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38240b7060 con 0x7f38240d16a0 2024-04-28T13:31:32.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.739+0000 7f3821600640 1 -- 172.21.15.196:0/2302883152 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2954061376 0 0) 0x7f3824070eb0 con 0x7f382406c2a0 2024-04-28T13:31:32.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.740+0000 7f3822a00640 1 --2- 172.21.15.196:0/2302883152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d16a0 0x7f3824160330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3814009870 tx=0x7f381402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc913cf01440da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:32.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.741+0000 7f3821600640 1 -- 172.21.15.196:0/2302883152 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814037070 con 0x7f38240d16a0 2024-04-28T13:31:32.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.741+0000 7f3821600640 1 -- 172.21.15.196:0/2302883152 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3814032440 con 0x7f38240d16a0 2024-04-28T13:31:32.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.741+0000 7f3821600640 1 -- 172.21.15.196:0/2302883152 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814032810 con 0x7f38240d16a0 2024-04-28T13:31:32.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.741+0000 7f3823e00640 1 -- 172.21.15.196:0/2302883152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d16a0 msgr2=0x7f3824160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:32.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.741+0000 7f3823e00640 1 --2- 172.21.15.196:0/2302883152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d16a0 0x7f3824160330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3814009870 tx=0x7f381402a2a0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.741+0000 7f3823e00640 1 -- 172.21.15.196:0/2302883152 shutdown_connections 2024-04-28T13:31:32.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.741+0000 7f3823e00640 1 --2- 172.21.15.196:0/2302883152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d16a0 0x7f3824160330 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.741+0000 7f3823e00640 1 -- 172.21.15.196:0/2302883152 >> 172.21.15.196:0/2302883152 conn(0x7f38240b3aa0 msgr2=0x7f38240b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:32.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.741+0000 7f3823e00640 1 -- 172.21.15.196:0/2302883152 shutdown_connections 2024-04-28T13:31:32.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.741+0000 7f3823e00640 1 -- 172.21.15.196:0/2302883152 wait complete. 2024-04-28T13:31:32.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.743+0000 7f3823e00640 1 Processor -- start 2024-04-28T13:31:32.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.743+0000 7f3823e00640 1 -- start start 2024-04-28T13:31:32.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.744+0000 7f3823e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f382406c2a0 0x7f38241468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:32.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.744+0000 7f3823e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38240b6850 0x7f3824161c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:32.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.744+0000 7f3823e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d16a0 0x7f3824160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:32.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.744+0000 7f3823e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38240d1a80 con 0x7f38240d16a0 2024-04-28T13:31:32.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.744+0000 7f3823e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f382415fb30 con 0x7f382406c2a0 2024-04-28T13:31:32.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.744+0000 7f3823e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f382406c750 con 0x7f38240b6850 2024-04-28T13:31:32.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.744+0000 7f3823400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d16a0 0x7f3824160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:32.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.744+0000 7f3823400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d16a0 0x7f3824160330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56342/0 (socket says 172.21.15.196:56342) 2024-04-28T13:31:32.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.744+0000 7f3823400640 1 -- 172.21.15.196:0/1388125291 learned_addr learned my addr 172.21.15.196:0/1388125291 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:32.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.744+0000 7f3822a00640 1 --2- 172.21.15.196:0/1388125291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f382406c2a0 0x7f38241468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:32.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.744+0000 7f3823400640 1 -- 172.21.15.196:0/1388125291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38240b6850 msgr2=0x7f3824161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:32.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.744+0000 7f3823400640 1 --2- 172.21.15.196:0/1388125291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38240b6850 0x7f3824161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.744+0000 7f3823400640 1 -- 172.21.15.196:0/1388125291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f382406c2a0 msgr2=0x7f38241468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:32.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.744+0000 7f3823400640 1 --2- 172.21.15.196:0/1388125291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f382406c2a0 0x7f38241468d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.744+0000 7f3823400640 1 -- 172.21.15.196:0/1388125291 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3824160b80 con 0x7f38240d16a0 2024-04-28T13:31:32.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.745+0000 7f3822a00640 1 --2- 172.21.15.196:0/1388125291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f382406c2a0 0x7f38241468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:32.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.745+0000 7f3823400640 1 --2- 172.21.15.196:0/1388125291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d16a0 0x7f3824160330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f381c00c7d0 tx=0x7f381c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:32.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.745+0000 7f381b400640 1 -- 172.21.15.196:0/1388125291 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c018070 con 0x7f38240d16a0 2024-04-28T13:31:32.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.745+0000 7f381b400640 1 -- 172.21.15.196:0/1388125291 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f381c00a640 con 0x7f38240d16a0 2024-04-28T13:31:32.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.745+0000 7f381b400640 1 -- 172.21.15.196:0/1388125291 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c00aa70 con 0x7f38240d16a0 2024-04-28T13:31:32.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.745+0000 7f3823e00640 1 -- 172.21.15.196:0/1388125291 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3824160e60 con 0x7f38240d16a0 2024-04-28T13:31:32.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.745+0000 7f3823e00640 1 -- 172.21.15.196:0/1388125291 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3824161420 con 0x7f38240d16a0 2024-04-28T13:31:32.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.746+0000 7f3823e00640 1 -- 172.21.15.196:0/1388125291 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f382406a290 con 0x7f38240d16a0 2024-04-28T13:31:32.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.747+0000 7f381b400640 1 -- 172.21.15.196:0/1388125291 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f381c00ac10 con 0x7f38240d16a0 2024-04-28T13:31:32.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.748+0000 7f381b400640 1 --2- 172.21.15.196:0/1388125291 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f380406c8f0 0x7f380406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:32.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.748+0000 7f381b400640 1 -- 172.21.15.196:0/1388125291 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f381c08b310 con 0x7f38240d16a0 2024-04-28T13:31:32.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.749+0000 7f3822a00640 1 --2- 172.21.15.196:0/1388125291 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f380406c8f0 0x7f380406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:32.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.750+0000 7f3822a00640 1 --2- 172.21.15.196:0/1388125291 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f380406c8f0 0x7f380406edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3814034000 tx=0x7f381402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:32.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.752+0000 7f381b400640 1 -- 172.21.15.196:0/1388125291 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f381c059610 con 0x7f38240d16a0 2024-04-28T13:31:32.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.938+0000 7f3823e00640 1 -- 172.21.15.196:0/1388125291 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f382406cea0 con 0x7f38240d16a0 2024-04-28T13:31:32.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.941+0000 7f381b400640 1 -- 172.21.15.196:0/1388125291 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f381c058f10 con 0x7f38240d16a0 2024-04-28T13:31:32.942 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:32.943 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":20,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:22.624564+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:32.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.944+0000 7f3823e00640 1 -- 172.21.15.196:0/1388125291 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f380406c8f0 msgr2=0x7f380406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:32.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.944+0000 7f3823e00640 1 --2- 172.21.15.196:0/1388125291 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f380406c8f0 0x7f380406edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3814034000 tx=0x7f381402e000 comp rx=0 tx=0).stop 2024-04-28T13:31:32.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.944+0000 7f3823e00640 1 -- 172.21.15.196:0/1388125291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d16a0 msgr2=0x7f3824160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:32.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.944+0000 7f3823e00640 1 --2- 172.21.15.196:0/1388125291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d16a0 0x7f3824160330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f381c00c7d0 tx=0x7f381c00cc90 comp rx=0 tx=0).stop 2024-04-28T13:31:32.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.944+0000 7f3823e00640 1 -- 172.21.15.196:0/1388125291 shutdown_connections 2024-04-28T13:31:32.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.944+0000 7f3823e00640 1 --2- 172.21.15.196:0/1388125291 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f380406c8f0 0x7f380406edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.944+0000 7f3823e00640 1 --2- 172.21.15.196:0/1388125291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d16a0 0x7f3824160330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.944+0000 7f3823e00640 1 --2- 172.21.15.196:0/1388125291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38240b6850 0x7f3824161c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.944+0000 7f3823e00640 1 --2- 172.21.15.196:0/1388125291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f382406c2a0 0x7f38241468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:32.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.944+0000 7f3823e00640 1 -- 172.21.15.196:0/1388125291 >> 172.21.15.196:0/1388125291 conn(0x7f38240b3aa0 msgr2=0x7f382415ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:32.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.944+0000 7f3823e00640 1 -- 172.21.15.196:0/1388125291 shutdown_connections 2024-04-28T13:31:32.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:32.944+0000 7f3823e00640 1 -- 172.21.15.196:0/1388125291 wait complete. 2024-04-28T13:31:32.964 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:31:19.280975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:31:20.037066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:31:32.965 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:31:33.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.161+0000 7f622de00640 1 Processor -- start 2024-04-28T13:31:33.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.161+0000 7f622de00640 1 -- start start 2024-04-28T13:31:33.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.163+0000 7f622de00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f62280d10a0 0x7f62280b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:33.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.163+0000 7f622de00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f622806e8f0 0x7f62280b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:33.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.163+0000 7f622de00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62280b61d0 con 0x7f62280d1480 2024-04-28T13:31:33.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.163+0000 7f622de00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f622815e310 con 0x7f62280d10a0 2024-04-28T13:31:33.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.163+0000 7f622de00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f622815d700 con 0x7f622806e8f0 2024-04-28T13:31:33.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.163+0000 7f622ca00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f62280d10a0 0x7f62280b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:33.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.163+0000 7f622ca00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f62280d10a0 0x7f62280b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56354/0 (socket says 172.21.15.196:56354) 2024-04-28T13:31:33.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.163+0000 7f622ca00640 1 -- 172.21.15.196:0/368777073 learned_addr learned my addr 172.21.15.196:0/368777073 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:33.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.163+0000 7f622d400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f62280d1480 0x7f6228160c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:59188/0 (socket says 172.21.15.196:59188) 2024-04-28T13:31:33.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.163+0000 7f6227e00640 1 --2- 172.21.15.196:0/368777073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f622806e8f0 0x7f62280b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:33.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.163+0000 7f622ca00640 1 -- 172.21.15.196:0/368777073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f622806e8f0 msgr2=0x7f62280b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:33.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.163+0000 7f622ca00640 1 --2- 172.21.15.196:0/368777073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f622806e8f0 0x7f62280b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.163+0000 7f622ca00640 1 -- 172.21.15.196:0/368777073 >> v1:172.21.15.196:6789/0 conn(0x7f62280d1480 legacy=0x7f6228160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:33.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.164+0000 7f622ca00640 1 -- 172.21.15.196:0/368777073 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622815d9e0 con 0x7f62280d10a0 2024-04-28T13:31:33.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.164+0000 7f6227400640 1 -- 172.21.15.196:0/368777073 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2450368566 0 0) 0x7f6220001490 con 0x7f62280d1480 2024-04-28T13:31:33.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.165+0000 7f622ca00640 1 --2- 172.21.15.196:0/368777073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62280d10a0 0x7f62280b3c90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f621c009870 tx=0x7f621c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342857ecb8a86a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:33.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.165+0000 7f6227400640 1 -- 172.21.15.196:0/368777073 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c037070 con 0x7f62280d10a0 2024-04-28T13:31:33.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.165+0000 7f6227400640 1 -- 172.21.15.196:0/368777073 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f621c032440 con 0x7f62280d10a0 2024-04-28T13:31:33.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.165+0000 7f6227400640 1 -- 172.21.15.196:0/368777073 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c032810 con 0x7f62280d10a0 2024-04-28T13:31:33.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.165+0000 7f622de00640 1 -- 172.21.15.196:0/368777073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62280d10a0 msgr2=0x7f62280b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:33.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.165+0000 7f622de00640 1 --2- 172.21.15.196:0/368777073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62280d10a0 0x7f62280b3c90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f621c009870 tx=0x7f621c02a250 comp rx=0 tx=0).stop 2024-04-28T13:31:33.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.166+0000 7f622de00640 1 -- 172.21.15.196:0/368777073 shutdown_connections 2024-04-28T13:31:33.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.166+0000 7f622de00640 1 --2- 172.21.15.196:0/368777073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f622806e8f0 0x7f62280b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.166+0000 7f622de00640 1 --2- 172.21.15.196:0/368777073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62280d10a0 0x7f62280b3c90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.166+0000 7f622de00640 1 -- 172.21.15.196:0/368777073 >> 172.21.15.196:0/368777073 conn(0x7f622806c6c0 msgr2=0x7f622806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:33.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.166+0000 7f622de00640 1 -- 172.21.15.196:0/368777073 shutdown_connections 2024-04-28T13:31:33.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.166+0000 7f622de00640 1 -- 172.21.15.196:0/368777073 wait complete. 2024-04-28T13:31:33.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.167+0000 7f622de00640 1 Processor -- start 2024-04-28T13:31:33.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.167+0000 7f622de00640 1 -- start start 2024-04-28T13:31:33.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.168+0000 7f622de00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f622806e8f0 0x7f62281497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:33.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.168+0000 7f622de00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62280d10a0 0x7f62280cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:33.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.168+0000 7f622de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62280d1480 0x7f62280cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:33.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.168+0000 7f622de00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62280d18f0 con 0x7f622806e8f0 2024-04-28T13:31:33.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.168+0000 7f622de00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f622815dfa0 con 0x7f62280d1480 2024-04-28T13:31:33.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.168+0000 7f622de00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f62281613a0 con 0x7f62280d10a0 2024-04-28T13:31:33.171 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.169+0000 7f622ca00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f622806e8f0 0x7f62281497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:33.171 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.169+0000 7f622ca00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f622806e8f0 0x7f62281497b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56358/0 (socket says 172.21.15.196:56358) 2024-04-28T13:31:33.171 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.169+0000 7f622ca00640 1 -- 172.21.15.196:0/3330952534 learned_addr learned my addr 172.21.15.196:0/3330952534 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:33.171 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.169+0000 7f6227e00640 1 --2- 172.21.15.196:0/3330952534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62280d10a0 0x7f62280cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:33.171 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.169+0000 7f622d400640 1 --2- 172.21.15.196:0/3330952534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62280d1480 0x7f62280cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:33.171 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.169+0000 7f622ca00640 1 -- 172.21.15.196:0/3330952534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62280d10a0 msgr2=0x7f62280cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:33.171 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.169+0000 7f622ca00640 1 --2- 172.21.15.196:0/3330952534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62280d10a0 0x7f62280cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.169+0000 7f622ca00640 1 -- 172.21.15.196:0/3330952534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62280d1480 msgr2=0x7f62280cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:33.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.169+0000 7f622ca00640 1 --2- 172.21.15.196:0/3330952534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62280d1480 0x7f62280cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.169+0000 7f622ca00640 1 -- 172.21.15.196:0/3330952534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622815a030 con 0x7f622806e8f0 2024-04-28T13:31:33.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.169+0000 7f622d400640 1 --2- 172.21.15.196:0/3330952534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62280d1480 0x7f62280cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:33.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.169+0000 7f622ca00640 1 --2- 172.21.15.196:0/3330952534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f622806e8f0 0x7f62281497b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f621c0090b0 tx=0x7f621c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:33.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.170+0000 7f6224c00640 1 -- 172.21.15.196:0/3330952534 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c037040 con 0x7f622806e8f0 2024-04-28T13:31:33.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.170+0000 7f6224c00640 1 -- 172.21.15.196:0/3330952534 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f621c004230 con 0x7f622806e8f0 2024-04-28T13:31:33.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.170+0000 7f6224c00640 1 -- 172.21.15.196:0/3330952534 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c003610 con 0x7f622806e8f0 2024-04-28T13:31:33.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.170+0000 7f622de00640 1 -- 172.21.15.196:0/3330952534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622815a310 con 0x7f622806e8f0 2024-04-28T13:31:33.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.170+0000 7f622de00640 1 -- 172.21.15.196:0/3330952534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f622815a8d0 con 0x7f622806e8f0 2024-04-28T13:31:33.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.171+0000 7f622de00640 1 -- 172.21.15.196:0/3330952534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6228050730 con 0x7f622806e8f0 2024-04-28T13:31:33.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.172+0000 7f6224c00640 1 -- 172.21.15.196:0/3330952534 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f621c0037b0 con 0x7f622806e8f0 2024-04-28T13:31:33.174 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.173+0000 7f6224c00640 1 --2- 172.21.15.196:0/3330952534 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f620806c8f0 0x7f620806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:33.174 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.173+0000 7f6224c00640 1 -- 172.21.15.196:0/3330952534 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f621c0ac940 con 0x7f622806e8f0 2024-04-28T13:31:33.174 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.174+0000 7f6227e00640 1 --2- 172.21.15.196:0/3330952534 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f620806c8f0 0x7f620806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:33.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.175+0000 7f6227e00640 1 --2- 172.21.15.196:0/3330952534 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f620806c8f0 0x7f620806edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f622809dd30 tx=0x7f6214008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:33.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.178+0000 7f6224c00640 1 -- 172.21.15.196:0/3330952534 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f621c07ad60 con 0x7f622806e8f0 2024-04-28T13:31:33.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.371+0000 7f622de00640 1 -- 172.21.15.196:0/3330952534 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f622806f700 con 0x7f622806e8f0 2024-04-28T13:31:33.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.372+0000 7f6224c00640 1 -- 172.21.15.196:0/3330952534 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f621c07a660 con 0x7f622806e8f0 2024-04-28T13:31:33.373 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:31:33.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.374+0000 7f622de00640 1 -- 172.21.15.196:0/3330952534 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f620806c8f0 msgr2=0x7f620806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:33.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.374+0000 7f622de00640 1 --2- 172.21.15.196:0/3330952534 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f620806c8f0 0x7f620806edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f622809dd30 tx=0x7f6214008040 comp rx=0 tx=0).stop 2024-04-28T13:31:33.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.374+0000 7f622de00640 1 -- 172.21.15.196:0/3330952534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f622806e8f0 msgr2=0x7f62281497b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:33.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.374+0000 7f622de00640 1 --2- 172.21.15.196:0/3330952534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f622806e8f0 0x7f62281497b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f621c0090b0 tx=0x7f621c0046c0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.375+0000 7f622de00640 1 -- 172.21.15.196:0/3330952534 shutdown_connections 2024-04-28T13:31:33.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.375+0000 7f622de00640 1 --2- 172.21.15.196:0/3330952534 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f620806c8f0 0x7f620806edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.375+0000 7f622de00640 1 --2- 172.21.15.196:0/3330952534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62280d1480 0x7f62280cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.375+0000 7f622de00640 1 --2- 172.21.15.196:0/3330952534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62280d10a0 0x7f62280cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.375+0000 7f622de00640 1 --2- 172.21.15.196:0/3330952534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f622806e8f0 0x7f62281497b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.375+0000 7f622de00640 1 -- 172.21.15.196:0/3330952534 >> 172.21.15.196:0/3330952534 conn(0x7f622806c6c0 msgr2=0x7f6228160c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:33.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.375+0000 7f622de00640 1 -- 172.21.15.196:0/3330952534 shutdown_connections 2024-04-28T13:31:33.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.375+0000 7f622de00640 1 -- 172.21.15.196:0/3330952534 wait complete. 2024-04-28T13:31:33.394 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:31:33.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.583+0000 7fee57e00640 1 Processor -- start 2024-04-28T13:31:33.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.583+0000 7fee57e00640 1 -- start start 2024-04-28T13:31:33.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.585+0000 7fee57e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fee580d3830 0x7fee580d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:33.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.585+0000 7fee57e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee580d0600 0x7fee580d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:33.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.585+0000 7fee57e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee580cfb80 con 0x7fee580d3c10 2024-04-28T13:31:33.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.585+0000 7fee57e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fee580d2ee0 con 0x7fee580d3830 2024-04-28T13:31:33.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.585+0000 7fee57e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fee580d31c0 con 0x7fee580d0600 2024-04-28T13:31:33.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.585+0000 7fee4fe00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fee580d3c10 0x7fee580b7e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:59200/0 (socket says 172.21.15.196:59200) 2024-04-28T13:31:33.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.585+0000 7fee4fe00640 1 -- 172.21.15.196:0/1886555946 learned_addr learned my addr 172.21.15.196:0/1886555946 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:33.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56a00640 1 --2- 172.21.15.196:0/1886555946 >> v2:172.21.15.203:3300/0 conn(0x7fee580d3830 0x7fee580d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:33.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee57400640 1 --2- 172.21.15.196:0/1886555946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee580d0600 0x7fee580d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:33.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 -- 172.21.15.196:0/1886555946 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4082978474 0 0) 0x7fee40001110 con 0x7fee580d3c10 2024-04-28T13:31:33.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 -- 172.21.15.196:0/1886555946 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee3c003520 con 0x7fee580d3c10 2024-04-28T13:31:33.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 -- 172.21.15.196:0/1886555946 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2543442909 0 0) 0x7fee3c003520 con 0x7fee580d3c10 2024-04-28T13:31:33.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 -- 172.21.15.196:0/1886555946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee580d0600 msgr2=0x7fee580d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:33.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 --2- 172.21.15.196:0/1886555946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee580d0600 0x7fee580d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 -- 172.21.15.196:0/1886555946 >> v2:172.21.15.203:3300/0 conn(0x7fee580d3830 msgr2=0x7fee580d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:33.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 --2- 172.21.15.196:0/1886555946 >> v2:172.21.15.203:3300/0 conn(0x7fee580d3830 0x7fee580d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 -- 172.21.15.196:0/1886555946 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee580cfe00 con 0x7fee580d3c10 2024-04-28T13:31:33.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 -- 172.21.15.196:0/1886555946 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fee40001840 con 0x7fee580d3c10 2024-04-28T13:31:33.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 -- 172.21.15.196:0/1886555946 >> v1:172.21.15.196:6789/0 conn(0x7fee580d3c10 legacy=0x7fee580b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:33.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 --2- 172.21.15.196:0/1886555946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee3c0039b0 0x7fee3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:33.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 --2- 172.21.15.196:0/1886555946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee3c004800 0x7fee3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:33.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 --2- 172.21.15.196:0/1886555946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee580d0600 0x7fee3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:33.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 -- 172.21.15.196:0/1886555946 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee580d2ee0 con 0x7fee3c0039b0 2024-04-28T13:31:33.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 -- 172.21.15.196:0/1886555946 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fee580d31c0 con 0x7fee3c004800 2024-04-28T13:31:33.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56000640 1 -- 172.21.15.196:0/1886555946 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fee3c00b9c0 con 0x7fee580d0600 2024-04-28T13:31:33.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.586+0000 7fee56a00640 1 --2- 172.21.15.196:0/1886555946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee3c0039b0 0x7fee3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:33.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.587+0000 7fee4fe00640 1 --2- 172.21.15.196:0/1886555946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee3c004800 0x7fee3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:33.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.587+0000 7fee56a00640 1 -- 172.21.15.196:0/1886555946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee580d0600 msgr2=0x7fee3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:33.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.587+0000 7fee56a00640 1 --2- 172.21.15.196:0/1886555946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee580d0600 0x7fee3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.587+0000 7fee56a00640 1 -- 172.21.15.196:0/1886555946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee3c004800 msgr2=0x7fee3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:33.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.587+0000 7fee56a00640 1 --2- 172.21.15.196:0/1886555946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee3c004800 0x7fee3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.587+0000 7fee56a00640 1 -- 172.21.15.196:0/1886555946 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3c003520 con 0x7fee3c0039b0 2024-04-28T13:31:33.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.587+0000 7fee4fe00640 1 --2- 172.21.15.196:0/1886555946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee3c004800 0x7fee3c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:33.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.588+0000 7fee56a00640 1 --2- 172.21.15.196:0/1886555946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee3c0039b0 0x7fee3c003dd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fee48009ac0 tx=0x7fee3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195b0a1dea7ea255 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:33.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.588+0000 7fee56000640 1 -- 172.21.15.196:0/1886555946 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48036070 con 0x7fee3c0039b0 2024-04-28T13:31:33.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.588+0000 7fee56000640 1 -- 172.21.15.196:0/1886555946 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee48031400 con 0x7fee3c0039b0 2024-04-28T13:31:33.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.588+0000 7fee56000640 1 -- 172.21.15.196:0/1886555946 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48009690 con 0x7fee3c0039b0 2024-04-28T13:31:33.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.589+0000 7fee57e00640 1 -- 172.21.15.196:0/1886555946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee3c0039b0 msgr2=0x7fee3c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:33.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.589+0000 7fee57e00640 1 --2- 172.21.15.196:0/1886555946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee3c0039b0 0x7fee3c003dd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fee48009ac0 tx=0x7fee3c005830 comp rx=0 tx=0).stop 2024-04-28T13:31:33.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.589+0000 7fee57400640 1 -- 172.21.15.196:0/1886555946 reap_dead start 2024-04-28T13:31:33.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.589+0000 7fee57e00640 1 -- 172.21.15.196:0/1886555946 shutdown_connections 2024-04-28T13:31:33.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.589+0000 7fee57e00640 1 -- 172.21.15.196:0/1886555946 >> 172.21.15.196:0/1886555946 conn(0x7fee580d4fe0 msgr2=0x7fee580d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:33.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.589+0000 7fee57e00640 1 -- 172.21.15.196:0/1886555946 shutdown_connections 2024-04-28T13:31:33.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.589+0000 7fee57e00640 1 -- 172.21.15.196:0/1886555946 wait complete. 2024-04-28T13:31:33.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.590+0000 7fee57e00640 1 Processor -- start 2024-04-28T13:31:33.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.591+0000 7fee57e00640 1 -- start start 2024-04-28T13:31:33.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.591+0000 7fee57e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee3c0039b0 0x7fee581e95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:33.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.591+0000 7fee57e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee581e7070 0x7fee581e74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:33.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.591+0000 7fee57e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee580c8b10 0x7fee580c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:33.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.591+0000 7fee57e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee580d2940 con 0x7fee3c0039b0 2024-04-28T13:31:33.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.591+0000 7fee57e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fee580b8540 con 0x7fee581e7070 2024-04-28T13:31:33.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.591+0000 7fee57e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fee580d5d90 con 0x7fee580c8b10 2024-04-28T13:31:33.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.592+0000 7fee56a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee3c0039b0 0x7fee581e95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:33.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.592+0000 7fee56a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee3c0039b0 0x7fee581e95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56384/0 (socket says 172.21.15.196:56384) 2024-04-28T13:31:33.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.592+0000 7fee56a00640 1 -- 172.21.15.196:0/757283438 learned_addr learned my addr 172.21.15.196:0/757283438 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:33.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.592+0000 7fee56a00640 1 -- 172.21.15.196:0/757283438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee580c8b10 msgr2=0x7fee580c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:33.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.592+0000 7fee56a00640 1 --2- 172.21.15.196:0/757283438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee580c8b10 0x7fee580c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.592+0000 7fee56a00640 1 -- 172.21.15.196:0/757283438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee581e7070 msgr2=0x7fee581e74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:33.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.592+0000 7fee4fe00640 1 --2- 172.21.15.196:0/757283438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee581e7070 0x7fee581e74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:33.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.592+0000 7fee56a00640 1 --2- 172.21.15.196:0/757283438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee581e7070 0x7fee581e74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.592+0000 7fee56a00640 1 -- 172.21.15.196:0/757283438 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee581e7c50 con 0x7fee3c0039b0 2024-04-28T13:31:33.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.592+0000 7fee56a00640 1 --2- 172.21.15.196:0/757283438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee3c0039b0 0x7fee581e95a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fee48034040 tx=0x7fee48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:33.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.593+0000 7fee4ea00640 1 -- 172.21.15.196:0/757283438 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48040020 con 0x7fee3c0039b0 2024-04-28T13:31:33.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.593+0000 7fee4ea00640 1 -- 172.21.15.196:0/757283438 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee480038d0 con 0x7fee3c0039b0 2024-04-28T13:31:33.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.593+0000 7fee4ea00640 1 -- 172.21.15.196:0/757283438 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48036040 con 0x7fee3c0039b0 2024-04-28T13:31:33.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.593+0000 7fee57e00640 1 -- 172.21.15.196:0/757283438 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee581e7f30 con 0x7fee3c0039b0 2024-04-28T13:31:33.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.593+0000 7fee57e00640 1 -- 172.21.15.196:0/757283438 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee581e84f0 con 0x7fee3c0039b0 2024-04-28T13:31:33.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.594+0000 7fee57e00640 1 -- 172.21.15.196:0/757283438 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee58050730 con 0x7fee3c0039b0 2024-04-28T13:31:33.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.595+0000 7fee4ea00640 1 -- 172.21.15.196:0/757283438 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee48043020 con 0x7fee3c0039b0 2024-04-28T13:31:33.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.595+0000 7fee4ea00640 1 --2- 172.21.15.196:0/757283438 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fee3406c8f0 0x7fee3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:33.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.596+0000 7fee4ea00640 1 -- 172.21.15.196:0/757283438 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fee480abbc0 con 0x7fee3c0039b0 2024-04-28T13:31:33.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.596+0000 7fee4fe00640 1 --2- 172.21.15.196:0/757283438 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fee3406c8f0 0x7fee3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:33.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.597+0000 7fee4fe00640 1 --2- 172.21.15.196:0/757283438 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fee3406c8f0 0x7fee3406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fee400019b0 tx=0x7fee40009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:33.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.600+0000 7fee4ea00640 1 -- 172.21.15.196:0/757283438 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee4804a070 con 0x7fee3c0039b0 2024-04-28T13:31:33.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.802+0000 7fee57e00640 1 -- 172.21.15.196:0/757283438 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fee580d17d0 con 0x7fee3c0039b0 2024-04-28T13:31:33.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.803+0000 7fee4ea00640 1 -- 172.21.15.196:0/757283438 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fee48079f60 con 0x7fee3c0039b0 2024-04-28T13:31:33.804 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:31:33.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.806+0000 7fee57e00640 1 -- 172.21.15.196:0/757283438 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fee3406c8f0 msgr2=0x7fee3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:33.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.806+0000 7fee57e00640 1 --2- 172.21.15.196:0/757283438 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fee3406c8f0 0x7fee3406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fee400019b0 tx=0x7fee40009040 comp rx=0 tx=0).stop 2024-04-28T13:31:33.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.806+0000 7fee57e00640 1 -- 172.21.15.196:0/757283438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee3c0039b0 msgr2=0x7fee581e95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:33.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.806+0000 7fee57e00640 1 --2- 172.21.15.196:0/757283438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee3c0039b0 0x7fee581e95a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fee48034040 tx=0x7fee48004730 comp rx=0 tx=0).stop 2024-04-28T13:31:33.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.806+0000 7fee57e00640 1 -- 172.21.15.196:0/757283438 shutdown_connections 2024-04-28T13:31:33.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.806+0000 7fee57e00640 1 --2- 172.21.15.196:0/757283438 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fee3406c8f0 0x7fee3406edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.806+0000 7fee57e00640 1 --2- 172.21.15.196:0/757283438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee580c8b10 0x7fee580c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.806+0000 7fee57e00640 1 --2- 172.21.15.196:0/757283438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee581e7070 0x7fee581e74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.806+0000 7fee57e00640 1 --2- 172.21.15.196:0/757283438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee3c0039b0 0x7fee581e95a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:33.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.806+0000 7fee57e00640 1 -- 172.21.15.196:0/757283438 >> 172.21.15.196:0/757283438 conn(0x7fee580d4fe0 msgr2=0x7fee580ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:33.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.806+0000 7fee57e00640 1 -- 172.21.15.196:0/757283438 shutdown_connections 2024-04-28T13:31:33.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:33.806+0000 7fee57e00640 1 -- 172.21.15.196:0/757283438 wait complete. 2024-04-28T13:31:33.824 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:31:34.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.027+0000 7faa2f600640 1 Processor -- start 2024-04-28T13:31:34.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.028+0000 7faa2f600640 1 -- start start 2024-04-28T13:31:34.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.029+0000 7faa2f600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7faa280d3820 0x7faa280b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.029+0000 7faa2f600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa280b60b0 0x7faa280d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.029+0000 7faa2f600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa280d3c90 con 0x7faa2815f680 2024-04-28T13:31:34.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.029+0000 7faa2f600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7faa2815fa60 con 0x7faa280d3820 2024-04-28T13:31:34.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.029+0000 7faa2f600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7faa280d3f50 con 0x7faa280b60b0 2024-04-28T13:31:34.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.029+0000 7faa2cc00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7faa2815f680 0x7faa28163aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:59214/0 (socket says 172.21.15.196:59214) 2024-04-28T13:31:34.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.029+0000 7faa2cc00640 1 -- 172.21.15.196:0/1987648784 learned_addr learned my addr 172.21.15.196:0/1987648784 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:34.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27e00640 1 --2- 172.21.15.196:0/1987648784 >> v2:172.21.15.203:3300/0 conn(0x7faa280d3820 0x7faa280b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa2d600640 1 --2- 172.21.15.196:0/1987648784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa280b60b0 0x7faa280d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 -- 172.21.15.196:0/1987648784 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2242186909 0 0) 0x7faa280d3c90 con 0x7faa2815f680 2024-04-28T13:31:34.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 -- 172.21.15.196:0/1987648784 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa10003520 con 0x7faa2815f680 2024-04-28T13:31:34.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 -- 172.21.15.196:0/1987648784 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3216269729 0 0) 0x7faa10003520 con 0x7faa2815f680 2024-04-28T13:31:34.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 -- 172.21.15.196:0/1987648784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa280b60b0 msgr2=0x7faa280d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:34.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 --2- 172.21.15.196:0/1987648784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa280b60b0 0x7faa280d4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 -- 172.21.15.196:0/1987648784 >> v2:172.21.15.203:3300/0 conn(0x7faa280d3820 msgr2=0x7faa280b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:34.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 --2- 172.21.15.196:0/1987648784 >> v2:172.21.15.203:3300/0 conn(0x7faa280d3820 0x7faa280b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 -- 172.21.15.196:0/1987648784 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa280b5090 con 0x7faa2815f680 2024-04-28T13:31:34.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 -- 172.21.15.196:0/1987648784 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7faa1c001690 con 0x7faa2815f680 2024-04-28T13:31:34.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 -- 172.21.15.196:0/1987648784 >> v1:172.21.15.196:6789/0 conn(0x7faa2815f680 legacy=0x7faa28163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 --2- 172.21.15.196:0/1987648784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa280b60b0 0x7faa100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 --2- 172.21.15.196:0/1987648784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10003f30 0x7faa10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 --2- 172.21.15.196:0/1987648784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa10004d40 0x7faa100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 -- 172.21.15.196:0/1987648784 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa2815fa60 con 0x7faa10003f30 2024-04-28T13:31:34.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 -- 172.21.15.196:0/1987648784 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faa280d3f50 con 0x7faa10004d40 2024-04-28T13:31:34.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.030+0000 7faa27400640 1 -- 172.21.15.196:0/1987648784 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7faa1000ba10 con 0x7faa280b60b0 2024-04-28T13:31:34.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.031+0000 7faa27e00640 1 --2- 172.21.15.196:0/1987648784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10003f30 0x7faa10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.031+0000 7faa2d600640 1 --2- 172.21.15.196:0/1987648784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa10004d40 0x7faa100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.031+0000 7faa2cc00640 1 --2- 172.21.15.196:0/1987648784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa280b60b0 0x7faa100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.031+0000 7faa27e00640 1 -- 172.21.15.196:0/1987648784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa280b60b0 msgr2=0x7faa100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:34.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.031+0000 7faa27e00640 1 --2- 172.21.15.196:0/1987648784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa280b60b0 0x7faa100039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.031+0000 7faa27e00640 1 -- 172.21.15.196:0/1987648784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa10004d40 msgr2=0x7faa100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:34.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.031+0000 7faa27e00640 1 --2- 172.21.15.196:0/1987648784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa10004d40 0x7faa100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.031+0000 7faa27e00640 1 -- 172.21.15.196:0/1987648784 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa10003520 con 0x7faa10003f30 2024-04-28T13:31:34.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.032+0000 7faa27e00640 1 --2- 172.21.15.196:0/1987648784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10003f30 0x7faa10004800 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faa14009ac0 tx=0x7faa10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d2621e4d097385 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:34.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.032+0000 7faa27400640 1 -- 172.21.15.196:0/1987648784 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14036070 con 0x7faa10003f30 2024-04-28T13:31:34.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.032+0000 7faa27400640 1 -- 172.21.15.196:0/1987648784 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa14031400 con 0x7faa10003f30 2024-04-28T13:31:34.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.032+0000 7faa27400640 1 -- 172.21.15.196:0/1987648784 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14009690 con 0x7faa10003f30 2024-04-28T13:31:34.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.033+0000 7faa2f600640 1 -- 172.21.15.196:0/1987648784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10003f30 msgr2=0x7faa10004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.033+0000 7faa2f600640 1 --2- 172.21.15.196:0/1987648784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10003f30 0x7faa10004800 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faa14009ac0 tx=0x7faa10005830 comp rx=0 tx=0).stop 2024-04-28T13:31:34.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.033+0000 7faa2d600640 1 -- 172.21.15.196:0/1987648784 reap_dead start 2024-04-28T13:31:34.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.033+0000 7faa2f600640 1 -- 172.21.15.196:0/1987648784 shutdown_connections 2024-04-28T13:31:34.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.033+0000 7faa2f600640 1 -- 172.21.15.196:0/1987648784 >> 172.21.15.196:0/1987648784 conn(0x7faa2808eca0 msgr2=0x7faa2809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:34.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.033+0000 7faa2f600640 1 -- 172.21.15.196:0/1987648784 shutdown_connections 2024-04-28T13:31:34.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.033+0000 7faa2f600640 1 -- 172.21.15.196:0/1987648784 wait complete. 2024-04-28T13:31:34.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.035+0000 7faa2f600640 1 Processor -- start 2024-04-28T13:31:34.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.035+0000 7faa2f600640 1 -- start start 2024-04-28T13:31:34.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.036+0000 7faa2f600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10003f30 0x7faa2815fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.036+0000 7faa2f600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa2815af40 0x7faa280ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.036+0000 7faa2f600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa2815b3c0 0x7faa280ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.036+0000 7faa2f600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa280d50a0 con 0x7faa10003f30 2024-04-28T13:31:34.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.036+0000 7faa2f600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faa280d4f20 con 0x7faa2815af40 2024-04-28T13:31:34.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.036+0000 7faa2f600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7faa280d5220 con 0x7faa2815b3c0 2024-04-28T13:31:34.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.036+0000 7faa2cc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10003f30 0x7faa2815fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.036+0000 7faa2cc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10003f30 0x7faa2815fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56398/0 (socket says 172.21.15.196:56398) 2024-04-28T13:31:34.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.036+0000 7faa2cc00640 1 -- 172.21.15.196:0/2726585016 learned_addr learned my addr 172.21.15.196:0/2726585016 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:34.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.037+0000 7faa27e00640 1 --2- 172.21.15.196:0/2726585016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa2815af40 0x7faa280ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.037+0000 7faa2cc00640 1 -- 172.21.15.196:0/2726585016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa2815b3c0 msgr2=0x7faa280ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.037+0000 7faa2d600640 1 --2- 172.21.15.196:0/2726585016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa2815b3c0 0x7faa280ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.037+0000 7faa2cc00640 1 --2- 172.21.15.196:0/2726585016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa2815b3c0 0x7faa280ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.037+0000 7faa2cc00640 1 -- 172.21.15.196:0/2726585016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa2815af40 msgr2=0x7faa280ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.037+0000 7faa2cc00640 1 --2- 172.21.15.196:0/2726585016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa2815af40 0x7faa280ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.037+0000 7faa2cc00640 1 -- 172.21.15.196:0/2726585016 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa280ca1a0 con 0x7faa10003f30 2024-04-28T13:31:34.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.037+0000 7faa27e00640 1 --2- 172.21.15.196:0/2726585016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa2815af40 0x7faa280ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:34.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.037+0000 7faa2cc00640 1 --2- 172.21.15.196:0/2726585016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10003f30 0x7faa2815fbe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00e840 tx=0x7faa1c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:34.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.037+0000 7faa24c00640 1 -- 172.21.15.196:0/2726585016 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c00c870 con 0x7faa10003f30 2024-04-28T13:31:34.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.037+0000 7faa24c00640 1 -- 172.21.15.196:0/2726585016 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa1c018070 con 0x7faa10003f30 2024-04-28T13:31:34.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.037+0000 7faa24c00640 1 -- 172.21.15.196:0/2726585016 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c00cc70 con 0x7faa10003f30 2024-04-28T13:31:34.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.037+0000 7faa2f600640 1 -- 172.21.15.196:0/2726585016 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa280ca480 con 0x7faa10003f30 2024-04-28T13:31:34.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.037+0000 7faa2f600640 1 -- 172.21.15.196:0/2726585016 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa280caa40 con 0x7faa10003f30 2024-04-28T13:31:34.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.039+0000 7faa2f600640 1 -- 172.21.15.196:0/2726585016 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa2806a290 con 0x7faa10003f30 2024-04-28T13:31:34.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.039+0000 7faa24c00640 1 -- 172.21.15.196:0/2726585016 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa1c01d460 con 0x7faa10003f30 2024-04-28T13:31:34.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.041+0000 7faa24c00640 1 --2- 172.21.15.196:0/2726585016 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faa0806c920 0x7faa0806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.041+0000 7faa24c00640 1 -- 172.21.15.196:0/2726585016 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faa1c08bba0 con 0x7faa10003f30 2024-04-28T13:31:34.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.041+0000 7faa27e00640 1 --2- 172.21.15.196:0/2726585016 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faa0806c920 0x7faa0806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.043+0000 7faa27e00640 1 --2- 172.21.15.196:0/2726585016 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faa0806c920 0x7faa0806ede0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faa14033000 tx=0x7faa14008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:34.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.045+0000 7faa24c00640 1 -- 172.21.15.196:0/2726585016 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa1c059f30 con 0x7faa10003f30 2024-04-28T13:31:34.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.245+0000 7faa2f600640 1 -- 172.21.15.196:0/2726585016 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faa280d30d0 con 0x7faa10003f30 2024-04-28T13:31:34.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.246+0000 7faa24c00640 1 -- 172.21.15.196:0/2726585016 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7faa1c059830 con 0x7faa10003f30 2024-04-28T13:31:34.247 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:31:34.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.248+0000 7faa2f600640 1 -- 172.21.15.196:0/2726585016 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faa0806c920 msgr2=0x7faa0806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.248+0000 7faa2f600640 1 --2- 172.21.15.196:0/2726585016 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faa0806c920 0x7faa0806ede0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faa14033000 tx=0x7faa14008030 comp rx=0 tx=0).stop 2024-04-28T13:31:34.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.248+0000 7faa2f600640 1 -- 172.21.15.196:0/2726585016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10003f30 msgr2=0x7faa2815fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.248+0000 7faa2f600640 1 --2- 172.21.15.196:0/2726585016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10003f30 0x7faa2815fbe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00e840 tx=0x7faa1c00ed00 comp rx=0 tx=0).stop 2024-04-28T13:31:34.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.249+0000 7faa2f600640 1 -- 172.21.15.196:0/2726585016 shutdown_connections 2024-04-28T13:31:34.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.249+0000 7faa2f600640 1 --2- 172.21.15.196:0/2726585016 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faa0806c920 0x7faa0806ede0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.249+0000 7faa2f600640 1 --2- 172.21.15.196:0/2726585016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa2815b3c0 0x7faa280ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.249+0000 7faa2f600640 1 --2- 172.21.15.196:0/2726585016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa2815af40 0x7faa280ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.249+0000 7faa2f600640 1 --2- 172.21.15.196:0/2726585016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa10003f30 0x7faa2815fbe0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.249+0000 7faa2f600640 1 -- 172.21.15.196:0/2726585016 >> 172.21.15.196:0/2726585016 conn(0x7faa2808eca0 msgr2=0x7faa280b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:34.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.249+0000 7faa2f600640 1 -- 172.21.15.196:0/2726585016 shutdown_connections 2024-04-28T13:31:34.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.249+0000 7faa2f600640 1 -- 172.21.15.196:0/2726585016 wait complete. 2024-04-28T13:31:34.268 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:31:34.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.462+0000 7f9280e00640 1 Processor -- start 2024-04-28T13:31:34.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.462+0000 7f9280e00640 1 -- start start 2024-04-28T13:31:34.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.463+0000 7f9280e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927c1469f0 0x7f927c0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.464+0000 7f9280e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f927c160f40 con 0x7f927c0cd420 2024-04-28T13:31:34.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.464+0000 7f9280e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f927c161220 con 0x7f927c0d1410 2024-04-28T13:31:34.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.464+0000 7f9280e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f927c0b5960 con 0x7f927c1469f0 2024-04-28T13:31:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.464+0000 7f9273e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f927c0cd420 0x7f927c0d0d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:59216/0 (socket says 172.21.15.196:59216) 2024-04-28T13:31:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.464+0000 7f9273e00640 1 -- 172.21.15.196:0/804310556 learned_addr learned my addr 172.21.15.196:0/804310556 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.464+0000 7f9279e00640 1 --2- 172.21.15.196:0/804310556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927c1469f0 0x7f927c0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.464+0000 7f9279400640 1 -- 172.21.15.196:0/804310556 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2365993711 0 0) 0x7f927c160f40 con 0x7f927c0cd420 2024-04-28T13:31:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.464+0000 7f9279400640 1 -- 172.21.15.196:0/804310556 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9260003520 con 0x7f927c0cd420 2024-04-28T13:31:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f9279400640 1 -- 172.21.15.196:0/804310556 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1430668430 0 0) 0x7f9260003520 con 0x7f927c0cd420 2024-04-28T13:31:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f9279400640 1 -- 172.21.15.196:0/804310556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927c1469f0 msgr2=0x7f927c0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f9279400640 1 --2- 172.21.15.196:0/804310556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927c1469f0 0x7f927c0ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f9279400640 1 -- 172.21.15.196:0/804310556 >> v1:172.21.15.203:6789/0 conn(0x7f927c0d1410 legacy=0x7f927c0b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f9279400640 1 -- 172.21.15.196:0/804310556 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c161500 con 0x7f927c0cd420 2024-04-28T13:31:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f9279400640 1 -- 172.21.15.196:0/804310556 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f9264001690 con 0x7f927c0cd420 2024-04-28T13:31:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f9279400640 1 -- 172.21.15.196:0/804310556 >> v1:172.21.15.196:6789/0 conn(0x7f927c0cd420 legacy=0x7f927c0d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f9279400640 1 --2- 172.21.15.196:0/804310556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92600039e0 0x7f9260003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f9279400640 1 --2- 172.21.15.196:0/804310556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927c1469f0 0x7f9260004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f9279400640 1 --2- 172.21.15.196:0/804310556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9260004d40 0x7f92600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f9279400640 1 -- 172.21.15.196:0/804310556 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f926000ba70 con 0x7f9260004d40 2024-04-28T13:31:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f9279400640 1 -- 172.21.15.196:0/804310556 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f926000bc20 con 0x7f92600039e0 2024-04-28T13:31:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f9279400640 1 -- 172.21.15.196:0/804310556 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f926000be20 con 0x7f927c1469f0 2024-04-28T13:31:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f927a800640 1 --2- 172.21.15.196:0/804310556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9260004d40 0x7f92600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f9279e00640 1 --2- 172.21.15.196:0/804310556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92600039e0 0x7f9260003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.465+0000 7f9273e00640 1 --2- 172.21.15.196:0/804310556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927c1469f0 0x7f9260004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.469 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.466+0000 7f927a800640 1 -- 172.21.15.196:0/804310556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927c1469f0 msgr2=0x7f9260004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:34.469 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.466+0000 7f927a800640 1 --2- 172.21.15.196:0/804310556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927c1469f0 0x7f9260004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.469 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.466+0000 7f927a800640 1 -- 172.21.15.196:0/804310556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92600039e0 msgr2=0x7f9260003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:34.469 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.466+0000 7f927a800640 1 --2- 172.21.15.196:0/804310556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92600039e0 0x7f9260003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.469 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.466+0000 7f927a800640 1 -- 172.21.15.196:0/804310556 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9260003520 con 0x7f9260004d40 2024-04-28T13:31:34.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.467+0000 7f927a800640 1 --2- 172.21.15.196:0/804310556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9260004d40 0x7f92600051c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f927c074650 tx=0x7f9260005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ff106f1723e446 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:34.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.467+0000 7f9279400640 1 -- 172.21.15.196:0/804310556 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274001a70 con 0x7f9260004d40 2024-04-28T13:31:34.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.467+0000 7f9279400640 1 -- 172.21.15.196:0/804310556 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f927402add0 con 0x7f9260004d40 2024-04-28T13:31:34.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.467+0000 7f9279400640 1 -- 172.21.15.196:0/804310556 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274003bb0 con 0x7f9260004d40 2024-04-28T13:31:34.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.467+0000 7f9280e00640 1 -- 172.21.15.196:0/804310556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9260004d40 msgr2=0x7f92600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.467+0000 7f9280e00640 1 --2- 172.21.15.196:0/804310556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9260004d40 0x7f92600051c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f927c074650 tx=0x7f9260005830 comp rx=0 tx=0).stop 2024-04-28T13:31:34.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.467+0000 7f927a800640 1 -- 172.21.15.196:0/804310556 reap_dead start 2024-04-28T13:31:34.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.468+0000 7f9280e00640 1 -- 172.21.15.196:0/804310556 shutdown_connections 2024-04-28T13:31:34.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.468+0000 7f9280e00640 1 -- 172.21.15.196:0/804310556 >> 172.21.15.196:0/804310556 conn(0x7f927c0cd830 msgr2=0x7f927c0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:34.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.468+0000 7f9280e00640 1 -- 172.21.15.196:0/804310556 shutdown_connections 2024-04-28T13:31:34.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.468+0000 7f9280e00640 1 -- 172.21.15.196:0/804310556 wait complete. 2024-04-28T13:31:34.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.469+0000 7f9280e00640 1 Processor -- start 2024-04-28T13:31:34.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.469+0000 7f9280e00640 1 -- start start 2024-04-28T13:31:34.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.470+0000 7f9280e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9260004d40 0x7f927c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.470+0000 7f9280e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f927c15ba50 0x7f927c15bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.470+0000 7f9280e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f927c13e820 0x7f927c13eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.470+0000 7f9280e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f927c072020 con 0x7f927c15ba50 2024-04-28T13:31:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.470+0000 7f9280e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f927c161ac0 con 0x7f927c13e820 2024-04-28T13:31:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.470+0000 7f9280e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f927c160b80 con 0x7f9260004d40 2024-04-28T13:31:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.471+0000 7f9273e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f927c15ba50 0x7f927c15bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.471+0000 7f9273e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f927c15ba50 0x7f927c15bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56412/0 (socket says 172.21.15.196:56412) 2024-04-28T13:31:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.471+0000 7f9273e00640 1 -- 172.21.15.196:0/596172351 learned_addr learned my addr 172.21.15.196:0/596172351 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.471+0000 7f9279e00640 1 --2- 172.21.15.196:0/596172351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9260004d40 0x7f927c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.471+0000 7f927a800640 1 --2- 172.21.15.196:0/596172351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f927c13e820 0x7f927c13eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.471+0000 7f9273e00640 1 -- 172.21.15.196:0/596172351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9260004d40 msgr2=0x7f927c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.471+0000 7f9273e00640 1 --2- 172.21.15.196:0/596172351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9260004d40 0x7f927c0b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.471+0000 7f9273e00640 1 -- 172.21.15.196:0/596172351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f927c13e820 msgr2=0x7f927c13eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.471+0000 7f9273e00640 1 --2- 172.21.15.196:0/596172351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f927c13e820 0x7f927c13eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.471+0000 7f9273e00640 1 -- 172.21.15.196:0/596172351 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c13d3d0 con 0x7f927c15ba50 2024-04-28T13:31:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.471+0000 7f927a800640 1 --2- 172.21.15.196:0/596172351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f927c13e820 0x7f927c13eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.471+0000 7f9273e00640 1 --2- 172.21.15.196:0/596172351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f927c15ba50 0x7f927c15bed0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f926400e730 tx=0x7f926400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.472+0000 7f9272000640 1 -- 172.21.15.196:0/596172351 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926400ee40 con 0x7f927c15ba50 2024-04-28T13:31:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.472+0000 7f9272000640 1 -- 172.21.15.196:0/596172351 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9264018070 con 0x7f927c15ba50 2024-04-28T13:31:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.472+0000 7f9272000640 1 -- 172.21.15.196:0/596172351 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926400cb10 con 0x7f927c15ba50 2024-04-28T13:31:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.472+0000 7f9280e00640 1 -- 172.21.15.196:0/596172351 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927c13d6b0 con 0x7f927c15ba50 2024-04-28T13:31:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.472+0000 7f9280e00640 1 -- 172.21.15.196:0/596172351 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927c13dc70 con 0x7f927c15ba50 2024-04-28T13:31:34.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.473+0000 7f9280e00640 1 -- 172.21.15.196:0/596172351 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f927c06a290 con 0x7f927c15ba50 2024-04-28T13:31:34.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.474+0000 7f9272000640 1 -- 172.21.15.196:0/596172351 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f926400ccb0 con 0x7f927c15ba50 2024-04-28T13:31:34.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.475+0000 7f9272000640 1 --2- 172.21.15.196:0/596172351 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f925806c8f0 0x7f925806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.475+0000 7f9272000640 1 -- 172.21.15.196:0/596172351 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f926408ba50 con 0x7f927c15ba50 2024-04-28T13:31:34.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.475+0000 7f9279e00640 1 --2- 172.21.15.196:0/596172351 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f925806c8f0 0x7f925806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.477+0000 7f9279e00640 1 --2- 172.21.15.196:0/596172351 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f925806c8f0 0x7f925806edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f926c009a10 tx=0x7f926c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:34.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.479+0000 7f9272000640 1 -- 172.21.15.196:0/596172351 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9264059e70 con 0x7f927c15ba50 2024-04-28T13:31:34.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.717+0000 7f9280e00640 1 -- 172.21.15.196:0/596172351 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f927c06fd90 con 0x7f927c15ba50 2024-04-28T13:31:34.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.718+0000 7f9272000640 1 -- 172.21.15.196:0/596172351 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f9264059770 con 0x7f927c15ba50 2024-04-28T13:31:34.719 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:34.720 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/422653807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":422653807},{"type":"v1","addr":"172.21.15.196:6834","nonce":422653807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2793552376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2793552376},{"type":"v1","addr":"172.21.15.196:6835","nonce":2793552376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1810956193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1810956193},{"type":"v1","addr":"172.21.15.203:6837","nonce":1810956193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:31:21.059586+0000","modified":"2024-04-28T13:31:24.132134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/2114798853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2114798853},{"type":"v1","addr":"172.21.15.203:6836","nonce":2114798853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4253,"qdb_cluster":[4253]},"id":1}]} 2024-04-28T13:31:34.720 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 7 2024-04-28T13:31:34.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.721+0000 7f9280e00640 1 -- 172.21.15.196:0/596172351 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f925806c8f0 msgr2=0x7f925806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.721+0000 7f9280e00640 1 --2- 172.21.15.196:0/596172351 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f925806c8f0 0x7f925806edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f926c009a10 tx=0x7f926c008040 comp rx=0 tx=0).stop 2024-04-28T13:31:34.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.721+0000 7f9280e00640 1 -- 172.21.15.196:0/596172351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f927c15ba50 msgr2=0x7f927c15bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.721+0000 7f9280e00640 1 --2- 172.21.15.196:0/596172351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f927c15ba50 0x7f927c15bed0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f926400e730 tx=0x7f926400ebf0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.721+0000 7f9280e00640 1 -- 172.21.15.196:0/596172351 shutdown_connections 2024-04-28T13:31:34.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.721+0000 7f9280e00640 1 --2- 172.21.15.196:0/596172351 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f925806c8f0 0x7f925806edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.721+0000 7f9280e00640 1 --2- 172.21.15.196:0/596172351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f927c13e820 0x7f927c13eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.721+0000 7f9280e00640 1 --2- 172.21.15.196:0/596172351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f927c15ba50 0x7f927c15bed0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.721+0000 7f9280e00640 1 --2- 172.21.15.196:0/596172351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9260004d40 0x7f927c0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.721+0000 7f9280e00640 1 -- 172.21.15.196:0/596172351 >> 172.21.15.196:0/596172351 conn(0x7f927c0cd830 msgr2=0x7f927c1410d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:34.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.721+0000 7f9280e00640 1 -- 172.21.15.196:0/596172351 shutdown_connections 2024-04-28T13:31:34.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.722+0000 7f9280e00640 1 -- 172.21.15.196:0/596172351 wait complete. 2024-04-28T13:31:34.743 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:34.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.932+0000 7f9a30a00640 1 Processor -- start 2024-04-28T13:31:34.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.932+0000 7f9a30a00640 1 -- start start 2024-04-28T13:31:34.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.934+0000 7f9a30a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9a2c0d0d00 0x7f9a2c0d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.934+0000 7f9a30a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9a2c0b5dd0 0x7f9a2c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.934+0000 7f9a30a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a2c160f40 con 0x7f9a2c1469f0 2024-04-28T13:31:34.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.934+0000 7f9a30a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9a2c0b5960 con 0x7f9a2c0b5dd0 2024-04-28T13:31:34.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.934+0000 7f9a30a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9a2c161ac0 con 0x7f9a2c0d0d00 2024-04-28T13:31:34.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.934+0000 7f9a2a800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9a2c0b5dd0 0x7f9a2c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.934+0000 7f9a2a800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9a2c0b5dd0 0x7f9a2c0b6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56422/0 (socket says 172.21.15.196:56422) 2024-04-28T13:31:34.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.934+0000 7f9a2a800640 1 -- 172.21.15.196:0/3086870084 learned_addr learned my addr 172.21.15.196:0/3086870084 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:34.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.934+0000 7f9a29e00640 1 --1- 172.21.15.196:0/3086870084 >> v1:172.21.15.196:6789/0 conn(0x7f9a2c1469f0 0x7f9a2c0ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:59226/0 (socket says 172.21.15.196:59226) 2024-04-28T13:31:34.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.935+0000 7f9a29400640 1 --2- 172.21.15.196:0/3086870084 >> v2:172.21.15.203:3300/0 conn(0x7f9a2c0d0d00 0x7f9a2c0d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.935+0000 7f9a2a800640 1 -- 172.21.15.196:0/3086870084 >> v2:172.21.15.203:3300/0 conn(0x7f9a2c0d0d00 msgr2=0x7f9a2c0d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:34.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.935+0000 7f9a2a800640 1 --2- 172.21.15.196:0/3086870084 >> v2:172.21.15.203:3300/0 conn(0x7f9a2c0d0d00 0x7f9a2c0d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.935+0000 7f9a2a800640 1 -- 172.21.15.196:0/3086870084 >> v1:172.21.15.196:6789/0 conn(0x7f9a2c1469f0 legacy=0x7f9a2c0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.935+0000 7f9a2a800640 1 -- 172.21.15.196:0/3086870084 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c161220 con 0x7f9a2c0b5dd0 2024-04-28T13:31:34.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.935+0000 7f9a28a00640 1 -- 172.21.15.196:0/3086870084 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 955604396 0 0) 0x7f9a2c160f40 con 0x7f9a2c1469f0 2024-04-28T13:31:34.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.936+0000 7f9a2a800640 1 --2- 172.21.15.196:0/3086870084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0b5dd0 0x7f9a2c0b6200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0d1de0 tx=0x7f9a2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb40bef1e27068ee server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:34.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.936+0000 7f9a28a00640 1 -- 172.21.15.196:0/3086870084 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24037070 con 0x7f9a2c0b5dd0 2024-04-28T13:31:34.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.936+0000 7f9a28a00640 1 -- 172.21.15.196:0/3086870084 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a24032440 con 0x7f9a2c0b5dd0 2024-04-28T13:31:34.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.936+0000 7f9a28a00640 1 -- 172.21.15.196:0/3086870084 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24032810 con 0x7f9a2c0b5dd0 2024-04-28T13:31:34.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.937+0000 7f9a30a00640 1 -- 172.21.15.196:0/3086870084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0b5dd0 msgr2=0x7f9a2c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.937+0000 7f9a30a00640 1 --2- 172.21.15.196:0/3086870084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0b5dd0 0x7f9a2c0b6200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0d1de0 tx=0x7f9a2402a270 comp rx=0 tx=0).stop 2024-04-28T13:31:34.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.937+0000 7f9a30a00640 1 -- 172.21.15.196:0/3086870084 shutdown_connections 2024-04-28T13:31:34.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.937+0000 7f9a30a00640 1 --2- 172.21.15.196:0/3086870084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0b5dd0 0x7f9a2c0b6200 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.937+0000 7f9a30a00640 1 --2- 172.21.15.196:0/3086870084 >> v2:172.21.15.203:3300/0 conn(0x7f9a2c0d0d00 0x7f9a2c0d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.937+0000 7f9a30a00640 1 -- 172.21.15.196:0/3086870084 >> 172.21.15.196:0/3086870084 conn(0x7f9a2c0cd830 msgr2=0x7f9a2c0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:34.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.937+0000 7f9a30a00640 1 -- 172.21.15.196:0/3086870084 shutdown_connections 2024-04-28T13:31:34.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.937+0000 7f9a30a00640 1 -- 172.21.15.196:0/3086870084 wait complete. 2024-04-28T13:31:34.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.939+0000 7f9a30a00640 1 Processor -- start 2024-04-28T13:31:34.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.939+0000 7f9a30a00640 1 -- start start 2024-04-28T13:31:34.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a30a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0b5dd0 0x7f9a2c144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a30a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a2c0d0d00 0x7f9a2c144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a30a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a2c1469f0 0x7f9a2c15f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a30a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c0cd5f0 con 0x7f9a2c0b5dd0 2024-04-28T13:31:34.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a30a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1617e0 con 0x7f9a2c1469f0 2024-04-28T13:31:34.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a30a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9a2c072020 con 0x7f9a2c0d0d00 2024-04-28T13:31:34.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a29e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0b5dd0 0x7f9a2c144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a29e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0b5dd0 0x7f9a2c144900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56436/0 (socket says 172.21.15.196:56436) 2024-04-28T13:31:34.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a29e00640 1 -- 172.21.15.196:0/3101416121 learned_addr learned my addr 172.21.15.196:0/3101416121 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:34.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a29400640 1 --2- 172.21.15.196:0/3101416121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a2c0d0d00 0x7f9a2c144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a29e00640 1 -- 172.21.15.196:0/3101416121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a2c0d0d00 msgr2=0x7f9a2c144e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a29e00640 1 --2- 172.21.15.196:0/3101416121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a2c0d0d00 0x7f9a2c144e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a2a800640 1 --2- 172.21.15.196:0/3101416121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a2c1469f0 0x7f9a2c15f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a29e00640 1 -- 172.21.15.196:0/3101416121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a2c1469f0 msgr2=0x7f9a2c15f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:34.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a29e00640 1 --2- 172.21.15.196:0/3101416121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a2c1469f0 0x7f9a2c15f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:34.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.940+0000 7f9a29e00640 1 -- 172.21.15.196:0/3101416121 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c0aea30 con 0x7f9a2c0b5dd0 2024-04-28T13:31:34.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.941+0000 7f9a29e00640 1 --2- 172.21.15.196:0/3101416121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0b5dd0 0x7f9a2c144900 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00c460 tx=0x7f9a1c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:34.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.941+0000 7f9a22000640 1 -- 172.21.15.196:0/3101416121 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c018070 con 0x7f9a2c0b5dd0 2024-04-28T13:31:34.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.941+0000 7f9a22000640 1 -- 172.21.15.196:0/3101416121 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a1c007500 con 0x7f9a2c0b5dd0 2024-04-28T13:31:34.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.941+0000 7f9a22000640 1 -- 172.21.15.196:0/3101416121 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c004a20 con 0x7f9a2c0b5dd0 2024-04-28T13:31:34.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.941+0000 7f9a30a00640 1 -- 172.21.15.196:0/3101416121 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2c0aed10 con 0x7f9a2c0b5dd0 2024-04-28T13:31:34.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.941+0000 7f9a30a00640 1 -- 172.21.15.196:0/3101416121 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2c0af7f0 con 0x7f9a2c0b5dd0 2024-04-28T13:31:34.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.942+0000 7f9a30a00640 1 -- 172.21.15.196:0/3101416121 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a2c06a290 con 0x7f9a2c0b5dd0 2024-04-28T13:31:34.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.944+0000 7f9a22000640 1 -- 172.21.15.196:0/3101416121 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a1c004bc0 con 0x7f9a2c0b5dd0 2024-04-28T13:31:34.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.945+0000 7f9a22000640 1 --2- 172.21.15.196:0/3101416121 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a0c06c920 0x7f9a0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:34.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.945+0000 7f9a22000640 1 -- 172.21.15.196:0/3101416121 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9a1c08ad70 con 0x7f9a2c0b5dd0 2024-04-28T13:31:34.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.946+0000 7f9a29400640 1 --2- 172.21.15.196:0/3101416121 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a0c06c920 0x7f9a0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:34.947 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.947+0000 7f9a29400640 1 --2- 172.21.15.196:0/3101416121 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a0c06c920 0x7f9a0c06ede0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c06f620 tx=0x7f9a1800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:34.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:34.950+0000 7f9a22000640 1 -- 172.21.15.196:0/3101416121 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a1c059230 con 0x7f9a2c0b5dd0 2024-04-28T13:31:35.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.150+0000 7f9a30a00640 1 -- 172.21.15.196:0/3101416121 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a2c06fd90 con 0x7f9a2c0b5dd0 2024-04-28T13:31:35.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.153+0000 7f9a22000640 1 -- 172.21.15.196:0/3101416121 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f9a1c058b30 con 0x7f9a2c0b5dd0 2024-04-28T13:31:35.154 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:35.154 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":20,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:22.624564+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:35.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.155+0000 7f9a30a00640 1 -- 172.21.15.196:0/3101416121 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a0c06c920 msgr2=0x7f9a0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:35.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.155+0000 7f9a30a00640 1 --2- 172.21.15.196:0/3101416121 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a0c06c920 0x7f9a0c06ede0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c06f620 tx=0x7f9a1800b040 comp rx=0 tx=0).stop 2024-04-28T13:31:35.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.156+0000 7f9a30a00640 1 -- 172.21.15.196:0/3101416121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0b5dd0 msgr2=0x7f9a2c144900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:35.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.156+0000 7f9a30a00640 1 --2- 172.21.15.196:0/3101416121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0b5dd0 0x7f9a2c144900 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00c460 tx=0x7f9a1c00c920 comp rx=0 tx=0).stop 2024-04-28T13:31:35.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.156+0000 7f9a30a00640 1 -- 172.21.15.196:0/3101416121 shutdown_connections 2024-04-28T13:31:35.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.156+0000 7f9a30a00640 1 --2- 172.21.15.196:0/3101416121 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a0c06c920 0x7f9a0c06ede0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.156+0000 7f9a30a00640 1 --2- 172.21.15.196:0/3101416121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a2c1469f0 0x7f9a2c15f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.156+0000 7f9a30a00640 1 --2- 172.21.15.196:0/3101416121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a2c0d0d00 0x7f9a2c144e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.156+0000 7f9a30a00640 1 --2- 172.21.15.196:0/3101416121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0b5dd0 0x7f9a2c144900 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.156+0000 7f9a30a00640 1 -- 172.21.15.196:0/3101416121 >> 172.21.15.196:0/3101416121 conn(0x7f9a2c0cd830 msgr2=0x7f9a2c0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:35.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.156+0000 7f9a30a00640 1 -- 172.21.15.196:0/3101416121 shutdown_connections 2024-04-28T13:31:35.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.156+0000 7f9a30a00640 1 -- 172.21.15.196:0/3101416121 wait complete. 2024-04-28T13:31:35.176 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:31:19.280975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:31:20.037066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:31:35.177 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:31:35.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.382+0000 7f41b1e00640 1 Processor -- start 2024-04-28T13:31:35.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.383+0000 7f41b1e00640 1 -- start start 2024-04-28T13:31:35.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.384+0000 7f41b1e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f41ac0d10a0 0x7f41ac0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:35.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.384+0000 7f41b1e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f41ac0d1480 0x7f41ac160c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:35.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.384+0000 7f41b1e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41ac15d700 con 0x7f41ac06e8f0 2024-04-28T13:31:35.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.384+0000 7f41b1e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f41ac15e310 con 0x7f41ac0d1480 2024-04-28T13:31:35.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.384+0000 7f41b1e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f41ac15d9e0 con 0x7f41ac0d10a0 2024-04-28T13:31:35.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.385+0000 7f41b1400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f41ac0d1480 0x7f41ac160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:35.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.385+0000 7f41b1400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f41ac0d1480 0x7f41ac160c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56438/0 (socket says 172.21.15.196:56438) 2024-04-28T13:31:35.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.385+0000 7f41b1400640 1 -- 172.21.15.196:0/2445346871 learned_addr learned my addr 172.21.15.196:0/2445346871 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:35.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.385+0000 7f41b0a00640 1 --2- 172.21.15.196:0/2445346871 >> v2:172.21.15.203:3300/0 conn(0x7f41ac0d10a0 0x7f41ac0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:35.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.385+0000 7f41b1400640 1 -- 172.21.15.196:0/2445346871 >> v2:172.21.15.203:3300/0 conn(0x7f41ac0d10a0 msgr2=0x7f41ac0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:35.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.385+0000 7f41b1400640 1 --2- 172.21.15.196:0/2445346871 >> v2:172.21.15.203:3300/0 conn(0x7f41ac0d10a0 0x7f41ac0b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.385+0000 7f41b1400640 1 -- 172.21.15.196:0/2445346871 >> v1:172.21.15.203:6789/0 conn(0x7f41ac06e8f0 legacy=0x7f41ac0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:35.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.385+0000 7f41b1400640 1 -- 172.21.15.196:0/2445346871 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac15dcc0 con 0x7f41ac0d1480 2024-04-28T13:31:35.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.386+0000 7f41b1400640 1 --2- 172.21.15.196:0/2445346871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41ac0d1480 0x7f41ac160c90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0d1c30 tx=0x7f41a402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f04a4f577ce522 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:35.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.387+0000 7f41ab400640 1 -- 172.21.15.196:0/2445346871 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4036070 con 0x7f41ac0d1480 2024-04-28T13:31:35.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.387+0000 7f41ab400640 1 -- 172.21.15.196:0/2445346871 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41a4031600 con 0x7f41ac0d1480 2024-04-28T13:31:35.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.387+0000 7f41ab400640 1 -- 172.21.15.196:0/2445346871 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40319d0 con 0x7f41ac0d1480 2024-04-28T13:31:35.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.387+0000 7f41b1e00640 1 -- 172.21.15.196:0/2445346871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41ac0d1480 msgr2=0x7f41ac160c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:35.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.387+0000 7f41b1e00640 1 --2- 172.21.15.196:0/2445346871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41ac0d1480 0x7f41ac160c90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0d1c30 tx=0x7f41a402a660 comp rx=0 tx=0).stop 2024-04-28T13:31:35.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.387+0000 7f41b1e00640 1 -- 172.21.15.196:0/2445346871 shutdown_connections 2024-04-28T13:31:35.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.387+0000 7f41b1e00640 1 --2- 172.21.15.196:0/2445346871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41ac0d1480 0x7f41ac160c90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.387+0000 7f41b1e00640 1 --2- 172.21.15.196:0/2445346871 >> v2:172.21.15.203:3300/0 conn(0x7f41ac0d10a0 0x7f41ac0b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.388+0000 7f41b1e00640 1 -- 172.21.15.196:0/2445346871 >> 172.21.15.196:0/2445346871 conn(0x7f41ac06c6c0 msgr2=0x7f41ac06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:35.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.388+0000 7f41b1e00640 1 -- 172.21.15.196:0/2445346871 shutdown_connections 2024-04-28T13:31:35.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.388+0000 7f41b1e00640 1 -- 172.21.15.196:0/2445346871 wait complete. 2024-04-28T13:31:35.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.389+0000 7f41b1e00640 1 Processor -- start 2024-04-28T13:31:35.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.389+0000 7f41b1e00640 1 -- start start 2024-04-28T13:31:35.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.390+0000 7f41b1e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41ac06e8f0 0x7f41ac09cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:35.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.390+0000 7f41b1e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41ac0d10a0 0x7f41ac13f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:35.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.390+0000 7f41b1e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41ac0d1480 0x7f41ac0cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:35.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.390+0000 7f41b1e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac0d1a30 con 0x7f41ac06e8f0 2024-04-28T13:31:35.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.390+0000 7f41b1e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac0d18b0 con 0x7f41ac0d10a0 2024-04-28T13:31:35.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.390+0000 7f41b1e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f41ac1611d0 con 0x7f41ac0d1480 2024-04-28T13:31:35.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.391+0000 7f41b0a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41ac06e8f0 0x7f41ac09cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:35.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.391+0000 7f41b0a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41ac06e8f0 0x7f41ac09cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56452/0 (socket says 172.21.15.196:56452) 2024-04-28T13:31:35.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.391+0000 7f41b0a00640 1 -- 172.21.15.196:0/4105189160 learned_addr learned my addr 172.21.15.196:0/4105189160 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:35.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.391+0000 7f41abe00640 1 --2- 172.21.15.196:0/4105189160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41ac0d10a0 0x7f41ac13f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:35.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.391+0000 7f41b0a00640 1 -- 172.21.15.196:0/4105189160 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41ac0d1480 msgr2=0x7f41ac0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:35.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.391+0000 7f41b1400640 1 --2- 172.21.15.196:0/4105189160 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41ac0d1480 0x7f41ac0cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:35.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.391+0000 7f41b0a00640 1 --2- 172.21.15.196:0/4105189160 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41ac0d1480 0x7f41ac0cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.391+0000 7f41b0a00640 1 -- 172.21.15.196:0/4105189160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41ac0d10a0 msgr2=0x7f41ac13f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:35.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.391+0000 7f41b0a00640 1 --2- 172.21.15.196:0/4105189160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41ac0d10a0 0x7f41ac13f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.391+0000 7f41b0a00640 1 -- 172.21.15.196:0/4105189160 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac13fa90 con 0x7f41ac06e8f0 2024-04-28T13:31:35.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.391+0000 7f41b0a00640 1 --2- 172.21.15.196:0/4105189160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41ac06e8f0 0x7f41ac09cae0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f41a000d830 tx=0x7f41a000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:35.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.392+0000 7f41a8c00640 1 -- 172.21.15.196:0/4105189160 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a000b6c0 con 0x7f41ac06e8f0 2024-04-28T13:31:35.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.392+0000 7f41a8c00640 1 -- 172.21.15.196:0/4105189160 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41a0019070 con 0x7f41ac06e8f0 2024-04-28T13:31:35.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.392+0000 7f41b1e00640 1 -- 172.21.15.196:0/4105189160 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ac09ba40 con 0x7f41ac06e8f0 2024-04-28T13:31:35.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.392+0000 7f41a8c00640 1 -- 172.21.15.196:0/4105189160 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a000ba80 con 0x7f41ac06e8f0 2024-04-28T13:31:35.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.392+0000 7f41b1e00640 1 -- 172.21.15.196:0/4105189160 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ac09c000 con 0x7f41ac06e8f0 2024-04-28T13:31:35.395 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.393+0000 7f41b1e00640 1 -- 172.21.15.196:0/4105189160 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ac050730 con 0x7f41ac06e8f0 2024-04-28T13:31:35.395 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.394+0000 7f41a8c00640 1 -- 172.21.15.196:0/4105189160 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41a001e460 con 0x7f41ac06e8f0 2024-04-28T13:31:35.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.395+0000 7f41a8c00640 1 --2- 172.21.15.196:0/4105189160 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f418c06c8f0 0x7f418c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:35.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.395+0000 7f41a8c00640 1 -- 172.21.15.196:0/4105189160 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f41a008c2e0 con 0x7f41ac06e8f0 2024-04-28T13:31:35.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.396+0000 7f41abe00640 1 --2- 172.21.15.196:0/4105189160 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f418c06c8f0 0x7f418c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:35.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.397+0000 7f41abe00640 1 --2- 172.21.15.196:0/4105189160 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f418c06c8f0 0x7f418c06edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0cfbc0 tx=0x7f4198009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:35.400 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.400+0000 7f41a8c00640 1 -- 172.21.15.196:0/4105189160 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f41a005a660 con 0x7f41ac06e8f0 2024-04-28T13:31:35.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.599+0000 7f41b1e00640 1 -- 172.21.15.196:0/4105189160 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f41ac06f700 con 0x7f41ac06e8f0 2024-04-28T13:31:35.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.600+0000 7f41a8c00640 1 -- 172.21.15.196:0/4105189160 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f41a0059f60 con 0x7f41ac06e8f0 2024-04-28T13:31:35.653 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:31:35.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.603+0000 7f41b1e00640 1 -- 172.21.15.196:0/4105189160 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f418c06c8f0 msgr2=0x7f418c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:35.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.603+0000 7f41b1e00640 1 --2- 172.21.15.196:0/4105189160 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f418c06c8f0 0x7f418c06edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0cfbc0 tx=0x7f4198009040 comp rx=0 tx=0).stop 2024-04-28T13:31:35.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.603+0000 7f41b1e00640 1 -- 172.21.15.196:0/4105189160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41ac06e8f0 msgr2=0x7f41ac09cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:35.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.603+0000 7f41b1e00640 1 --2- 172.21.15.196:0/4105189160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41ac06e8f0 0x7f41ac09cae0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f41a000d830 tx=0x7f41a000dcf0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.603+0000 7f41b1e00640 1 -- 172.21.15.196:0/4105189160 shutdown_connections 2024-04-28T13:31:35.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.603+0000 7f41b1e00640 1 --2- 172.21.15.196:0/4105189160 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f418c06c8f0 0x7f418c06edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.603+0000 7f41b1e00640 1 --2- 172.21.15.196:0/4105189160 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41ac0d1480 0x7f41ac0cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.603+0000 7f41b1e00640 1 --2- 172.21.15.196:0/4105189160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41ac0d10a0 0x7f41ac13f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.603+0000 7f41b1e00640 1 --2- 172.21.15.196:0/4105189160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41ac06e8f0 0x7f41ac09cae0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.603+0000 7f41b1e00640 1 -- 172.21.15.196:0/4105189160 >> 172.21.15.196:0/4105189160 conn(0x7f41ac06c6c0 msgr2=0x7f41ac06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:35.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.603+0000 7f41b1e00640 1 -- 172.21.15.196:0/4105189160 shutdown_connections 2024-04-28T13:31:35.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.603+0000 7f41b1e00640 1 -- 172.21.15.196:0/4105189160 wait complete. 2024-04-28T13:31:35.655 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:31:35.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.861+0000 7f862b200640 1 Processor -- start 2024-04-28T13:31:35.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.861+0000 7f862b200640 1 -- start start 2024-04-28T13:31:35.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.863+0000 7f862b200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f862c0d3830 0x7f862c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:35.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.863+0000 7f862b200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0d3c10 0x7f862c0b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:35.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.863+0000 7f862b200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f862c0cfb80 con 0x7f862c0b8370 2024-04-28T13:31:35.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.863+0000 7f862b200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f862c0d2ee0 con 0x7f862c0d3830 2024-04-28T13:31:35.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.863+0000 7f862b200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f862c0d31c0 con 0x7f862c0d3c10 2024-04-28T13:31:35.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.863+0000 7f8629e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f862c0d3830 0x7f862c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:35.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.863+0000 7f8629e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f862c0d3830 0x7f862c0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56468/0 (socket says 172.21.15.196:56468) 2024-04-28T13:31:35.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.863+0000 7f8629e00640 1 -- 172.21.15.196:0/3339056776 learned_addr learned my addr 172.21.15.196:0/3339056776 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:35.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.863+0000 7f8623e00640 1 --2- 172.21.15.196:0/3339056776 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0d3c10 0x7f862c0b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:35.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.864+0000 7f8629e00640 1 -- 172.21.15.196:0/3339056776 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0d3c10 msgr2=0x7f862c0b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:35.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.864+0000 7f8629e00640 1 --2- 172.21.15.196:0/3339056776 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0d3c10 0x7f862c0b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.864+0000 7f8629e00640 1 -- 172.21.15.196:0/3339056776 >> v1:172.21.15.203:6789/0 conn(0x7f862c0b8370 legacy=0x7f862c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:35.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.864+0000 7f8629e00640 1 -- 172.21.15.196:0/3339056776 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c0cfe00 con 0x7f862c0d3830 2024-04-28T13:31:35.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.865+0000 7f8629e00640 1 --2- 172.21.15.196:0/3339056776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f862c0d3830 0x7f862c0d2400 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f861c009870 tx=0x7f861c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7814436b17cb50 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:35.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.865+0000 7f8629400640 1 -- 172.21.15.196:0/3339056776 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c037070 con 0x7f862c0d3830 2024-04-28T13:31:35.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.865+0000 7f8629400640 1 -- 172.21.15.196:0/3339056776 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f861c032440 con 0x7f862c0d3830 2024-04-28T13:31:35.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.865+0000 7f8629400640 1 -- 172.21.15.196:0/3339056776 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c032810 con 0x7f862c0d3830 2024-04-28T13:31:35.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.865+0000 7f862b200640 1 -- 172.21.15.196:0/3339056776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f862c0d3830 msgr2=0x7f862c0d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:35.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.865+0000 7f862b200640 1 --2- 172.21.15.196:0/3339056776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f862c0d3830 0x7f862c0d2400 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f861c009870 tx=0x7f861c02a210 comp rx=0 tx=0).stop 2024-04-28T13:31:35.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.866+0000 7f862b200640 1 -- 172.21.15.196:0/3339056776 shutdown_connections 2024-04-28T13:31:35.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.866+0000 7f862b200640 1 --2- 172.21.15.196:0/3339056776 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0d3c10 0x7f862c0b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.866+0000 7f862b200640 1 --2- 172.21.15.196:0/3339056776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f862c0d3830 0x7f862c0d2400 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.866+0000 7f862b200640 1 -- 172.21.15.196:0/3339056776 >> 172.21.15.196:0/3339056776 conn(0x7f862c0d4fe0 msgr2=0x7f862c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:35.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.866+0000 7f862b200640 1 -- 172.21.15.196:0/3339056776 shutdown_connections 2024-04-28T13:31:35.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.866+0000 7f862b200640 1 -- 172.21.15.196:0/3339056776 wait complete. 2024-04-28T13:31:35.868 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.867+0000 7f862b200640 1 Processor -- start 2024-04-28T13:31:35.868 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.868+0000 7f862b200640 1 -- start start 2024-04-28T13:31:35.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.868+0000 7f862b200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f862c0b8370 0x7f862c0ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:35.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f862b200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0d3830 0x7f862c14c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:35.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f862b200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f862c0d3c10 0x7f862c0d0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:35.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f862b200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f862c0d5d90 con 0x7f862c0d3c10 2024-04-28T13:31:35.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f862b200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f862c0d2940 con 0x7f862c0b8370 2024-04-28T13:31:35.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f862b200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f862c0d2bb0 con 0x7f862c0d3830 2024-04-28T13:31:35.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f862a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f862c0d3c10 0x7f862c0d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:35.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f862a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f862c0d3c10 0x7f862c0d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56476/0 (socket says 172.21.15.196:56476) 2024-04-28T13:31:35.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f862a800640 1 -- 172.21.15.196:0/3796217736 learned_addr learned my addr 172.21.15.196:0/3796217736 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:35.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f8623e00640 1 --2- 172.21.15.196:0/3796217736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0d3830 0x7f862c14c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:35.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f8629e00640 1 --2- 172.21.15.196:0/3796217736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f862c0b8370 0x7f862c0ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:35.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f862a800640 1 -- 172.21.15.196:0/3796217736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0d3830 msgr2=0x7f862c14c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:35.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f862a800640 1 --2- 172.21.15.196:0/3796217736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0d3830 0x7f862c14c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f862a800640 1 -- 172.21.15.196:0/3796217736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f862c0b8370 msgr2=0x7f862c0ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:35.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f862a800640 1 --2- 172.21.15.196:0/3796217736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f862c0b8370 0x7f862c0ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:35.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f862a800640 1 -- 172.21.15.196:0/3796217736 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c0cddd0 con 0x7f862c0d3c10 2024-04-28T13:31:35.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.869+0000 7f8629e00640 1 --2- 172.21.15.196:0/3796217736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f862c0b8370 0x7f862c0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:35.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.870+0000 7f862a800640 1 --2- 172.21.15.196:0/3796217736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f862c0d3c10 0x7f862c0d0600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f862400c700 tx=0x7f862400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:35.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.870+0000 7f8622000640 1 -- 172.21.15.196:0/3796217736 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624016020 con 0x7f862c0d3c10 2024-04-28T13:31:35.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.870+0000 7f8622000640 1 -- 172.21.15.196:0/3796217736 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f862400aa50 con 0x7f862c0d3c10 2024-04-28T13:31:35.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.870+0000 7f8622000640 1 -- 172.21.15.196:0/3796217736 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862400ce10 con 0x7f862c0d3c10 2024-04-28T13:31:35.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.870+0000 7f862b200640 1 -- 172.21.15.196:0/3796217736 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862c0ce0b0 con 0x7f862c0d3c10 2024-04-28T13:31:35.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.870+0000 7f862b200640 1 -- 172.21.15.196:0/3796217736 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f862c0ce670 con 0x7f862c0d3c10 2024-04-28T13:31:35.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.871+0000 7f862b200640 1 -- 172.21.15.196:0/3796217736 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f862c050730 con 0x7f862c0d3c10 2024-04-28T13:31:35.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.872+0000 7f8622000640 1 -- 172.21.15.196:0/3796217736 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8624019020 con 0x7f862c0d3c10 2024-04-28T13:31:35.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.873+0000 7f8622000640 1 --2- 172.21.15.196:0/3796217736 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f860806c8f0 0x7f860806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:35.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.873+0000 7f8622000640 1 -- 172.21.15.196:0/3796217736 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f862408beb0 con 0x7f862c0d3c10 2024-04-28T13:31:35.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.874+0000 7f8629e00640 1 --2- 172.21.15.196:0/3796217736 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f860806c8f0 0x7f860806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:35.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.875+0000 7f8629e00640 1 --2- 172.21.15.196:0/3796217736 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f860806c8f0 0x7f860806edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f861c034000 tx=0x7f861c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:35.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:35.878+0000 7f8622000640 1 -- 172.21.15.196:0/3796217736 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8624020020 con 0x7f862c0d3c10 2024-04-28T13:31:36.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.076+0000 7f862b200640 1 -- 172.21.15.196:0/3796217736 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f862c0d12c0 con 0x7f862c0d3c10 2024-04-28T13:31:36.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.077+0000 7f8622000640 1 -- 172.21.15.196:0/3796217736 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f862405a2e0 con 0x7f862c0d3c10 2024-04-28T13:31:36.078 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:31:36.081 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.079+0000 7f862b200640 1 -- 172.21.15.196:0/3796217736 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f860806c8f0 msgr2=0x7f860806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:36.081 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.079+0000 7f862b200640 1 --2- 172.21.15.196:0/3796217736 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f860806c8f0 0x7f860806edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f861c034000 tx=0x7f861c02e000 comp rx=0 tx=0).stop 2024-04-28T13:31:36.081 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.079+0000 7f862b200640 1 -- 172.21.15.196:0/3796217736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f862c0d3c10 msgr2=0x7f862c0d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:36.081 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.079+0000 7f862b200640 1 --2- 172.21.15.196:0/3796217736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f862c0d3c10 0x7f862c0d0600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f862400c700 tx=0x7f862400cbc0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.081 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.080+0000 7f862b200640 1 -- 172.21.15.196:0/3796217736 shutdown_connections 2024-04-28T13:31:36.081 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.080+0000 7f862b200640 1 --2- 172.21.15.196:0/3796217736 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f860806c8f0 0x7f860806edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.081 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.080+0000 7f862b200640 1 --2- 172.21.15.196:0/3796217736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f862c0d3c10 0x7f862c0d0600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.081 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.080+0000 7f862b200640 1 --2- 172.21.15.196:0/3796217736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0d3830 0x7f862c14c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.081 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.080+0000 7f862b200640 1 --2- 172.21.15.196:0/3796217736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f862c0b8370 0x7f862c0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.081 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.080+0000 7f862b200640 1 -- 172.21.15.196:0/3796217736 >> 172.21.15.196:0/3796217736 conn(0x7f862c0d4fe0 msgr2=0x7f862c0b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:36.081 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.080+0000 7f862b200640 1 -- 172.21.15.196:0/3796217736 shutdown_connections 2024-04-28T13:31:36.082 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.080+0000 7f862b200640 1 -- 172.21.15.196:0/3796217736 wait complete. 2024-04-28T13:31:36.101 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:31:36.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.305+0000 7f77e0a00640 1 Processor -- start 2024-04-28T13:31:36.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.305+0000 7f77e0a00640 1 -- start start 2024-04-28T13:31:36.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.307+0000 7f77e0a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f77dc0723d0 0x7f77dc06f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:36.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.307+0000 7f77e0a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77dc160570 con 0x7f77dc072d00 2024-04-28T13:31:36.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.307+0000 7f77e0a00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77dc06df30 con 0x7f77dc15d0a0 2024-04-28T13:31:36.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.307+0000 7f77e0a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f77dc0b3c90 con 0x7f77dc0723d0 2024-04-28T13:31:36.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.307+0000 7f77da800640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f77dc072d00 0x7f77dc0730e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:59228/0 (socket says 172.21.15.196:59228) 2024-04-28T13:31:36.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.307+0000 7f77da800640 1 -- 172.21.15.196:0/760562608 learned_addr learned my addr 172.21.15.196:0/760562608 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:36.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.307+0000 7f77db200640 1 --2- 172.21.15.196:0/760562608 >> v2:172.21.15.203:3300/0 conn(0x7f77dc0723d0 0x7f77dc06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:36.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.307+0000 7f77d9e00640 1 -- 172.21.15.196:0/760562608 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2348188307 0 0) 0x7f77dc160570 con 0x7f77dc072d00 2024-04-28T13:31:36.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.307+0000 7f77d9e00640 1 -- 172.21.15.196:0/760562608 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77c0003520 con 0x7f77dc072d00 2024-04-28T13:31:36.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.308+0000 7f77d9e00640 1 -- 172.21.15.196:0/760562608 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3775016132 0 0) 0x7f77c0003520 con 0x7f77dc072d00 2024-04-28T13:31:36.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.308+0000 7f77d9e00640 1 -- 172.21.15.196:0/760562608 >> v2:172.21.15.203:3300/0 conn(0x7f77dc0723d0 msgr2=0x7f77dc06f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:36.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.308+0000 7f77d9e00640 1 --2- 172.21.15.196:0/760562608 >> v2:172.21.15.203:3300/0 conn(0x7f77dc0723d0 0x7f77dc06f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.308+0000 7f77d9e00640 1 -- 172.21.15.196:0/760562608 >> v1:172.21.15.203:6789/0 conn(0x7f77dc15d0a0 legacy=0x7f77dc0b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:36.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.308+0000 7f77d9e00640 1 -- 172.21.15.196:0/760562608 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77dc06e210 con 0x7f77dc072d00 2024-04-28T13:31:36.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.308+0000 7f77d9e00640 1 -- 172.21.15.196:0/760562608 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f77cc001670 con 0x7f77dc072d00 2024-04-28T13:31:36.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.308+0000 7f77d9e00640 1 -- 172.21.15.196:0/760562608 >> v1:172.21.15.196:6789/0 conn(0x7f77dc072d00 legacy=0x7f77dc0730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:36.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.308+0000 7f77d9e00640 1 --2- 172.21.15.196:0/760562608 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c0003a00 0x7f77c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:36.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.308+0000 7f77d9e00640 1 --2- 172.21.15.196:0/760562608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77c0004800 0x7f77c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:36.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.308+0000 7f77d9e00640 1 --2- 172.21.15.196:0/760562608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00051c0 0x7f77c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:36.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.308+0000 7f77d9e00640 1 -- 172.21.15.196:0/760562608 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc0b3c90 con 0x7f77c00051c0 2024-04-28T13:31:36.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.308+0000 7f77d9e00640 1 -- 172.21.15.196:0/760562608 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f77c000c0c0 con 0x7f77c0004800 2024-04-28T13:31:36.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.308+0000 7f77d9e00640 1 -- 172.21.15.196:0/760562608 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f77c000c2c0 con 0x7f77c0003a00 2024-04-28T13:31:36.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.308+0000 7f77db200640 1 --2- 172.21.15.196:0/760562608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00051c0 0x7f77c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:36.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.308+0000 7f77d1e00640 1 --2- 172.21.15.196:0/760562608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77c0004800 0x7f77c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:36.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.309+0000 7f77da800640 1 --2- 172.21.15.196:0/760562608 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c0003a00 0x7f77c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:36.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.309+0000 7f77db200640 1 -- 172.21.15.196:0/760562608 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c0003a00 msgr2=0x7f77c0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:36.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.309+0000 7f77db200640 1 --2- 172.21.15.196:0/760562608 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c0003a00 0x7f77c0003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.309+0000 7f77db200640 1 -- 172.21.15.196:0/760562608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77c0004800 msgr2=0x7f77c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:36.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.309+0000 7f77db200640 1 --2- 172.21.15.196:0/760562608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77c0004800 0x7f77c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.309+0000 7f77db200640 1 -- 172.21.15.196:0/760562608 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c0003520 con 0x7f77c00051c0 2024-04-28T13:31:36.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.309+0000 7f77db200640 1 -- 172.21.15.196:0/760562608 reap_dead start 2024-04-28T13:31:36.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.310+0000 7f77db200640 1 --2- 172.21.15.196:0/760562608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00051c0 0x7f77c000b9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009ac0 tx=0x7f77c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaae8d8928e4c0de server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:36.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.310+0000 7f77d9e00640 1 -- 172.21.15.196:0/760562608 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4009690 con 0x7f77c00051c0 2024-04-28T13:31:36.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.310+0000 7f77d9e00640 1 -- 172.21.15.196:0/760562608 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d4004020 con 0x7f77c00051c0 2024-04-28T13:31:36.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.310+0000 7f77d9e00640 1 -- 172.21.15.196:0/760562608 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4004410 con 0x7f77c00051c0 2024-04-28T13:31:36.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.310+0000 7f77e0a00640 1 -- 172.21.15.196:0/760562608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00051c0 msgr2=0x7f77c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:36.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.310+0000 7f77e0a00640 1 --2- 172.21.15.196:0/760562608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00051c0 0x7f77c000b9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009ac0 tx=0x7f77c0005830 comp rx=0 tx=0).stop 2024-04-28T13:31:36.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.310+0000 7f77e0a00640 1 -- 172.21.15.196:0/760562608 shutdown_connections 2024-04-28T13:31:36.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.310+0000 7f77e0a00640 1 --2- 172.21.15.196:0/760562608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00051c0 0x7f77c000b9c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.310+0000 7f77e0a00640 1 -- 172.21.15.196:0/760562608 >> 172.21.15.196:0/760562608 conn(0x7f77dc0b2990 msgr2=0x7f77dc0b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:36.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.311+0000 7f77e0a00640 1 -- 172.21.15.196:0/760562608 shutdown_connections 2024-04-28T13:31:36.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.311+0000 7f77e0a00640 1 -- 172.21.15.196:0/760562608 wait complete. 2024-04-28T13:31:36.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.312+0000 7f77e0a00640 1 Processor -- start 2024-04-28T13:31:36.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.312+0000 7f77e0a00640 1 -- start start 2024-04-28T13:31:36.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.313+0000 7f77e0a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00051c0 0x7f77dc1f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:36.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.313+0000 7f77e0a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77dc1f5dc0 0x7f77dc141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:36.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.313+0000 7f77e0a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77dc14df70 0x7f77dc1f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:36.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.313+0000 7f77e0a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc073260 con 0x7f77c00051c0 2024-04-28T13:31:36.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.313+0000 7f77e0a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc0730e0 con 0x7f77dc14df70 2024-04-28T13:31:36.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.313+0000 7f77e0a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f77dc0733e0 con 0x7f77dc1f5dc0 2024-04-28T13:31:36.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.314+0000 7f77da800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00051c0 0x7f77dc1f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:36.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.314+0000 7f77da800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00051c0 0x7f77dc1f6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56500/0 (socket says 172.21.15.196:56500) 2024-04-28T13:31:36.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.314+0000 7f77da800640 1 -- 172.21.15.196:0/3627761632 learned_addr learned my addr 172.21.15.196:0/3627761632 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:36.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.314+0000 7f77d1e00640 1 --2- 172.21.15.196:0/3627761632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77dc1f5dc0 0x7f77dc141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:36.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.314+0000 7f77db200640 1 --2- 172.21.15.196:0/3627761632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77dc14df70 0x7f77dc1f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:36.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.314+0000 7f77da800640 1 -- 172.21.15.196:0/3627761632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77dc1f5dc0 msgr2=0x7f77dc141230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:36.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.314+0000 7f77da800640 1 --2- 172.21.15.196:0/3627761632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77dc1f5dc0 0x7f77dc141230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.314+0000 7f77da800640 1 -- 172.21.15.196:0/3627761632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77dc14df70 msgr2=0x7f77dc1f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:36.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.314+0000 7f77da800640 1 --2- 172.21.15.196:0/3627761632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77dc14df70 0x7f77dc1f4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.314+0000 7f77da800640 1 -- 172.21.15.196:0/3627761632 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77dc14e8a0 con 0x7f77c00051c0 2024-04-28T13:31:36.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.314+0000 7f77da800640 1 --2- 172.21.15.196:0/3627761632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00051c0 0x7f77dc1f6360 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00d7d0 tx=0x7f77cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:36.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.314+0000 7f77d3400640 1 -- 172.21.15.196:0/3627761632 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc019070 con 0x7f77c00051c0 2024-04-28T13:31:36.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.315+0000 7f77d3400640 1 -- 172.21.15.196:0/3627761632 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77cc0072d0 con 0x7f77c00051c0 2024-04-28T13:31:36.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.315+0000 7f77d3400640 1 -- 172.21.15.196:0/3627761632 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0047e0 con 0x7f77c00051c0 2024-04-28T13:31:36.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.315+0000 7f77e0a00640 1 -- 172.21.15.196:0/3627761632 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77dc15d9b0 con 0x7f77c00051c0 2024-04-28T13:31:36.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.315+0000 7f77e0a00640 1 -- 172.21.15.196:0/3627761632 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77dc15df70 con 0x7f77c00051c0 2024-04-28T13:31:36.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.316+0000 7f77e0a00640 1 -- 172.21.15.196:0/3627761632 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77dc050730 con 0x7f77c00051c0 2024-04-28T13:31:36.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.317+0000 7f77d3400640 1 -- 172.21.15.196:0/3627761632 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77cc004980 con 0x7f77c00051c0 2024-04-28T13:31:36.318 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.317+0000 7f77d3400640 1 --2- 172.21.15.196:0/3627761632 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f77b806c920 0x7f77b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:36.318 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.318+0000 7f77d3400640 1 -- 172.21.15.196:0/3627761632 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f77cc08b3b0 con 0x7f77c00051c0 2024-04-28T13:31:36.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.318+0000 7f77d1e00640 1 --2- 172.21.15.196:0/3627761632 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f77b806c920 0x7f77b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:36.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.319+0000 7f77d1e00640 1 --2- 172.21.15.196:0/3627761632 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f77b806c920 0x7f77b806ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f77c4007fd0 tx=0x7f77c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:36.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.322+0000 7f77d3400640 1 -- 172.21.15.196:0/3627761632 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f77cc059740 con 0x7f77c00051c0 2024-04-28T13:31:36.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.520+0000 7f77e0a00640 1 -- 172.21.15.196:0/3627761632 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f77dc0d09d0 con 0x7f77c00051c0 2024-04-28T13:31:36.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.521+0000 7f77d3400640 1 -- 172.21.15.196:0/3627761632 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f77cc059040 con 0x7f77c00051c0 2024-04-28T13:31:36.522 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:31:36.525 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.524+0000 7f77e0a00640 1 -- 172.21.15.196:0/3627761632 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f77b806c920 msgr2=0x7f77b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:36.525 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.524+0000 7f77e0a00640 1 --2- 172.21.15.196:0/3627761632 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f77b806c920 0x7f77b806ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f77c4007fd0 tx=0x7f77c4009040 comp rx=0 tx=0).stop 2024-04-28T13:31:36.525 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.524+0000 7f77e0a00640 1 -- 172.21.15.196:0/3627761632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00051c0 msgr2=0x7f77dc1f6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:36.525 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.524+0000 7f77e0a00640 1 --2- 172.21.15.196:0/3627761632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00051c0 0x7f77dc1f6360 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00d7d0 tx=0x7f77cc00dc90 comp rx=0 tx=0).stop 2024-04-28T13:31:36.525 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.524+0000 7f77e0a00640 1 -- 172.21.15.196:0/3627761632 shutdown_connections 2024-04-28T13:31:36.525 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.524+0000 7f77e0a00640 1 --2- 172.21.15.196:0/3627761632 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f77b806c920 0x7f77b806ede0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.525 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.524+0000 7f77e0a00640 1 --2- 172.21.15.196:0/3627761632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77dc14df70 0x7f77dc1f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.525 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.524+0000 7f77e0a00640 1 --2- 172.21.15.196:0/3627761632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77dc1f5dc0 0x7f77dc141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.524+0000 7f77e0a00640 1 --2- 172.21.15.196:0/3627761632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00051c0 0x7f77dc1f6360 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.524+0000 7f77e0a00640 1 -- 172.21.15.196:0/3627761632 >> 172.21.15.196:0/3627761632 conn(0x7f77dc0b2990 msgr2=0x7f77dc0d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:36.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.524+0000 7f77e0a00640 1 -- 172.21.15.196:0/3627761632 shutdown_connections 2024-04-28T13:31:36.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.524+0000 7f77e0a00640 1 -- 172.21.15.196:0/3627761632 wait complete. 2024-04-28T13:31:36.543 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:31:36.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.747+0000 7f3ae2000640 1 Processor -- start 2024-04-28T13:31:36.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.747+0000 7f3ae2000640 1 -- start start 2024-04-28T13:31:36.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.748+0000 7f3ae2000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3adc0d19a0 0x7f3adc0b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:36.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.749+0000 7f3ae2000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3adc06d060 con 0x7f3adc1468f0 2024-04-28T13:31:36.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.749+0000 7f3ae2000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3adc0d0940 con 0x7f3adc0b41f0 2024-04-28T13:31:36.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.749+0000 7f3ae2000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3adc0b4640 con 0x7f3adc0d19a0 2024-04-28T13:31:36.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.749+0000 7f3adaa00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3adc0d19a0 0x7f3adc0b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:36.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.749+0000 7f3adaa00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3adc0d19a0 0x7f3adc0b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56512/0 (socket says 172.21.15.196:56512) 2024-04-28T13:31:36.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.749+0000 7f3adaa00640 1 -- 172.21.15.196:0/3840119698 learned_addr learned my addr 172.21.15.196:0/3840119698 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:36.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.749+0000 7f3adaa00640 1 -- 172.21.15.196:0/3840119698 >> v1:172.21.15.203:6789/0 conn(0x7f3adc0b41f0 legacy=0x7f3adc15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:36.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.749+0000 7f3adaa00640 1 -- 172.21.15.196:0/3840119698 >> v1:172.21.15.196:6789/0 conn(0x7f3adc1468f0 legacy=0x7f3adc06f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:36.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.749+0000 7f3adaa00640 1 -- 172.21.15.196:0/3840119698 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3adc0d0be0 con 0x7f3adc0d19a0 2024-04-28T13:31:36.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.751+0000 7f3adaa00640 1 --2- 172.21.15.196:0/3840119698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3adc0d19a0 0x7f3adc0b3cb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009870 tx=0x7f3ac802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25db8d27cab2ae0c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:36.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.751+0000 7f3ada000640 1 -- 172.21.15.196:0/3840119698 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8037070 con 0x7f3adc0d19a0 2024-04-28T13:31:36.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.751+0000 7f3ada000640 1 -- 172.21.15.196:0/3840119698 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ac8032440 con 0x7f3adc0d19a0 2024-04-28T13:31:36.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.751+0000 7f3ada000640 1 -- 172.21.15.196:0/3840119698 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8032810 con 0x7f3adc0d19a0 2024-04-28T13:31:36.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.752+0000 7f3ae2000640 1 -- 172.21.15.196:0/3840119698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3adc0d19a0 msgr2=0x7f3adc0b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:36.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.752+0000 7f3ae2000640 1 --2- 172.21.15.196:0/3840119698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3adc0d19a0 0x7f3adc0b3cb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009870 tx=0x7f3ac802a210 comp rx=0 tx=0).stop 2024-04-28T13:31:36.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.752+0000 7f3ae2000640 1 -- 172.21.15.196:0/3840119698 shutdown_connections 2024-04-28T13:31:36.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.752+0000 7f3ae2000640 1 --2- 172.21.15.196:0/3840119698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3adc0d19a0 0x7f3adc0b3cb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.752+0000 7f3ae2000640 1 -- 172.21.15.196:0/3840119698 >> 172.21.15.196:0/3840119698 conn(0x7f3adc06d3b0 msgr2=0x7f3adc06d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:36.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.752+0000 7f3ae2000640 1 -- 172.21.15.196:0/3840119698 shutdown_connections 2024-04-28T13:31:36.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.752+0000 7f3ae2000640 1 -- 172.21.15.196:0/3840119698 wait complete. 2024-04-28T13:31:36.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.754+0000 7f3ae2000640 1 Processor -- start 2024-04-28T13:31:36.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.754+0000 7f3ae2000640 1 -- start start 2024-04-28T13:31:36.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.755+0000 7f3ae2000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3adc0b41f0 0x7f3adc0b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:36.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.755+0000 7f3ae2000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3adc0d19a0 0x7f3adc13af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:36.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.755+0000 7f3ae2000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3adc1468f0 0x7f3adc0731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:36.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.755+0000 7f3ae2000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc0d11a0 con 0x7f3adc0b41f0 2024-04-28T13:31:36.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.755+0000 7f3ae2000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc0d1510 con 0x7f3adc0d19a0 2024-04-28T13:31:36.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.755+0000 7f3ae2000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3adc146cd0 con 0x7f3adc1468f0 2024-04-28T13:31:36.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.756+0000 7f3adb400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3adc0b41f0 0x7f3adc0b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:36.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.756+0000 7f3adb400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3adc0b41f0 0x7f3adc0b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56528/0 (socket says 172.21.15.196:56528) 2024-04-28T13:31:36.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.756+0000 7f3adb400640 1 -- 172.21.15.196:0/1812486962 learned_addr learned my addr 172.21.15.196:0/1812486962 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:36.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.756+0000 7f3adb400640 1 -- 172.21.15.196:0/1812486962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3adc1468f0 msgr2=0x7f3adc0731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:36.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.756+0000 7f3adb400640 1 --2- 172.21.15.196:0/1812486962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3adc1468f0 0x7f3adc0731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.756+0000 7f3adb400640 1 -- 172.21.15.196:0/1812486962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3adc0d19a0 msgr2=0x7f3adc13af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:36.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.756+0000 7f3adaa00640 1 --2- 172.21.15.196:0/1812486962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3adc0d19a0 0x7f3adc13af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:36.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.756+0000 7f3adb400640 1 --2- 172.21.15.196:0/1812486962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3adc0d19a0 0x7f3adc13af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:36.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.756+0000 7f3adb400640 1 -- 172.21.15.196:0/1812486962 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3adc13dac0 con 0x7f3adc0b41f0 2024-04-28T13:31:36.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.756+0000 7f3adb400640 1 --2- 172.21.15.196:0/1812486962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3adc0b41f0 0x7f3adc0b0dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00c400 tx=0x7f3acc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:36.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.757+0000 7f3ad3400640 1 -- 172.21.15.196:0/1812486962 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0052d0 con 0x7f3adc0b41f0 2024-04-28T13:31:36.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.757+0000 7f3ad3400640 1 -- 172.21.15.196:0/1812486962 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3acc005490 con 0x7f3adc0b41f0 2024-04-28T13:31:36.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.757+0000 7f3ad3400640 1 -- 172.21.15.196:0/1812486962 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc007260 con 0x7f3adc0b41f0 2024-04-28T13:31:36.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.757+0000 7f3ae2000640 1 -- 172.21.15.196:0/1812486962 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3adc13dda0 con 0x7f3adc0b41f0 2024-04-28T13:31:36.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.757+0000 7f3ae2000640 1 -- 172.21.15.196:0/1812486962 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3adc13e360 con 0x7f3adc0b41f0 2024-04-28T13:31:36.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.758+0000 7f3ae2000640 1 -- 172.21.15.196:0/1812486962 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3adc0693d0 con 0x7f3adc0b41f0 2024-04-28T13:31:36.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.759+0000 7f3ad3400640 1 -- 172.21.15.196:0/1812486962 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3acc007400 con 0x7f3adc0b41f0 2024-04-28T13:31:36.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.760+0000 7f3ad3400640 1 --2- 172.21.15.196:0/1812486962 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3abc06c8f0 0x7f3abc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:36.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.760+0000 7f3ad3400640 1 -- 172.21.15.196:0/1812486962 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3acc08b120 con 0x7f3adc0b41f0 2024-04-28T13:31:36.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.760+0000 7f3adaa00640 1 --2- 172.21.15.196:0/1812486962 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3abc06c8f0 0x7f3abc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:36.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.762+0000 7f3adaa00640 1 --2- 172.21.15.196:0/1812486962 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3abc06c8f0 0x7f3abc06edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8034000 tx=0x7f3ac802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:36.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:36.764+0000 7f3ad3400640 1 -- 172.21.15.196:0/1812486962 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3acc0594b0 con 0x7f3adc0b41f0 2024-04-28T13:31:37.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.002+0000 7f3ae2000640 1 -- 172.21.15.196:0/1812486962 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3adc0b2620 con 0x7f3adc0b41f0 2024-04-28T13:31:37.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.004+0000 7f3ad3400640 1 -- 172.21.15.196:0/1812486962 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f3acc058db0 con 0x7f3adc0b41f0 2024-04-28T13:31:37.005 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/422653807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":422653807},{"type":"v1","addr":"172.21.15.196:6834","nonce":422653807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2793552376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2793552376},{"type":"v1","addr":"172.21.15.196:6835","nonce":2793552376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1810956193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1810956193},{"type":"v1","addr":"172.21.15.203:6837","nonce":1810956193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:31:21.059586+0000","modified":"2024-04-28T13:31:24.132134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/2114798853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2114798853},{"type":"v1","addr":"172.21.15.203:6836","nonce":2114798853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4253,"qdb_cluster":[4253]},"id":1}]} 2024-04-28T13:31:37.005 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 7 2024-04-28T13:31:37.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.006+0000 7f3ae2000640 1 -- 172.21.15.196:0/1812486962 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3abc06c8f0 msgr2=0x7f3abc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:37.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.006+0000 7f3ae2000640 1 --2- 172.21.15.196:0/1812486962 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3abc06c8f0 0x7f3abc06edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8034000 tx=0x7f3ac802e000 comp rx=0 tx=0).stop 2024-04-28T13:31:37.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.006+0000 7f3ae2000640 1 -- 172.21.15.196:0/1812486962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3adc0b41f0 msgr2=0x7f3adc0b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:37.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.006+0000 7f3ae2000640 1 --2- 172.21.15.196:0/1812486962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3adc0b41f0 0x7f3adc0b0dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00c400 tx=0x7f3acc00c8c0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.006+0000 7f3ae2000640 1 -- 172.21.15.196:0/1812486962 shutdown_connections 2024-04-28T13:31:37.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.006+0000 7f3ae2000640 1 --2- 172.21.15.196:0/1812486962 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3abc06c8f0 0x7f3abc06edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.006+0000 7f3ae2000640 1 --2- 172.21.15.196:0/1812486962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3adc1468f0 0x7f3adc0731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.006+0000 7f3ae2000640 1 --2- 172.21.15.196:0/1812486962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3adc0d19a0 0x7f3adc13af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.006+0000 7f3ae2000640 1 --2- 172.21.15.196:0/1812486962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3adc0b41f0 0x7f3adc0b0dc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.006+0000 7f3ae2000640 1 -- 172.21.15.196:0/1812486962 >> 172.21.15.196:0/1812486962 conn(0x7f3adc06d3b0 msgr2=0x7f3adc142a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:37.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.006+0000 7f3ae2000640 1 -- 172.21.15.196:0/1812486962 shutdown_connections 2024-04-28T13:31:37.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.006+0000 7f3ae2000640 1 -- 172.21.15.196:0/1812486962 wait complete. 2024-04-28T13:31:37.025 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:37.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.215+0000 7f6973e00640 1 Processor -- start 2024-04-28T13:31:37.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.215+0000 7f6973e00640 1 -- start start 2024-04-28T13:31:37.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.216+0000 7f6973e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f69740b5ee0 0x7f69740b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:37.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.217+0000 7f6973e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f69740d1760 0x7f697406df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:37.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.217+0000 7f6973e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f697415e380 con 0x7f697409ec80 2024-04-28T13:31:37.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.217+0000 7f6973e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f69740b4b20 con 0x7f69740b5ee0 2024-04-28T13:31:37.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.217+0000 7f6973e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f69740b3f30 con 0x7f69740d1760 2024-04-28T13:31:37.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.217+0000 7f6972a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f69740b5ee0 0x7f69740b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:37.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.217+0000 7f6972a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f69740b5ee0 0x7f69740b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56538/0 (socket says 172.21.15.196:56538) 2024-04-28T13:31:37.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.217+0000 7f6972a00640 1 -- 172.21.15.196:0/2730015981 learned_addr learned my addr 172.21.15.196:0/2730015981 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:37.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.217+0000 7f6972a00640 1 -- 172.21.15.196:0/2730015981 >> v2:172.21.15.203:3300/0 conn(0x7f69740d1760 msgr2=0x7f697406df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:37.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.217+0000 7f6973400640 1 --2- 172.21.15.196:0/2730015981 >> v2:172.21.15.203:3300/0 conn(0x7f69740d1760 0x7f697406df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:37.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.217+0000 7f6972a00640 1 --2- 172.21.15.196:0/2730015981 >> v2:172.21.15.203:3300/0 conn(0x7f69740d1760 0x7f697406df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.217+0000 7f6972a00640 1 -- 172.21.15.196:0/2730015981 >> v1:172.21.15.203:6789/0 conn(0x7f697409ec80 legacy=0x7f69740d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:37.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.217+0000 7f6972a00640 1 -- 172.21.15.196:0/2730015981 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69740b41f0 con 0x7f69740b5ee0 2024-04-28T13:31:37.220 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.218+0000 7f6972a00640 1 --2- 172.21.15.196:0/2730015981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69740b5ee0 0x7f69740b4f40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6974074210 tx=0x7f696402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53311bb52d60fd40 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:37.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.219+0000 7f6971600640 1 -- 172.21.15.196:0/2730015981 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964037070 con 0x7f69740b5ee0 2024-04-28T13:31:37.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.219+0000 7f6971600640 1 -- 172.21.15.196:0/2730015981 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6964032440 con 0x7f69740b5ee0 2024-04-28T13:31:37.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.219+0000 7f6971600640 1 -- 172.21.15.196:0/2730015981 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964032810 con 0x7f69740b5ee0 2024-04-28T13:31:37.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.219+0000 7f6973e00640 1 -- 172.21.15.196:0/2730015981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69740b5ee0 msgr2=0x7f69740b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:37.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.219+0000 7f6973e00640 1 --2- 172.21.15.196:0/2730015981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69740b5ee0 0x7f69740b4f40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6974074210 tx=0x7f696402a230 comp rx=0 tx=0).stop 2024-04-28T13:31:37.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.219+0000 7f6973e00640 1 -- 172.21.15.196:0/2730015981 shutdown_connections 2024-04-28T13:31:37.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.219+0000 7f6973e00640 1 --2- 172.21.15.196:0/2730015981 >> v2:172.21.15.203:3300/0 conn(0x7f69740d1760 0x7f697406df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.219+0000 7f6973e00640 1 --2- 172.21.15.196:0/2730015981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69740b5ee0 0x7f69740b4f40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.219+0000 7f6973e00640 1 -- 172.21.15.196:0/2730015981 >> 172.21.15.196:0/2730015981 conn(0x7f697406c6c0 msgr2=0x7f697406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:37.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.219+0000 7f6973e00640 1 -- 172.21.15.196:0/2730015981 shutdown_connections 2024-04-28T13:31:37.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.219+0000 7f6973e00640 1 -- 172.21.15.196:0/2730015981 wait complete. 2024-04-28T13:31:37.222 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.221+0000 7f6973e00640 1 Processor -- start 2024-04-28T13:31:37.222 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.221+0000 7f6973e00640 1 -- start start 2024-04-28T13:31:37.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.222+0000 7f6973e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f697409ec80 0x7f697409e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:37.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.222+0000 7f6973e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69740b5ee0 0x7f69740caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:37.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.222+0000 7f6973e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69740d1760 0x7f697414b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:37.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.222+0000 7f6973e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f69740b3c20 con 0x7f69740b5ee0 2024-04-28T13:31:37.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.222+0000 7f6973e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f69740b47b0 con 0x7f69740d1760 2024-04-28T13:31:37.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.222+0000 7f6973e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f69740add70 con 0x7f697409ec80 2024-04-28T13:31:37.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.222+0000 7f6972000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69740b5ee0 0x7f69740caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:37.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.222+0000 7f6972000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69740b5ee0 0x7f69740caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56544/0 (socket says 172.21.15.196:56544) 2024-04-28T13:31:37.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.223+0000 7f6972000640 1 -- 172.21.15.196:0/832069981 learned_addr learned my addr 172.21.15.196:0/832069981 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:37.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.223+0000 7f6972a00640 1 --2- 172.21.15.196:0/832069981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f697409ec80 0x7f697409e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:37.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.223+0000 7f6973400640 1 --2- 172.21.15.196:0/832069981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69740d1760 0x7f697414b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:37.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.223+0000 7f6972000640 1 -- 172.21.15.196:0/832069981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f697409ec80 msgr2=0x7f697409e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:37.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.223+0000 7f6972000640 1 --2- 172.21.15.196:0/832069981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f697409ec80 0x7f697409e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.223+0000 7f6972000640 1 -- 172.21.15.196:0/832069981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69740d1760 msgr2=0x7f697414b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:37.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.223+0000 7f6972000640 1 --2- 172.21.15.196:0/832069981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69740d1760 0x7f697414b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.223+0000 7f6972000640 1 -- 172.21.15.196:0/832069981 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69740cb2c0 con 0x7f69740b5ee0 2024-04-28T13:31:37.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.223+0000 7f6972000640 1 --2- 172.21.15.196:0/832069981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69740b5ee0 0x7f69740caa50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f696000c430 tx=0x7f696000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:37.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.223+0000 7f696b400640 1 -- 172.21.15.196:0/832069981 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960017070 con 0x7f69740b5ee0 2024-04-28T13:31:37.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.223+0000 7f696b400640 1 -- 172.21.15.196:0/832069981 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69600035f0 con 0x7f69740b5ee0 2024-04-28T13:31:37.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.223+0000 7f696b400640 1 -- 172.21.15.196:0/832069981 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69600039e0 con 0x7f69740b5ee0 2024-04-28T13:31:37.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.223+0000 7f6973e00640 1 -- 172.21.15.196:0/832069981 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69740cb5a0 con 0x7f69740b5ee0 2024-04-28T13:31:37.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.224+0000 7f6973e00640 1 -- 172.21.15.196:0/832069981 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69740cbb60 con 0x7f69740b5ee0 2024-04-28T13:31:37.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.225+0000 7f6973e00640 1 -- 172.21.15.196:0/832069981 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697406a290 con 0x7f69740b5ee0 2024-04-28T13:31:37.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.226+0000 7f696b400640 1 -- 172.21.15.196:0/832069981 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6960003b80 con 0x7f69740b5ee0 2024-04-28T13:31:37.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.226+0000 7f696b400640 1 --2- 172.21.15.196:0/832069981 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f695406c920 0x7f695406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:37.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.227+0000 7f696b400640 1 -- 172.21.15.196:0/832069981 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f696008af00 con 0x7f69740b5ee0 2024-04-28T13:31:37.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.227+0000 7f6972a00640 1 --2- 172.21.15.196:0/832069981 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f695406c920 0x7f695406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:37.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.228+0000 7f6972a00640 1 --2- 172.21.15.196:0/832069981 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f695406c920 0x7f695406ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6964034000 tx=0x7f696402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:37.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.231+0000 7f696b400640 1 -- 172.21.15.196:0/832069981 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6960059330 con 0x7f69740b5ee0 2024-04-28T13:31:37.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.424+0000 7f6973e00640 1 -- 172.21.15.196:0/832069981 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69740008d0 con 0x7f69740b5ee0 2024-04-28T13:31:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.427+0000 7f696b400640 1 -- 172.21.15.196:0/832069981 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f6960058c30 con 0x7f69740b5ee0 2024-04-28T13:31:37.428 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:37.428 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":20,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:22.624564+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:37.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.429+0000 7f6973e00640 1 -- 172.21.15.196:0/832069981 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f695406c920 msgr2=0x7f695406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:37.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.429+0000 7f6973e00640 1 --2- 172.21.15.196:0/832069981 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f695406c920 0x7f695406ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6964034000 tx=0x7f696402e000 comp rx=0 tx=0).stop 2024-04-28T13:31:37.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.429+0000 7f6973e00640 1 -- 172.21.15.196:0/832069981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69740b5ee0 msgr2=0x7f69740caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:37.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.429+0000 7f6973e00640 1 --2- 172.21.15.196:0/832069981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69740b5ee0 0x7f69740caa50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f696000c430 tx=0x7f696000c8f0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.429+0000 7f6973e00640 1 -- 172.21.15.196:0/832069981 shutdown_connections 2024-04-28T13:31:37.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.429+0000 7f6973e00640 1 --2- 172.21.15.196:0/832069981 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f695406c920 0x7f695406ede0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.429+0000 7f6973e00640 1 --2- 172.21.15.196:0/832069981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69740d1760 0x7f697414b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.429+0000 7f6973e00640 1 --2- 172.21.15.196:0/832069981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69740b5ee0 0x7f69740caa50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.429+0000 7f6973e00640 1 --2- 172.21.15.196:0/832069981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f697409ec80 0x7f697409e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.429+0000 7f6973e00640 1 -- 172.21.15.196:0/832069981 >> 172.21.15.196:0/832069981 conn(0x7f697406c6c0 msgr2=0x7f69740cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:37.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.429+0000 7f6973e00640 1 -- 172.21.15.196:0/832069981 shutdown_connections 2024-04-28T13:31:37.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.429+0000 7f6973e00640 1 -- 172.21.15.196:0/832069981 wait complete. 2024-04-28T13:31:37.449 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:31:37.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.641+0000 7f0315a00640 1 Processor -- start 2024-04-28T13:31:37.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.641+0000 7f0315a00640 1 -- start start 2024-04-28T13:31:37.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.642+0000 7f0315a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03101469f0 0x7f03100ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:37.644 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.642+0000 7f0315a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03100cd420 0x7f03100d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:37.644 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.643+0000 7f0315a00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0310160f40 con 0x7f03100d1240 2024-04-28T13:31:37.644 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.643+0000 7f0315a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f03100b5960 con 0x7f03101469f0 2024-04-28T13:31:37.644 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.643+0000 7f0315a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0310161ac0 con 0x7f03100cd420 2024-04-28T13:31:37.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.643+0000 7f030ea00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03101469f0 0x7f03100ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:37.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.643+0000 7f030e000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03100cd420 0x7f03100d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:37.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.643+0000 7f030ea00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03101469f0 0x7f03100ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:55980/0 (socket says 172.21.15.196:55980) 2024-04-28T13:31:37.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.643+0000 7f030ea00640 1 -- 172.21.15.196:0/1797440448 learned_addr learned my addr 172.21.15.196:0/1797440448 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:37.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.644+0000 7f030e000640 1 -- 172.21.15.196:0/1797440448 >> v2:172.21.15.203:3300/0 conn(0x7f03101469f0 msgr2=0x7f03100ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:37.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.644+0000 7f030e000640 1 --2- 172.21.15.196:0/1797440448 >> v2:172.21.15.203:3300/0 conn(0x7f03101469f0 0x7f03100ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.644+0000 7f030e000640 1 -- 172.21.15.196:0/1797440448 >> v1:172.21.15.203:6789/0 conn(0x7f03100d1240 legacy=0x7f031006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:37.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.644+0000 7f030e000640 1 -- 172.21.15.196:0/1797440448 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0310161220 con 0x7f03100cd420 2024-04-28T13:31:37.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.645+0000 7f030e000640 1 --2- 172.21.15.196:0/1797440448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03100cd420 0x7f03100d0d00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0097a0 tx=0x7f02fc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ded546500ca4f9d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:37.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.646+0000 7f030d600640 1 -- 172.21.15.196:0/1797440448 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc037070 con 0x7f03100cd420 2024-04-28T13:31:37.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.646+0000 7f030d600640 1 -- 172.21.15.196:0/1797440448 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02fc032440 con 0x7f03100cd420 2024-04-28T13:31:37.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.646+0000 7f030d600640 1 -- 172.21.15.196:0/1797440448 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc032810 con 0x7f03100cd420 2024-04-28T13:31:37.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.647+0000 7f0315a00640 1 -- 172.21.15.196:0/1797440448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03100cd420 msgr2=0x7f03100d0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:37.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.647+0000 7f0315a00640 1 --2- 172.21.15.196:0/1797440448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03100cd420 0x7f03100d0d00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0097a0 tx=0x7f02fc02a180 comp rx=0 tx=0).stop 2024-04-28T13:31:37.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.647+0000 7f0315a00640 1 -- 172.21.15.196:0/1797440448 shutdown_connections 2024-04-28T13:31:37.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.647+0000 7f0315a00640 1 --2- 172.21.15.196:0/1797440448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03100cd420 0x7f03100d0d00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.647+0000 7f0315a00640 1 --2- 172.21.15.196:0/1797440448 >> v2:172.21.15.203:3300/0 conn(0x7f03101469f0 0x7f03100ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.647+0000 7f0315a00640 1 -- 172.21.15.196:0/1797440448 >> 172.21.15.196:0/1797440448 conn(0x7f03100cd830 msgr2=0x7f03100cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:37.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.647+0000 7f0315a00640 1 -- 172.21.15.196:0/1797440448 shutdown_connections 2024-04-28T13:31:37.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.647+0000 7f0315a00640 1 -- 172.21.15.196:0/1797440448 wait complete. 2024-04-28T13:31:37.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.649+0000 7f0315a00640 1 Processor -- start 2024-04-28T13:31:37.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.649+0000 7f0315a00640 1 -- start start 2024-04-28T13:31:37.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f0315a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03100cd420 0x7f031015c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:37.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f0315a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03100d1240 0x7f031015f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:37.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f0315a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03101469f0 0x7f031015f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:37.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f0315a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0310072020 con 0x7f03100cd420 2024-04-28T13:31:37.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f0315a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f03101617e0 con 0x7f03100d1240 2024-04-28T13:31:37.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f0315a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f03100b6de0 con 0x7f03101469f0 2024-04-28T13:31:37.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f030ea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03100cd420 0x7f031015c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:37.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f030ea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03100cd420 0x7f031015c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56552/0 (socket says 172.21.15.196:56552) 2024-04-28T13:31:37.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f030ea00640 1 -- 172.21.15.196:0/3372856946 learned_addr learned my addr 172.21.15.196:0/3372856946 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:37.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f030e000640 1 --2- 172.21.15.196:0/3372856946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03100d1240 0x7f031015f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:37.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f030f400640 1 --2- 172.21.15.196:0/3372856946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03101469f0 0x7f031015f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:37.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f030ea00640 1 -- 172.21.15.196:0/3372856946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03101469f0 msgr2=0x7f031015f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:37.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f030ea00640 1 --2- 172.21.15.196:0/3372856946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03101469f0 0x7f031015f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f030ea00640 1 -- 172.21.15.196:0/3372856946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03100d1240 msgr2=0x7f031015f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:37.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f030ea00640 1 --2- 172.21.15.196:0/3372856946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03100d1240 0x7f031015f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f030ea00640 1 -- 172.21.15.196:0/3372856946 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031015cdc0 con 0x7f03100cd420 2024-04-28T13:31:37.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.650+0000 7f030f400640 1 --2- 172.21.15.196:0/3372856946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03101469f0 0x7f031015f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:37.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.651+0000 7f030ea00640 1 --2- 172.21.15.196:0/3372856946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03100cd420 0x7f031015c880 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f030000e840 tx=0x7f030000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:37.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.651+0000 7f0306a00640 1 -- 172.21.15.196:0/3372856946 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300017070 con 0x7f03100cd420 2024-04-28T13:31:37.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.651+0000 7f0306a00640 1 -- 172.21.15.196:0/3372856946 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f030000c8e0 con 0x7f03100cd420 2024-04-28T13:31:37.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.651+0000 7f0306a00640 1 -- 172.21.15.196:0/3372856946 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030000ccd0 con 0x7f03100cd420 2024-04-28T13:31:37.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.651+0000 7f0315a00640 1 -- 172.21.15.196:0/3372856946 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031015e2c0 con 0x7f03100cd420 2024-04-28T13:31:37.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.651+0000 7f0315a00640 1 -- 172.21.15.196:0/3372856946 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f031015e880 con 0x7f03100cd420 2024-04-28T13:31:37.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.652+0000 7f0315a00640 1 -- 172.21.15.196:0/3372856946 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f031006a290 con 0x7f03100cd420 2024-04-28T13:31:37.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.653+0000 7f0306a00640 1 -- 172.21.15.196:0/3372856946 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f030001a070 con 0x7f03100cd420 2024-04-28T13:31:37.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.655+0000 7f0306a00640 1 --2- 172.21.15.196:0/3372856946 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f02f006c8f0 0x7f02f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:37.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.655+0000 7f0306a00640 1 -- 172.21.15.196:0/3372856946 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f030008b9c0 con 0x7f03100cd420 2024-04-28T13:31:37.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.655+0000 7f030e000640 1 --2- 172.21.15.196:0/3372856946 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f02f006c8f0 0x7f02f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:37.658 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.657+0000 7f030e000640 1 --2- 172.21.15.196:0/3372856946 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f02f006c8f0 0x7f02f006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f02fc034000 tx=0x7f02fc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:37.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.659+0000 7f0306a00640 1 -- 172.21.15.196:0/3372856946 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f030008de20 con 0x7f03100cd420 2024-04-28T13:31:37.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.893+0000 7f0315a00640 1 -- 172.21.15.196:0/3372856946 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f031006fd90 con 0x7f03100cd420 2024-04-28T13:31:37.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.894+0000 7f0306a00640 1 -- 172.21.15.196:0/3372856946 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f031015cdc0 con 0x7f03100cd420 2024-04-28T13:31:37.895 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:37.895 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/422653807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":422653807},{"type":"v1","addr":"172.21.15.196:6834","nonce":422653807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2793552376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2793552376},{"type":"v1","addr":"172.21.15.196:6835","nonce":2793552376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1810956193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1810956193},{"type":"v1","addr":"172.21.15.203:6837","nonce":1810956193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:31:21.059586+0000","modified":"2024-04-28T13:31:24.132134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/2114798853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2114798853},{"type":"v1","addr":"172.21.15.203:6836","nonce":2114798853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4253,"qdb_cluster":[4253]},"id":1}]} 2024-04-28T13:31:37.895 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 7 2024-04-28T13:31:37.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.896+0000 7f0315a00640 1 -- 172.21.15.196:0/3372856946 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f02f006c8f0 msgr2=0x7f02f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:37.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.896+0000 7f0315a00640 1 --2- 172.21.15.196:0/3372856946 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f02f006c8f0 0x7f02f006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f02fc034000 tx=0x7f02fc02e000 comp rx=0 tx=0).stop 2024-04-28T13:31:37.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.896+0000 7f0315a00640 1 -- 172.21.15.196:0/3372856946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03100cd420 msgr2=0x7f031015c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:37.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.897+0000 7f0315a00640 1 --2- 172.21.15.196:0/3372856946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03100cd420 0x7f031015c880 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f030000e840 tx=0x7f030000ed00 comp rx=0 tx=0).stop 2024-04-28T13:31:37.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.897+0000 7f0315a00640 1 -- 172.21.15.196:0/3372856946 shutdown_connections 2024-04-28T13:31:37.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.897+0000 7f0315a00640 1 --2- 172.21.15.196:0/3372856946 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f02f006c8f0 0x7f02f006edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.897+0000 7f0315a00640 1 --2- 172.21.15.196:0/3372856946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03101469f0 0x7f031015f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.897+0000 7f0315a00640 1 --2- 172.21.15.196:0/3372856946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03100d1240 0x7f031015f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.897+0000 7f0315a00640 1 --2- 172.21.15.196:0/3372856946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03100cd420 0x7f031015c880 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:37.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.897+0000 7f0315a00640 1 -- 172.21.15.196:0/3372856946 >> 172.21.15.196:0/3372856946 conn(0x7f03100cd830 msgr2=0x7f031006e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:37.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.897+0000 7f0315a00640 1 -- 172.21.15.196:0/3372856946 shutdown_connections 2024-04-28T13:31:37.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:37.897+0000 7f0315a00640 1 -- 172.21.15.196:0/3372856946 wait complete. 2024-04-28T13:31:38.317 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-28T13:31:21.059586+0000', 'modified': '2024-04-28T13:31:24.132134+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4253}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4253': {'gid': 4253, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6836/2114798853', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 2114798853}, {'type': 'v1', 'addr': '172.21.15.203:6836', 'nonce': 2114798853}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4253, 'qdb_cluster': [4253]} 2024-04-28T13:31:38.317 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-28T13:31:38.317 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-28T13:31:38.527 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.526+0000 7f085a400640 1 Processor -- start 2024-04-28T13:31:38.527 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.526+0000 7f085a400640 1 -- start start 2024-04-28T13:31:38.529 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.528+0000 7f085a400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f08541629c0 0x7f08540d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.529 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.528+0000 7f085a400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0854163bf0 0x7f0854163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.529 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.528+0000 7f085a400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0854162da0 con 0x7f0854163570 2024-04-28T13:31:38.529 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.528+0000 7f085a400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f08540b6190 con 0x7f0854163bf0 2024-04-28T13:31:38.529 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.528+0000 7f085a400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f08540b6470 con 0x7f08541629c0 2024-04-28T13:31:38.529 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.528+0000 7f084be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0854163bf0 0x7f0854163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.528+0000 7f0859a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f0854163570 0x7f08540b46b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:59244/0 (socket says 172.21.15.196:59244) 2024-04-28T13:31:38.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.528+0000 7f084be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0854163bf0 0x7f0854163030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56560/0 (socket says 172.21.15.196:56560) 2024-04-28T13:31:38.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.528+0000 7f0859a00640 1 -- 172.21.15.196:0/868827730 learned_addr learned my addr 172.21.15.196:0/868827730 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:38.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.529+0000 7f0859000640 1 --2- 172.21.15.196:0/868827730 >> v2:172.21.15.203:3300/0 conn(0x7f08541629c0 0x7f08540d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.529+0000 7f084be00640 1 -- 172.21.15.196:0/868827730 >> v2:172.21.15.203:3300/0 conn(0x7f08541629c0 msgr2=0x7f08540d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:38.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.529+0000 7f084be00640 1 --2- 172.21.15.196:0/868827730 >> v2:172.21.15.203:3300/0 conn(0x7f08541629c0 0x7f08540d3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.529+0000 7f084be00640 1 -- 172.21.15.196:0/868827730 >> v1:172.21.15.196:6789/0 conn(0x7f0854163570 legacy=0x7f08540b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:38.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.529+0000 7f084be00640 1 -- 172.21.15.196:0/868827730 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08540b6750 con 0x7f0854163bf0 2024-04-28T13:31:38.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.530+0000 7f084be00640 1 --2- 172.21.15.196:0/868827730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854163bf0 0x7f0854163030 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f085408f8a0 tx=0x7f083c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a749a22689454b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:38.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.530+0000 7f0853e00640 1 -- 172.21.15.196:0/868827730 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c037070 con 0x7f0854163bf0 2024-04-28T13:31:38.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.530+0000 7f0853e00640 1 -- 172.21.15.196:0/868827730 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083c032440 con 0x7f0854163bf0 2024-04-28T13:31:38.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.530+0000 7f0853e00640 1 -- 172.21.15.196:0/868827730 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c032810 con 0x7f0854163bf0 2024-04-28T13:31:38.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.531+0000 7f085a400640 1 -- 172.21.15.196:0/868827730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854163bf0 msgr2=0x7f0854163030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:38.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.531+0000 7f085a400640 1 --2- 172.21.15.196:0/868827730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854163bf0 0x7f0854163030 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f085408f8a0 tx=0x7f083c02a1a0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.531+0000 7f085a400640 1 -- 172.21.15.196:0/868827730 shutdown_connections 2024-04-28T13:31:38.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.531+0000 7f085a400640 1 --2- 172.21.15.196:0/868827730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854163bf0 0x7f0854163030 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.531+0000 7f085a400640 1 --2- 172.21.15.196:0/868827730 >> v2:172.21.15.203:3300/0 conn(0x7f08541629c0 0x7f08540d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.531+0000 7f085a400640 1 -- 172.21.15.196:0/868827730 >> 172.21.15.196:0/868827730 conn(0x7f085409e570 msgr2=0x7f08540d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:38.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.531+0000 7f085a400640 1 -- 172.21.15.196:0/868827730 shutdown_connections 2024-04-28T13:31:38.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.531+0000 7f085a400640 1 -- 172.21.15.196:0/868827730 wait complete. 2024-04-28T13:31:38.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.532+0000 7f085a400640 1 Processor -- start 2024-04-28T13:31:38.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.533+0000 7f085a400640 1 -- start start 2024-04-28T13:31:38.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.534+0000 7f085a400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08541629c0 0x7f08540cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.534+0000 7f085a400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854163570 0x7f08540ba730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.534+0000 7f085a400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854163bf0 0x7f08540cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.534+0000 7f085a400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08540d3c50 con 0x7f0854163bf0 2024-04-28T13:31:38.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.534+0000 7f085a400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f08540b1e60 con 0x7f0854163570 2024-04-28T13:31:38.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.534+0000 7f085a400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f08540d3dd0 con 0x7f08541629c0 2024-04-28T13:31:38.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.534+0000 7f0859a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854163bf0 0x7f08540cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.534+0000 7f0859a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854163bf0 0x7f08540cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56576/0 (socket says 172.21.15.196:56576) 2024-04-28T13:31:38.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.534+0000 7f0859a00640 1 -- 172.21.15.196:0/3078743333 learned_addr learned my addr 172.21.15.196:0/3078743333 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:38.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.534+0000 7f084be00640 1 --2- 172.21.15.196:0/3078743333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854163570 0x7f08540ba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.534+0000 7f0859000640 1 --2- 172.21.15.196:0/3078743333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08541629c0 0x7f08540cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.534+0000 7f0859a00640 1 -- 172.21.15.196:0/3078743333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08541629c0 msgr2=0x7f08540cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:38.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.534+0000 7f0859a00640 1 --2- 172.21.15.196:0/3078743333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08541629c0 0x7f08540cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.534+0000 7f0859a00640 1 -- 172.21.15.196:0/3078743333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854163570 msgr2=0x7f08540ba730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:38.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.534+0000 7f0859a00640 1 --2- 172.21.15.196:0/3078743333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854163570 0x7f08540ba730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.535+0000 7f0859a00640 1 -- 172.21.15.196:0/3078743333 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08540d1c90 con 0x7f0854163bf0 2024-04-28T13:31:38.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.535+0000 7f0859a00640 1 --2- 172.21.15.196:0/3078743333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854163bf0 0x7f08540cfa20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f084c00cbc0 tx=0x7f084c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:38.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.535+0000 7f0851600640 1 -- 172.21.15.196:0/3078743333 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c017070 con 0x7f0854163bf0 2024-04-28T13:31:38.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.535+0000 7f0851600640 1 -- 172.21.15.196:0/3078743333 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c00ad40 con 0x7f0854163bf0 2024-04-28T13:31:38.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.535+0000 7f0851600640 1 -- 172.21.15.196:0/3078743333 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c012600 con 0x7f0854163bf0 2024-04-28T13:31:38.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.535+0000 7f085a400640 1 -- 172.21.15.196:0/3078743333 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08540d1f70 con 0x7f0854163bf0 2024-04-28T13:31:38.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.535+0000 7f085a400640 1 -- 172.21.15.196:0/3078743333 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08540ce900 con 0x7f0854163bf0 2024-04-28T13:31:38.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.537+0000 7f0851600640 1 -- 172.21.15.196:0/3078743333 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f084c01a070 con 0x7f0854163bf0 2024-04-28T13:31:38.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.538+0000 7f0851600640 1 --2- 172.21.15.196:0/3078743333 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f083006c8f0 0x7f083006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.538+0000 7f0851600640 1 -- 172.21.15.196:0/3078743333 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f084c08b820 con 0x7f0854163bf0 2024-04-28T13:31:38.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.539+0000 7f0859000640 1 --2- 172.21.15.196:0/3078743333 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f083006c8f0 0x7f083006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.539+0000 7f085e87f740 1 Processor -- start 2024-04-28T13:31:38.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.539+0000 7f085e87f740 1 -- start start 2024-04-28T13:31:38.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.540+0000 7f085e87f740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6d6f7b0 0x55c6c6d71910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.540+0000 7f085e87f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6dabe80 0x55c6c6d6fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.540+0000 7f085e87f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 0x55c6c6d9a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.540+0000 7f085e87f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c6c6cd3600 con 0x55c6c6dabe80 2024-04-28T13:31:38.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.540+0000 7f085e87f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c6c6c70100 con 0x55c6c6d9a270 2024-04-28T13:31:38.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.540+0000 7f085e87f740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55c6c6ccf5c0 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.540+0000 7f0859000640 1 --2- 172.21.15.196:0/3078743333 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f083006c8f0 0x7f083006edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0844006fd0 tx=0x7f0844008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:38.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.540+0000 7f084be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6dabe80 0x55c6c6d6fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.540+0000 7f084be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6dabe80 0x55c6c6d6fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56580/0 (socket says 172.21.15.196:56580) 2024-04-28T13:31:38.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.540+0000 7f084be00640 1 -- 172.21.15.196:0/3241057760 learned_addr learned my addr 172.21.15.196:0/3241057760 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:38.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.541+0000 7f0859000640 1 --2- 172.21.15.196:0/3241057760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6d6f7b0 0x55c6c6d71910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.541+0000 7f0859a00640 1 --2- 172.21.15.196:0/3241057760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 0x55c6c6d9a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.541+0000 7f084be00640 1 -- 172.21.15.196:0/3241057760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6d6f7b0 msgr2=0x55c6c6d71910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:38.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.541+0000 7f084be00640 1 --2- 172.21.15.196:0/3241057760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6d6f7b0 0x55c6c6d71910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.541+0000 7f084be00640 1 -- 172.21.15.196:0/3241057760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 msgr2=0x55c6c6d9a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:38.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.541+0000 7f084be00640 1 --2- 172.21.15.196:0/3241057760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 0x55c6c6d9a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.541+0000 7f084be00640 1 -- 172.21.15.196:0/3241057760 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c69e52d0 con 0x55c6c6dabe80 2024-04-28T13:31:38.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.541+0000 7f084be00640 1 --2- 172.21.15.196:0/3241057760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6dabe80 0x55c6c6d6fb90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f083c009890 tx=0x7f083c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ef23e4d68104f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:38.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.541+0000 7f0849600640 1 -- 172.21.15.196:0/3241057760 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c037040 con 0x55c6c6dabe80 2024-04-28T13:31:38.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.541+0000 7f0849600640 1 -- 172.21.15.196:0/3241057760 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083c03e720 con 0x55c6c6dabe80 2024-04-28T13:31:38.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.541+0000 7f0849600640 1 -- 172.21.15.196:0/3241057760 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c03ead0 con 0x55c6c6dabe80 2024-04-28T13:31:38.547 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.542+0000 7f085e87f740 1 -- 172.21.15.196:0/3241057760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6dabe80 msgr2=0x55c6c6d6fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:38.547 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.542+0000 7f085e87f740 1 --2- 172.21.15.196:0/3241057760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6dabe80 0x55c6c6d6fb90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f083c009890 tx=0x7f083c02c000 comp rx=0 tx=0).stop 2024-04-28T13:31:38.547 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.542+0000 7f085e87f740 1 -- 172.21.15.196:0/3241057760 shutdown_connections 2024-04-28T13:31:38.547 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.542+0000 7f085e87f740 1 --2- 172.21.15.196:0/3241057760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 0x55c6c6d9a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.547 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.542+0000 7f085e87f740 1 --2- 172.21.15.196:0/3241057760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6dabe80 0x55c6c6d6fb90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.547 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.542+0000 7f085e87f740 1 --2- 172.21.15.196:0/3241057760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6d6f7b0 0x55c6c6d71910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.547 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.542+0000 7f085e87f740 1 -- 172.21.15.196:0/3241057760 >> 172.21.15.196:0/3241057760 conn(0x55c6c6d03d10 msgr2=0x55c6c6db5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:38.547 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.542+0000 7f085e87f740 1 -- 172.21.15.196:0/3241057760 shutdown_connections 2024-04-28T13:31:38.547 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.542+0000 7f085e87f740 1 -- 172.21.15.196:0/3241057760 wait complete. 2024-04-28T13:31:38.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.543+0000 7f085e87f740 1 -- start start 2024-04-28T13:31:38.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.543+0000 7f085e87f740 1 Processor -- start 2024-04-28T13:31:38.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.544+0000 7f085e87f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6d95910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.544+0000 7f085e87f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 0x55c6c6d9cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.544+0000 7f085e87f740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6dabe80 0x55c6c6d9d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.544+0000 7f085e87f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c6c6c5af70 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.544+0000 7f085e87f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c6c6d97dc0 con 0x55c6c6d9a270 2024-04-28T13:31:38.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.544+0000 7f085e87f740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55c6c6dc60b0 con 0x55c6c6dabe80 2024-04-28T13:31:38.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.544+0000 7f085e87f740 10 client.0 _ll_register_callbacks cb 0x55c6c6cc9480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T13:31:38.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.544+0000 7f0859000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6d95910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.544+0000 7f0859000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6d95910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56588/0 (socket says 172.21.15.196:56588) 2024-04-28T13:31:38.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.544+0000 7f0859000640 1 -- 172.21.15.196:0/3716642894 learned_addr learned my addr 172.21.15.196:0/3716642894 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:38.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.545+0000 7f084be00640 1 --2- 172.21.15.196:0/3716642894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 0x55c6c6d9cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.545+0000 7f0859a00640 1 --2- 172.21.15.196:0/3716642894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6dabe80 0x55c6c6d9d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.545+0000 7f0859000640 1 -- 172.21.15.196:0/3716642894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6dabe80 msgr2=0x55c6c6d9d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:38.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.545+0000 7f0859000640 1 --2- 172.21.15.196:0/3716642894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6dabe80 0x55c6c6d9d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.545+0000 7f0859000640 1 -- 172.21.15.196:0/3716642894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 msgr2=0x55c6c6d9cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:38.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.545+0000 7f0859000640 1 --2- 172.21.15.196:0/3716642894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 0x55c6c6d9cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.545+0000 7f0859000640 1 -- 172.21.15.196:0/3716642894 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c6aaf6b0 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.545+0000 7f084be00640 1 --2- 172.21.15.196:0/3716642894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 0x55c6c6d9cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:38.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.545+0000 7f0859a00640 1 --2- 172.21.15.196:0/3716642894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6dabe80 0x55c6c6d9d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:38.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.545+0000 7f0859000640 1 --2- 172.21.15.196:0/3716642894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6d95910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f084400b550 tx=0x7f0844004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39766d3db6230c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:38.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.545+0000 7f0842000640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-04-28T13:31:38.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.545+0000 7f0842000640 1 -- 172.21.15.196:0/3716642894 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844025020 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.545+0000 7f0842000640 1 -- 172.21.15.196:0/3716642894 <== mon.0 v2:172.21.15.196:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f08440218f0 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.545+0000 7f085e87f740 1 -- 172.21.15.196:0/3716642894 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f083c03e720 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.546+0000 7f0842000640 1 -- 172.21.15.196:0/3716642894 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f084402f050 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.546+0000 7f085e87f740 10 client.4638 fetch_fsmap learned FSMap version 7 2024-04-28T13:31:38.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.546+0000 7f085e87f740 1 -- 172.21.15.196:0/3716642894 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55c6c6dc5df0 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.546+0000 7f0842000640 1 -- 172.21.15.196:0/3716642894 <== mon.0 v2:172.21.15.196:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083c03e720 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.546+0000 7f0842000640 1 -- 172.21.15.196:0/3716642894 <== mon.0 v2:172.21.15.196:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7f0844032020 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.546+0000 7f085e87f740 10 client.4638 fetch_fsmap finished waiting for FSMap version 7 2024-04-28T13:31:38.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.546+0000 7f085e87f740 10 client.4638 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T13:31:38.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.546+0000 7f085e87f740 20 client.4638 populate_metadata read hostname 'smithi196' 2024-04-28T13:31:38.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.546+0000 7f085e87f740 1 --2- 172.21.15.196:0/3716642894 >> [v2:172.21.15.203:6834/2114798853,v1:172.21.15.203:6836/2114798853] conn(0x55c6c6dd09e0 0x55c6c6dd2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.546+0000 7f085e87f740 4 client.4638 mds_command: new command op to 4253 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T13:31:38.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.546+0000 7f085e87f740 1 -- 172.21.15.196:0/3716642894 --> [v2:172.21.15.203:6834/2114798853,v1:172.21.15.203:6836/2114798853] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6c6ce1210 con 0x55c6c6dd09e0 2024-04-28T13:31:38.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.547+0000 7f084be00640 1 --2- 172.21.15.196:0/3716642894 >> [v2:172.21.15.203:6834/2114798853,v1:172.21.15.203:6836/2114798853] conn(0x55c6c6dd09e0 0x55c6c6dd2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.547+0000 7f084be00640 1 --2- 172.21.15.196:0/3716642894 >> [v2:172.21.15.203:6834/2114798853,v1:172.21.15.203:6836/2114798853] conn(0x55c6c6dd09e0 0x55c6c6dd2dd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c6a9cfa652640c5 server_cookie=73d2654fa4aab687 in_seq=0 out_seq=0 2024-04-28T13:31:38.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.547+0000 7f0842000640 10 client.4638 ms_handle_connect on v2:172.21.15.203:6834/2114798853 2024-04-28T13:31:38.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.550+0000 7f0842000640 1 -- 172.21.15.196:0/3716642894 <== mds.0 v2:172.21.15.203:6834/2114798853 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6c6ce1210 con 0x55c6c6dd09e0 2024-04-28T13:31:38.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.550+0000 7f0842000640 10 client.4638 handle_command_reply: tid=0 2024-04-28T13:31:38.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.550+0000 7f085e87f740 1 client.4638 shutdown 2024-04-28T13:31:38.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.550+0000 7f085e87f740 1 -- 172.21.15.196:0/3716642894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6d6f7b0 msgr2=0x55c6c6d95910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:38.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.550+0000 7f085e87f740 1 --2- 172.21.15.196:0/3716642894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6d95910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f084400b550 tx=0x7f0844004730 comp rx=0 tx=0).stop 2024-04-28T13:31:38.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.550+0000 7f085e87f740 1 -- 172.21.15.196:0/3716642894 shutdown_connections 2024-04-28T13:31:38.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.550+0000 7f085e87f740 1 --2- 172.21.15.196:0/3716642894 >> [v2:172.21.15.203:6834/2114798853,v1:172.21.15.203:6836/2114798853] conn(0x55c6c6dd09e0 0x55c6c6dd2dd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.550+0000 7f085e87f740 1 --2- 172.21.15.196:0/3716642894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6dabe80 0x55c6c6d9d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.550+0000 7f085e87f740 1 --2- 172.21.15.196:0/3716642894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 0x55c6c6d9cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.550+0000 7f085e87f740 1 --2- 172.21.15.196:0/3716642894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6d95910 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.550+0000 7f0842000640 0 client.4638 ms_handle_reset on v2:172.21.15.203:6834/2114798853 2024-04-28T13:31:38.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.550+0000 7f085e87f740 1 -- 172.21.15.196:0/3716642894 >> 172.21.15.196:0/3716642894 conn(0x55c6c6d03d10 msgr2=0x55c6c6db8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:38.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.550+0000 7f085e87f740 1 -- 172.21.15.196:0/3716642894 shutdown_connections 2024-04-28T13:31:38.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.550+0000 7f085e87f740 1 -- 172.21.15.196:0/3716642894 wait complete. 2024-04-28T13:31:38.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.551+0000 7f085e87f740 20 client.4638 trim_cache size 0 max 16384 2024-04-28T13:31:38.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.570+0000 7f085e87f740 1 Processor -- start 2024-04-28T13:31:38.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.570+0000 7f085e87f740 1 -- start start 2024-04-28T13:31:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f085e87f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6dce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f085e87f740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6d9a270 0x55c6c6dd8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f085e87f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6dabe80 0x55c6c6dd8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f085e87f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0854200690 con 0x55c6c6dabe80 2024-04-28T13:31:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f085e87f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f08541f9770 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f085e87f740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0854200040 con 0x55c6c6d9a270 2024-04-28T13:31:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f0859a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6dabe80 0x55c6c6dd8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f0859a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6dabe80 0x55c6c6dd8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56598/0 (socket says 172.21.15.196:56598) 2024-04-28T13:31:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f0859a00640 1 -- 172.21.15.196:0/676598168 learned_addr learned my addr 172.21.15.196:0/676598168 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f084be00640 1 --2- 172.21.15.196:0/676598168 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6d9a270 0x55c6c6dd8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f0859a00640 1 -- 172.21.15.196:0/676598168 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6d9a270 msgr2=0x55c6c6dd8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f0859000640 1 --2- 172.21.15.196:0/676598168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6dce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f0859a00640 1 --2- 172.21.15.196:0/676598168 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6d9a270 0x55c6c6dd8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f0859a00640 1 -- 172.21.15.196:0/676598168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d6f7b0 msgr2=0x55c6c6dce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f0859a00640 1 --2- 172.21.15.196:0/676598168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6dce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.571+0000 7f0859a00640 1 -- 172.21.15.196:0/676598168 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c6d6f3c0 con 0x55c6c6dabe80 2024-04-28T13:31:38.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.572+0000 7f0859a00640 1 --2- 172.21.15.196:0/676598168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6dabe80 0x55c6c6dd8790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f084c055a60 tx=0x7f084c05d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8545df4cc77905dc server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:38.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.572+0000 7f0848c00640 1 -- 172.21.15.196:0/676598168 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c08b040 con 0x55c6c6dabe80 2024-04-28T13:31:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.572+0000 7f0848c00640 1 -- 172.21.15.196:0/676598168 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c055c70 con 0x55c6c6dabe80 2024-04-28T13:31:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.572+0000 7f0848c00640 1 -- 172.21.15.196:0/676598168 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c061ea0 con 0x55c6c6dabe80 2024-04-28T13:31:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.572+0000 7f085e87f740 1 -- 172.21.15.196:0/676598168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6dabe80 msgr2=0x55c6c6dd8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.572+0000 7f085e87f740 1 --2- 172.21.15.196:0/676598168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6dabe80 0x55c6c6dd8790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f084c055a60 tx=0x7f084c05d9c0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.572+0000 7f085e87f740 1 -- 172.21.15.196:0/676598168 shutdown_connections 2024-04-28T13:31:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.572+0000 7f085e87f740 1 --2- 172.21.15.196:0/676598168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6dabe80 0x55c6c6dd8790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.572+0000 7f085e87f740 1 --2- 172.21.15.196:0/676598168 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6d9a270 0x55c6c6dd8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.572+0000 7f085e87f740 1 --2- 172.21.15.196:0/676598168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6dce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.572+0000 7f085e87f740 1 -- 172.21.15.196:0/676598168 >> 172.21.15.196:0/676598168 conn(0x55c6c6d03d10 msgr2=0x55c6c6d932b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.573+0000 7f085e87f740 1 -- 172.21.15.196:0/676598168 shutdown_connections 2024-04-28T13:31:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.573+0000 7f085e87f740 1 -- 172.21.15.196:0/676598168 wait complete. 2024-04-28T13:31:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.573+0000 7f085e87f740 1 -- start start 2024-04-28T13:31:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.573+0000 7f085e87f740 1 Processor -- start 2024-04-28T13:31:38.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.574+0000 7f085e87f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6e2ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.574+0000 7f085e87f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 0x55c6c6e32cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.574+0000 7f085e87f740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6dabe80 0x55c6c6dd8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.574+0000 7f085e87f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c6c6dadea0 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.574+0000 7f085e87f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c6c6dada00 con 0x55c6c6d9a270 2024-04-28T13:31:38.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.574+0000 7f085e87f740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55c6c6dc54e0 con 0x55c6c6dabe80 2024-04-28T13:31:38.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.574+0000 7f085e87f740 10 client.0 _ll_register_callbacks cb 0x7f084400bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T13:31:38.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.575+0000 7f0859000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6e2ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.575+0000 7f0859000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6e2ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56610/0 (socket says 172.21.15.196:56610) 2024-04-28T13:31:38.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.575+0000 7f0859000640 1 -- 172.21.15.196:0/2306663929 learned_addr learned my addr 172.21.15.196:0/2306663929 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:38.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.575+0000 7f0859000640 1 -- 172.21.15.196:0/2306663929 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6dabe80 msgr2=0x55c6c6dd8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:38.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.575+0000 7f0859000640 1 --2- 172.21.15.196:0/2306663929 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6dabe80 0x55c6c6dd8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.575+0000 7f0859000640 1 -- 172.21.15.196:0/2306663929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 msgr2=0x55c6c6e32cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.575+0000 7f084be00640 1 --2- 172.21.15.196:0/2306663929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 0x55c6c6e32cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.575+0000 7f0859000640 1 --2- 172.21.15.196:0/2306663929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 0x55c6c6e32cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.575+0000 7f0859000640 1 -- 172.21.15.196:0/2306663929 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c6d9c1f0 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.575+0000 7f0859000640 1 --2- 172.21.15.196:0/2306663929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6e2ee10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f084400b1d0 tx=0x7f0844014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b4ff42b8f1cecf server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.575+0000 7f0842000640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-04-28T13:31:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.575+0000 7f0842000640 1 -- 172.21.15.196:0/2306663929 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844025040 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.575+0000 7f0842000640 1 -- 172.21.15.196:0/2306663929 <== mon.0 v2:172.21.15.196:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0844005b90 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.575+0000 7f085e87f740 1 -- 172.21.15.196:0/2306663929 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f084c055c70 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.576+0000 7f0842000640 1 -- 172.21.15.196:0/2306663929 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f084403d050 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.576+0000 7f085e87f740 10 client.4644 fetch_fsmap learned FSMap version 7 2024-04-28T13:31:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.576+0000 7f085e87f740 1 -- 172.21.15.196:0/2306663929 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55c6c6e2f6c0 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.576+0000 7f0842000640 1 -- 172.21.15.196:0/2306663929 <== mon.0 v2:172.21.15.196:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c055c70 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.576+0000 7f0842000640 1 -- 172.21.15.196:0/2306663929 <== mon.0 v2:172.21.15.196:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7f0844040020 con 0x55c6c6d6f7b0 2024-04-28T13:31:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.576+0000 7f085e87f740 10 client.4644 fetch_fsmap finished waiting for FSMap version 7 2024-04-28T13:31:38.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.576+0000 7f085e87f740 10 client.4644 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T13:31:38.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.576+0000 7f085e87f740 20 client.4644 populate_metadata read hostname 'smithi196' 2024-04-28T13:31:38.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.576+0000 7f085e87f740 1 --2- 172.21.15.196:0/2306663929 >> [v2:172.21.15.203:6834/2114798853,v1:172.21.15.203:6836/2114798853] conn(0x55c6c6dd8760 0x55c6c6d98320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:38.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.576+0000 7f085e87f740 4 client.4644 mds_command: new command op to 4253 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T13:31:38.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.576+0000 7f085e87f740 1 -- 172.21.15.196:0/2306663929 --> [v2:172.21.15.203:6834/2114798853,v1:172.21.15.203:6836/2114798853] -- command(tid 0: {"prefix": "status"}) -- 0x55c6c6cdb080 con 0x55c6c6dd8760 2024-04-28T13:31:38.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.577+0000 7f084be00640 1 --2- 172.21.15.196:0/2306663929 >> [v2:172.21.15.203:6834/2114798853,v1:172.21.15.203:6836/2114798853] conn(0x55c6c6dd8760 0x55c6c6d98320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:38.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.578+0000 7f084be00640 1 --2- 172.21.15.196:0/2306663929 >> [v2:172.21.15.203:6834/2114798853,v1:172.21.15.203:6836/2114798853] conn(0x55c6c6dd8760 0x55c6c6d98320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1303d4be7536581 server_cookie=4e8cf9b1588758fb in_seq=0 out_seq=0 2024-04-28T13:31:38.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.578+0000 7f0842000640 10 client.4644 ms_handle_connect on v2:172.21.15.203:6834/2114798853 2024-04-28T13:31:38.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.578+0000 7f0842000640 1 -- 172.21.15.196:0/2306663929 <== mds.0 v2:172.21.15.203:6834/2114798853 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55c6c6cdb080 con 0x55c6c6dd8760 2024-04-28T13:31:38.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.578+0000 7f0842000640 10 client.4644 handle_command_reply: tid=0 2024-04-28T13:31:38.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.578+0000 7f085e87f740 1 client.4644 shutdown 2024-04-28T13:31:38.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.578+0000 7f085e87f740 1 -- 172.21.15.196:0/2306663929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6d6f7b0 msgr2=0x55c6c6e2ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:38.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.578+0000 7f085e87f740 1 --2- 172.21.15.196:0/2306663929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6e2ee10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f084400b1d0 tx=0x7f0844014000 comp rx=0 tx=0).stop 2024-04-28T13:31:38.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.578+0000 7f085e87f740 1 -- 172.21.15.196:0/2306663929 shutdown_connections 2024-04-28T13:31:38.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.578+0000 7f085e87f740 1 --2- 172.21.15.196:0/2306663929 >> [v2:172.21.15.203:6834/2114798853,v1:172.21.15.203:6836/2114798853] conn(0x55c6c6dd8760 0x55c6c6d98320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.578+0000 7f085e87f740 1 --2- 172.21.15.196:0/2306663929 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c6c6dabe80 0x55c6c6dd8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.578+0000 7f0842000640 0 client.4644 ms_handle_reset on v2:172.21.15.203:6834/2114798853 2024-04-28T13:31:38.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.578+0000 7f085e87f740 1 --2- 172.21.15.196:0/2306663929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c6c6d9a270 0x55c6c6e32cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.578+0000 7f085e87f740 1 --2- 172.21.15.196:0/2306663929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c6d6f7b0 0x55c6c6e2ee10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.578+0000 7f085e87f740 1 -- 172.21.15.196:0/2306663929 >> 172.21.15.196:0/2306663929 conn(0x55c6c6d03d10 msgr2=0x55c6c6dc45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:38.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.578+0000 7f085e87f740 1 -- 172.21.15.196:0/2306663929 shutdown_connections 2024-04-28T13:31:38.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.579+0000 7f085e87f740 1 -- 172.21.15.196:0/2306663929 wait complete. 2024-04-28T13:31:38.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.579+0000 7f085e87f740 20 client.4644 trim_cache size 0 max 16384 2024-04-28T13:31:38.582 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:31:38.582 INFO:teuthology.orchestra.run.smithi196.stdout: "cluster_fsid": "270aea32-2e16-4a97-85f7-aeae39bc8c76", 2024-04-28T13:31:38.583 INFO:teuthology.orchestra.run.smithi196.stdout: "whoami": 0, 2024-04-28T13:31:38.583 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 4253, 2024-04-28T13:31:38.583 INFO:teuthology.orchestra.run.smithi196.stdout: "want_state": "up:active", 2024-04-28T13:31:38.583 INFO:teuthology.orchestra.run.smithi196.stdout: "state": "up:active", 2024-04-28T13:31:38.583 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs", 2024-04-28T13:31:38.583 INFO:teuthology.orchestra.run.smithi196.stdout: "rank_uptime": 16.967102754999999, 2024-04-28T13:31:38.583 INFO:teuthology.orchestra.run.smithi196.stdout: "mdsmap_epoch": 7, 2024-04-28T13:31:38.583 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_epoch": 20, 2024-04-28T13:31:38.583 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_epoch_barrier": 20, 2024-04-28T13:31:38.583 INFO:teuthology.orchestra.run.smithi196.stdout: "uptime": 22.160081458000001 2024-04-28T13:31:38.583 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:31:38.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.579+0000 7f085a400640 1 -- 172.21.15.196:0/3078743333 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f083006c8f0 msgr2=0x7f083006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:38.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.579+0000 7f085a400640 1 --2- 172.21.15.196:0/3078743333 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f083006c8f0 0x7f083006edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0844006fd0 tx=0x7f0844008040 comp rx=0 tx=0).stop 2024-04-28T13:31:38.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.579+0000 7f085a400640 1 -- 172.21.15.196:0/3078743333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854163bf0 msgr2=0x7f08540cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:38.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.579+0000 7f085a400640 1 --2- 172.21.15.196:0/3078743333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854163bf0 0x7f08540cfa20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f084c00cbc0 tx=0x7f084c00a430 comp rx=0 tx=0).stop 2024-04-28T13:31:38.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.579+0000 7f085a400640 1 -- 172.21.15.196:0/3078743333 shutdown_connections 2024-04-28T13:31:38.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.579+0000 7f085a400640 1 --2- 172.21.15.196:0/3078743333 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f083006c8f0 0x7f083006edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.579+0000 7f085a400640 1 --2- 172.21.15.196:0/3078743333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854163bf0 0x7f08540cfa20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.579+0000 7f085a400640 1 --2- 172.21.15.196:0/3078743333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854163570 0x7f08540ba730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.579+0000 7f085a400640 1 --2- 172.21.15.196:0/3078743333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08541629c0 0x7f08540cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:38.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.579+0000 7f085a400640 1 -- 172.21.15.196:0/3078743333 >> 172.21.15.196:0/3078743333 conn(0x7f085409e570 msgr2=0x7f08540b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:38.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.579+0000 7f085a400640 1 -- 172.21.15.196:0/3078743333 shutdown_connections 2024-04-28T13:31:38.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:38.580+0000 7f085a400640 1 -- 172.21.15.196:0/3078743333 wait complete. 2024-04-28T13:31:38.597 INFO:teuthology.run_tasks:Running task sequential... 2024-04-28T13:31:38.607 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-28T13:31:38.607 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-28T13:31:38.608 INFO:teuthology.task.print:Enabling mgr modules 2024-04-28T13:31:38.608 INFO:teuthology.run_tasks:Running task kclient... 2024-04-28T13:31:38.634 INFO:tasks.kclient:Mounting kernel clients... 2024-04-28T13:31:38.634 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-28T13:31:38.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:38.634 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-28T13:31:38.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:38.687 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link delete ceph-brx 2024-04-28T13:31:38.774 INFO:teuthology.orchestra.run.smithi196.stderr:Cannot find device "ceph-brx" 2024-04-28T13:31:38.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:31:38.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:38.778 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-28T13:31:38.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:38.830 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link delete ceph-brx 2024-04-28T13:31:38.917 INFO:teuthology.orchestra.run.smithi196.stderr:Cannot find device "ceph-brx" 2024-04-28T13:31:38.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:31:38.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:38.921 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-28T13:31:38.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:38.980 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2024-04-28T13:31:39.072 INFO:teuthology.orchestra.run.smithi203.stderr:Cannot find device "ceph-brx" 2024-04-28T13:31:39.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:31:39.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:39.077 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-28T13:31:39.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:39.130 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2024-04-28T13:31:39.222 INFO:teuthology.orchestra.run.smithi203.stderr:Cannot find device "ceph-brx" 2024-04-28T13:31:39.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:31:39.226 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-28T13:31:39.226 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-28T13:31:39.227 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-28T13:31:39.227 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T13:31:39.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.434+0000 7fcde5200640 1 Processor -- start 2024-04-28T13:31:39.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.434+0000 7fcde5200640 1 -- start start 2024-04-28T13:31:39.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.435+0000 7fcde5200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcde015a9a0 0x7fcde00cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:39.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.436+0000 7fcde5200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcde00d1790 0x7fcde015d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:39.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.436+0000 7fcde5200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcde0160b00 con 0x7fcde00add70 2024-04-28T13:31:39.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.436+0000 7fcde5200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcde0160de0 con 0x7fcde015a9a0 2024-04-28T13:31:39.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.436+0000 7fcde5200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fcde01610c0 con 0x7fcde00d1790 2024-04-28T13:31:39.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.436+0000 7fcddf400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcde015a9a0 0x7fcde00cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:39.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.436+0000 7fcdd7e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fcde00add70 0x7fcde00d1080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:59250/0 (socket says 172.21.15.196:59250) 2024-04-28T13:31:39.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.436+0000 7fcddf400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcde015a9a0 0x7fcde00cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56620/0 (socket says 172.21.15.196:56620) 2024-04-28T13:31:39.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.436+0000 7fcdd7e00640 1 -- 172.21.15.196:0/3536673925 learned_addr learned my addr 172.21.15.196:0/3536673925 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:39.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.436+0000 7fcddfe00640 1 --2- 172.21.15.196:0/3536673925 >> v2:172.21.15.203:3300/0 conn(0x7fcde00d1790 0x7fcde015d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:39.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.436+0000 7fcddf400640 1 -- 172.21.15.196:0/3536673925 >> v2:172.21.15.203:3300/0 conn(0x7fcde00d1790 msgr2=0x7fcde015d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:39.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.436+0000 7fcddf400640 1 --2- 172.21.15.196:0/3536673925 >> v2:172.21.15.203:3300/0 conn(0x7fcde00d1790 0x7fcde015d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:39.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.436+0000 7fcddf400640 1 -- 172.21.15.196:0/3536673925 >> v1:172.21.15.196:6789/0 conn(0x7fcde00add70 legacy=0x7fcde00d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:39.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.436+0000 7fcddf400640 1 -- 172.21.15.196:0/3536673925 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde01613a0 con 0x7fcde015a9a0 2024-04-28T13:31:39.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.437+0000 7fcddf400640 1 --2- 172.21.15.196:0/3536673925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde015a9a0 0x7fcde00cfe90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcde00cd610 tx=0x7fcdd002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9af734ab4ba2476 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:39.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.438+0000 7fcddea00640 1 -- 172.21.15.196:0/3536673925 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0037070 con 0x7fcde015a9a0 2024-04-28T13:31:39.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.438+0000 7fcddea00640 1 -- 172.21.15.196:0/3536673925 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd0032440 con 0x7fcde015a9a0 2024-04-28T13:31:39.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.438+0000 7fcddea00640 1 -- 172.21.15.196:0/3536673925 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0032810 con 0x7fcde015a9a0 2024-04-28T13:31:39.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.438+0000 7fcde5200640 1 -- 172.21.15.196:0/3536673925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde015a9a0 msgr2=0x7fcde00cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:39.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.438+0000 7fcde5200640 1 --2- 172.21.15.196:0/3536673925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde015a9a0 0x7fcde00cfe90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcde00cd610 tx=0x7fcdd002a1a0 comp rx=0 tx=0).stop 2024-04-28T13:31:39.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.438+0000 7fcde5200640 1 -- 172.21.15.196:0/3536673925 shutdown_connections 2024-04-28T13:31:39.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.438+0000 7fcde5200640 1 --2- 172.21.15.196:0/3536673925 >> v2:172.21.15.203:3300/0 conn(0x7fcde00d1790 0x7fcde015d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:39.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.438+0000 7fcde5200640 1 --2- 172.21.15.196:0/3536673925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde015a9a0 0x7fcde00cfe90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:39.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.438+0000 7fcde5200640 1 -- 172.21.15.196:0/3536673925 >> 172.21.15.196:0/3536673925 conn(0x7fcde006c6c0 msgr2=0x7fcde006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:39.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.439+0000 7fcde5200640 1 -- 172.21.15.196:0/3536673925 shutdown_connections 2024-04-28T13:31:39.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.439+0000 7fcde5200640 1 -- 172.21.15.196:0/3536673925 wait complete. 2024-04-28T13:31:39.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.440+0000 7fcde5200640 1 Processor -- start 2024-04-28T13:31:39.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.440+0000 7fcde5200640 1 -- start start 2024-04-28T13:31:39.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.441+0000 7fcde5200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde00add70 0x7fcde0149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:39.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.441+0000 7fcde5200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde00d1790 0x7fcde00ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:39.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.441+0000 7fcde5200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde015a9a0 0x7fcde00ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:39.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.441+0000 7fcde5200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcde00d06e0 con 0x7fcde00add70 2024-04-28T13:31:39.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.441+0000 7fcde5200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01618c0 con 0x7fcde015a9a0 2024-04-28T13:31:39.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.441+0000 7fcde5200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcde015ad80 con 0x7fcde00d1790 2024-04-28T13:31:39.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.442+0000 7fcddf400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde00add70 0x7fcde0149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:39.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.442+0000 7fcddf400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde00add70 0x7fcde0149700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56632/0 (socket says 172.21.15.196:56632) 2024-04-28T13:31:39.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.442+0000 7fcddf400640 1 -- 172.21.15.196:0/1188750753 learned_addr learned my addr 172.21.15.196:0/1188750753 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:39.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.442+0000 7fcddf400640 1 -- 172.21.15.196:0/1188750753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde00d1790 msgr2=0x7fcde00ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:39.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.442+0000 7fcddfe00640 1 --2- 172.21.15.196:0/1188750753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde015a9a0 0x7fcde00ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:39.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.442+0000 7fcddf400640 1 --2- 172.21.15.196:0/1188750753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde00d1790 0x7fcde00ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:39.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.442+0000 7fcddf400640 1 -- 172.21.15.196:0/1188750753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde015a9a0 msgr2=0x7fcde00ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:39.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.442+0000 7fcddf400640 1 --2- 172.21.15.196:0/1188750753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde015a9a0 0x7fcde00ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:39.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.442+0000 7fcddf400640 1 -- 172.21.15.196:0/1188750753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde00cab80 con 0x7fcde00add70 2024-04-28T13:31:39.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.442+0000 7fcddf400640 1 --2- 172.21.15.196:0/1188750753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde00add70 0x7fcde0149700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0000c00 tx=0x7fcdd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:39.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.443+0000 7fcdd7400640 1 -- 172.21.15.196:0/1188750753 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0037040 con 0x7fcde00add70 2024-04-28T13:31:39.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.443+0000 7fcdd7400640 1 -- 172.21.15.196:0/1188750753 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd0042070 con 0x7fcde00add70 2024-04-28T13:31:39.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.443+0000 7fcdd7400640 1 -- 172.21.15.196:0/1188750753 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0030bd0 con 0x7fcde00add70 2024-04-28T13:31:39.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.443+0000 7fcde5200640 1 -- 172.21.15.196:0/1188750753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde00cae60 con 0x7fcde00add70 2024-04-28T13:31:39.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.443+0000 7fcde5200640 1 -- 172.21.15.196:0/1188750753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde00cb420 con 0x7fcde00add70 2024-04-28T13:31:39.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.444+0000 7fcde5200640 1 -- 172.21.15.196:0/1188750753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde009f0f0 con 0x7fcde00add70 2024-04-28T13:31:39.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.445+0000 7fcdd7400640 1 -- 172.21.15.196:0/1188750753 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdd0047460 con 0x7fcde00add70 2024-04-28T13:31:39.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.446+0000 7fcdd7400640 1 --2- 172.21.15.196:0/1188750753 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcdbc06c8f0 0x7fcdbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:39.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.447+0000 7fcdd7400640 1 -- 172.21.15.196:0/1188750753 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcdd00ac810 con 0x7fcde00add70 2024-04-28T13:31:39.448 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.447+0000 7fcdd7e00640 1 --2- 172.21.15.196:0/1188750753 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcdbc06c8f0 0x7fcdbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:39.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.449+0000 7fcdd7e00640 1 --2- 172.21.15.196:0/1188750753 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcdbc06c8f0 0x7fcdbc06edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8000f60 tx=0x7fcdc8007a80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:39.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.451+0000 7fcdd7400640 1 -- 172.21.15.196:0/1188750753 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcdd0032e20 con 0x7fcde00add70 2024-04-28T13:31:39.554 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:31:39.561 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:31:39.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.693+0000 7fcde5200640 1 -- 172.21.15.196:0/1188750753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcde006f790 con 0x7fcde00add70 2024-04-28T13:31:39.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.694+0000 7fcdd7400640 1 -- 172.21.15.196:0/1188750753 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fcdd007aba0 con 0x7fcde00add70 2024-04-28T13:31:39.695 INFO:teuthology.orchestra.run.smithi196.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-28T13:31:39.697 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.696+0000 7fcde5200640 1 -- 172.21.15.196:0/1188750753 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcdbc06c8f0 msgr2=0x7fcdbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:39.697 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.696+0000 7fcde5200640 1 --2- 172.21.15.196:0/1188750753 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcdbc06c8f0 0x7fcdbc06edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8000f60 tx=0x7fcdc8007a80 comp rx=0 tx=0).stop 2024-04-28T13:31:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.696+0000 7fcde5200640 1 -- 172.21.15.196:0/1188750753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde00add70 msgr2=0x7fcde0149700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.696+0000 7fcde5200640 1 --2- 172.21.15.196:0/1188750753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde00add70 0x7fcde0149700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0000c00 tx=0x7fcdd00046c0 comp rx=0 tx=0).stop 2024-04-28T13:31:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.696+0000 7fcde5200640 1 -- 172.21.15.196:0/1188750753 shutdown_connections 2024-04-28T13:31:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.696+0000 7fcde5200640 1 --2- 172.21.15.196:0/1188750753 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcdbc06c8f0 0x7fcdbc06edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.696+0000 7fcde5200640 1 --2- 172.21.15.196:0/1188750753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde015a9a0 0x7fcde00ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.696+0000 7fcde5200640 1 --2- 172.21.15.196:0/1188750753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde00d1790 0x7fcde00ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.696+0000 7fcde5200640 1 --2- 172.21.15.196:0/1188750753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde00add70 0x7fcde0149700 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.696+0000 7fcde5200640 1 -- 172.21.15.196:0/1188750753 >> 172.21.15.196:0/1188750753 conn(0x7fcde006c6c0 msgr2=0x7fcde0159540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.697+0000 7fcde5200640 1 -- 172.21.15.196:0/1188750753 shutdown_connections 2024-04-28T13:31:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39.697+0000 7fcde5200640 1 -- 172.21.15.196:0/1188750753 wait complete. 2024-04-28T13:31:39.716 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-28T13:31:39.716 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T13:31:39.716 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-04-28T13:31:39.716 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-28T13:31:39.716 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-28T13:31:39.716 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-28T13:31:39.716 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-28T13:31:39.716 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T13:31:39.716 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-28T13:31:39.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:39.717 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-04-28T13:31:39.767 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:31:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:31:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:31:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-28T13:31:39.767 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:39.767 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:31:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-04-28T13:31:39.768 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:31:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-04-28T13:31:39.768 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:31:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:39.768 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T13:31:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T13:31:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42651sec preferred_lft 42651sec 2024-04-28T13:31:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link 2024-04-28T13:31:39.768 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:39.769 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-28T13:31:39.769 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:39.769 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-28T13:31:39.770 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add name ceph-brx type bridge 2024-04-28T13:31:39.770 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip addr flush dev ceph-brx 2024-04-28T13:31:39.770 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set ceph-brx up 2024-04-28T13:31:39.770 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-28T13:31:39.770 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-28T13:31:39.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:40.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:40.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:40.250 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T13:31:40.317 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-04-28T13:31:40.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:40.321 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-04-28T13:31:40.374 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-04-28T13:31:40.374 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T13:31:40.374 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T13:31:40.374 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T13:31:40.374 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T13:31:40.374 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:40.374 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-28T13:31:40.375 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T13:31:40.375 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T13:31:40.375 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T13:31:40.375 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-28T13:31:40.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:41.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:41.111 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-28T13:31:41.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:41.163 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list-id 2024-04-28T13:31:41.217 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:41.217 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-28T13:31:41.217 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T13:31:41.217 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-28T13:31:41.218 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-28T13:31:41.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:41.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:41.408 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-28T13:31:41.408 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:41.408 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-28T13:31:41.408 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-28T13:31:41.408 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-28T13:31:41.408 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-28T13:31:41.408 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-28T13:31:41.408 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-28T13:31:41.408 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-28T13:31:41.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:41.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:41.686 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:41.686 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-28T13:31:41.687 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-04-28T13:31:41.687 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set dev brx.0 master ceph-brx 2024-04-28T13:31:41.687 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-28T13:31:41.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:41.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:41.885 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:31:41.886 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-28T13:31:41.931 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-28T13:31:41.932 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:31:41.932 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-28T13:31:42.026 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-28T13:31:42.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:42.026 DEBUG:teuthology.orchestra.run.smithi196:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:31:42.155 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:31:42.155 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-28T13:31:42.155 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:31:42.867 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:31:42.867 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-28T13:31:42.868 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:31:42.868 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:31:42.868 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:31:42.905 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:31:42.906 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-28T13:31:42.906 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:31:42.906 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:31:42.906 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:31:42.906 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-28T13:31:42.906 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-28T13:31:42.906 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=270aea32-2e16-4a97-85f7-aeae39bc8c76" will pass to kernel 2024-04-28T13:31:42.913 INFO:teuthology.orchestra.run.smithi196.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-28T13:31:42.913 INFO:teuthology.orchestra.run.smithi196.stdout: You just mounted a file system that supports labels which does not 2024-04-28T13:31:42.913 INFO:teuthology.orchestra.run.smithi196.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-28T13:31:42.913 INFO:teuthology.orchestra.run.smithi196.stdout: applications will generate AVC messages and not be allowed access to 2024-04-28T13:31:42.913 INFO:teuthology.orchestra.run.smithi196.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-28T13:31:42.918 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-28T13:31:42.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:42.919 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-28T13:31:43.003 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:31:43.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:31:43.139 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:43.149 INFO:teuthology.orchestra.run.smithi196.stdout:client4424 2024-04-28T13:31:43.149 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:31:43.149 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:31:43.203 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:31:43.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:31:43.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:43.338 INFO:teuthology.orchestra.run.smithi196.stdout:client4424 2024-04-28T13:31:43.339 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:31:43.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:31:43.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:43.477 INFO:teuthology.orchestra.run.smithi196.stdout:client4424 2024-04-28T13:31:43.477 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/270aea32-2e16-4a97-85f7-aeae39bc8c76.client4424/status') 2024-04-28T13:31:43.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:43.630 INFO:teuthology.orchestra.run.smithi196.stdout:instance: client.4424 (3)172.21.15.196:0/515569031 2024-04-28T13:31:43.631 INFO:teuthology.orchestra.run.smithi196.stdout:blocklisted: false 2024-04-28T13:31:43.631 INFO:teuthology.orchestra.run.smithi196.stderr:0+1 records in 2024-04-28T13:31:43.631 INFO:teuthology.orchestra.run.smithi196.stderr:0+1 records out 2024-04-28T13:31:43.631 INFO:teuthology.orchestra.run.smithi196.stderr:70 bytes copied, 7.5826e-05 s, 923 kB/s 2024-04-28T13:31:43.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:43.644 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-28T13:31:43.644 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-28T13:31:43.644 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-28T13:31:43.644 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T13:31:43.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.845+0000 7fd0f4e00640 1 Processor -- start 2024-04-28T13:31:43.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.845+0000 7fd0f4e00640 1 -- start start 2024-04-28T13:31:43.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.847+0000 7fd0f4e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd0f013ff70 0x7fd0f0146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:43.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.847+0000 7fd0f4e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f00cff20 0x7fd0f01465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:43.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.847+0000 7fd0f4e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0f00d05b0 con 0x7fd0f0140350 2024-04-28T13:31:43.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.847+0000 7fd0f4e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f00d0890 con 0x7fd0f013ff70 2024-04-28T13:31:43.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.847+0000 7fd0f4e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f006e450 con 0x7fd0f00cff20 2024-04-28T13:31:43.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.848+0000 7fd0ee800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f00cff20 0x7fd0f01465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:43.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.848+0000 7fd0ede00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd0f013ff70 0x7fd0f0146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:43.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.848+0000 7fd0ee800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f00cff20 0x7fd0f01465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:49094/0 (socket says 172.21.15.196:49094) 2024-04-28T13:31:43.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.848+0000 7fd0ede00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd0f013ff70 0x7fd0f0146080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:46032/0 (socket says 172.21.15.196:46032) 2024-04-28T13:31:43.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.848+0000 7fd0ee800640 1 -- 172.21.15.196:0/1751087434 learned_addr learned my addr 172.21.15.196:0/1751087434 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:43.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.848+0000 7fd0ed400640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd0f0140350 0x7fd0f00cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.196:42126/0 (socket says 172.21.15.196:42126) 2024-04-28T13:31:43.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.849+0000 7fd0ede00640 1 -- 172.21.15.196:0/1751087434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f00cff20 msgr2=0x7fd0f01465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:43.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.849+0000 7fd0ede00640 1 --2- 172.21.15.196:0/1751087434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f00cff20 0x7fd0f01465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:43.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.849+0000 7fd0ede00640 1 -- 172.21.15.196:0/1751087434 >> v1:172.21.15.203:6789/0 conn(0x7fd0f0140350 legacy=0x7fd0f00cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:43.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.849+0000 7fd0ede00640 1 -- 172.21.15.196:0/1751087434 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f006e730 con 0x7fd0f013ff70 2024-04-28T13:31:43.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.849+0000 7fd0ee800640 1 --2- 172.21.15.196:0/1751087434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f00cff20 0x7fd0f01465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:31:43.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.850+0000 7fd0ede00640 1 --2- 172.21.15.196:0/1751087434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0f013ff70 0x7fd0f0146080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd0e00097a0 tx=0x7fd0e002a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e489b4ac87463bec server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:43.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.850+0000 7fd0eca00640 1 -- 172.21.15.196:0/1751087434 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0037070 con 0x7fd0f013ff70 2024-04-28T13:31:43.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.850+0000 7fd0eca00640 1 -- 172.21.15.196:0/1751087434 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e002ae40 con 0x7fd0f013ff70 2024-04-28T13:31:43.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.850+0000 7fd0eca00640 1 -- 172.21.15.196:0/1751087434 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0032660 con 0x7fd0f013ff70 2024-04-28T13:31:43.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.851+0000 7fd0f4e00640 1 -- 172.21.15.196:0/1751087434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0f013ff70 msgr2=0x7fd0f0146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:43.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.851+0000 7fd0f4e00640 1 --2- 172.21.15.196:0/1751087434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0f013ff70 0x7fd0f0146080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd0e00097a0 tx=0x7fd0e002a0f0 comp rx=0 tx=0).stop 2024-04-28T13:31:43.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.851+0000 7fd0f4e00640 1 -- 172.21.15.196:0/1751087434 shutdown_connections 2024-04-28T13:31:43.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.851+0000 7fd0f4e00640 1 --2- 172.21.15.196:0/1751087434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f00cff20 0x7fd0f01465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:43.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.851+0000 7fd0f4e00640 1 --2- 172.21.15.196:0/1751087434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0f013ff70 0x7fd0f0146080 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:43.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.851+0000 7fd0f4e00640 1 -- 172.21.15.196:0/1751087434 >> 172.21.15.196:0/1751087434 conn(0x7fd0f00d0d30 msgr2=0x7fd0f00d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:43.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.851+0000 7fd0f4e00640 1 -- 172.21.15.196:0/1751087434 shutdown_connections 2024-04-28T13:31:43.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.851+0000 7fd0f4e00640 1 -- 172.21.15.196:0/1751087434 wait complete. 2024-04-28T13:31:43.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.853+0000 7fd0f4e00640 1 Processor -- start 2024-04-28T13:31:43.854 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.853+0000 7fd0f4e00640 1 -- start start 2024-04-28T13:31:43.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.854+0000 7fd0f4e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f00cff20 0x7fd0f013b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:43.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.854+0000 7fd0f4e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0f013ff70 0x7fd0f0144760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:43.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.854+0000 7fd0f4e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f0140350 0x7fd0f0144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:43.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.854+0000 7fd0f4e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f006efe0 con 0x7fd0f00cff20 2024-04-28T13:31:43.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.854+0000 7fd0f4e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f006ee60 con 0x7fd0f013ff70 2024-04-28T13:31:43.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.854+0000 7fd0f4e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f006f160 con 0x7fd0f0140350 2024-04-28T13:31:43.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.854+0000 7fd0ede00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f00cff20 0x7fd0f013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:43.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.854+0000 7fd0ede00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f00cff20 0x7fd0f013b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43600/0 (socket says 172.21.15.196:43600) 2024-04-28T13:31:43.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.854+0000 7fd0ede00640 1 -- 172.21.15.196:0/532647548 learned_addr learned my addr 172.21.15.196:0/532647548 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:43.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.855+0000 7fd0ed400640 1 --2- 172.21.15.196:0/532647548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0f013ff70 0x7fd0f0144760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:43.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.855+0000 7fd0ee800640 1 --2- 172.21.15.196:0/532647548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f0140350 0x7fd0f0144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:43.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.855+0000 7fd0ede00640 1 -- 172.21.15.196:0/532647548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f0140350 msgr2=0x7fd0f0144ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:43.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.855+0000 7fd0ede00640 1 --2- 172.21.15.196:0/532647548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f0140350 0x7fd0f0144ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:43.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.855+0000 7fd0ede00640 1 -- 172.21.15.196:0/532647548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0f013ff70 msgr2=0x7fd0f0144760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:43.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.855+0000 7fd0ede00640 1 --2- 172.21.15.196:0/532647548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0f013ff70 0x7fd0f0144760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:43.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.855+0000 7fd0ede00640 1 -- 172.21.15.196:0/532647548 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f013e700 con 0x7fd0f00cff20 2024-04-28T13:31:43.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.855+0000 7fd0ede00640 1 --2- 172.21.15.196:0/532647548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f00cff20 0x7fd0f013b630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd0e00098c0 tx=0x7fd0e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:43.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.855+0000 7fd0e6000640 1 -- 172.21.15.196:0/532647548 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0041030 con 0x7fd0f00cff20 2024-04-28T13:31:43.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.855+0000 7fd0e6000640 1 -- 172.21.15.196:0/532647548 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e002ac40 con 0x7fd0f00cff20 2024-04-28T13:31:43.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.855+0000 7fd0f4e00640 1 -- 172.21.15.196:0/532647548 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f013e980 con 0x7fd0f00cff20 2024-04-28T13:31:43.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.856+0000 7fd0e6000640 1 -- 172.21.15.196:0/532647548 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0037040 con 0x7fd0f00cff20 2024-04-28T13:31:43.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.856+0000 7fd0f4e00640 1 -- 172.21.15.196:0/532647548 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f013ef40 con 0x7fd0f00cff20 2024-04-28T13:31:43.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.857+0000 7fd0f4e00640 1 -- 172.21.15.196:0/532647548 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f006a290 con 0x7fd0f00cff20 2024-04-28T13:31:43.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.858+0000 7fd0e6000640 1 -- 172.21.15.196:0/532647548 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0e0044070 con 0x7fd0f00cff20 2024-04-28T13:31:43.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.859+0000 7fd0e6000640 1 --2- 172.21.15.196:0/532647548 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd0d006c8f0 0x7fd0d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:43.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.859+0000 7fd0e6000640 1 -- 172.21.15.196:0/532647548 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd0e00ac120 con 0x7fd0f00cff20 2024-04-28T13:31:43.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.859+0000 7fd0ed400640 1 --2- 172.21.15.196:0/532647548 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd0d006c8f0 0x7fd0d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:43.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.860+0000 7fd0ed400640 1 --2- 172.21.15.196:0/532647548 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd0d006c8f0 0x7fd0d006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd0f013c780 tx=0x7fd0dc007ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:43.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:43.863+0000 7fd0e6000640 1 -- 172.21.15.196:0/532647548 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd0e004b070 con 0x7fd0f00cff20 2024-04-28T13:31:44.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44.105+0000 7fd0f4e00640 1 -- 172.21.15.196:0/532647548 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd0f0072ba0 con 0x7fd0f00cff20 2024-04-28T13:31:44.107 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44.106+0000 7fd0e6000640 1 -- 172.21.15.196:0/532647548 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fd0f013e700 con 0x7fd0f00cff20 2024-04-28T13:31:44.107 INFO:teuthology.orchestra.run.smithi196.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-28T13:31:44.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44.108+0000 7fd0f4e00640 1 -- 172.21.15.196:0/532647548 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd0d006c8f0 msgr2=0x7fd0d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:44.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44.108+0000 7fd0f4e00640 1 --2- 172.21.15.196:0/532647548 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd0d006c8f0 0x7fd0d006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd0f013c780 tx=0x7fd0dc007ce0 comp rx=0 tx=0).stop 2024-04-28T13:31:44.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44.108+0000 7fd0f4e00640 1 -- 172.21.15.196:0/532647548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f00cff20 msgr2=0x7fd0f013b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:44.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44.108+0000 7fd0f4e00640 1 --2- 172.21.15.196:0/532647548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f00cff20 0x7fd0f013b630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd0e00098c0 tx=0x7fd0e00046c0 comp rx=0 tx=0).stop 2024-04-28T13:31:44.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44.108+0000 7fd0f4e00640 1 -- 172.21.15.196:0/532647548 shutdown_connections 2024-04-28T13:31:44.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44.108+0000 7fd0f4e00640 1 --2- 172.21.15.196:0/532647548 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd0d006c8f0 0x7fd0d006edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:44.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44.108+0000 7fd0f4e00640 1 --2- 172.21.15.196:0/532647548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f0140350 0x7fd0f0144ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:44.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44.108+0000 7fd0f4e00640 1 --2- 172.21.15.196:0/532647548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0f013ff70 0x7fd0f0144760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:44.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44.108+0000 7fd0f4e00640 1 --2- 172.21.15.196:0/532647548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f00cff20 0x7fd0f013b630 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:44.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44.108+0000 7fd0f4e00640 1 -- 172.21.15.196:0/532647548 >> 172.21.15.196:0/532647548 conn(0x7fd0f00d0d30 msgr2=0x7fd0f00af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:44.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44.109+0000 7fd0f4e00640 1 -- 172.21.15.196:0/532647548 shutdown_connections 2024-04-28T13:31:44.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44.109+0000 7fd0f4e00640 1 -- 172.21.15.196:0/532647548 wait complete. 2024-04-28T13:31:44.129 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-28T13:31:44.129 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T13:31:44.129 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-04-28T13:31:44.129 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-28T13:31:44.129 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-28T13:31:44.129 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-28T13:31:44.129 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-28T13:31:44.129 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T13:31:44.130 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-28T13:31:44.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:44.130 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-04-28T13:31:44.182 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:31:44.182 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:31:44.182 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:31:44.182 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:44.182 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-28T13:31:44.182 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:44.182 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:31:44.183 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:44.183 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-04-28T13:31:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:31:44.183 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:44.183 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-04-28T13:31:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:31:44.183 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T13:31:44.183 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:44.183 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T13:31:44.183 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42646sec preferred_lft 42646sec 2024-04-28T13:31:44.183 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link 2024-04-28T13:31:44.183 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T13:31:44.184 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether c6:b9:8c:18:7a:ec brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:44.184 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-28T13:31:44.184 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:44.184 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::3cb7:a8ff:fe55:18c0/64 scope link 2024-04-28T13:31:44.184 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:44.184 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:31:44.184 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether c6:b9:8c:18:7a:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T13:31:44.184 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::c4b9:8cff:fe18:7aec/64 scope link 2024-04-28T13:31:44.184 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:44.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:44.185 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T13:31:44.263 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-04-28T13:31:44.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:44.267 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-04-28T13:31:44.319 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-04-28T13:31:44.319 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T13:31:44.320 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T13:31:44.320 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T13:31:44.320 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T13:31:44.321 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:44.321 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-28T13:31:44.321 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T13:31:44.321 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T13:31:44.321 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T13:31:44.321 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-28T13:31:44.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:44.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:44.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:44.532 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-28T13:31:44.576 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-28T13:31:44.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:44.577 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list-id 2024-04-28T13:31:44.674 INFO:teuthology.orchestra.run.smithi196.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-28T13:31:44.675 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:44.675 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-28T13:31:44.675 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T13:31:44.675 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-28T13:31:44.676 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-28T13:31:44.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:44.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:44.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:44.856 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-28T13:31:44.931 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:31:44.931 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:31:44.931 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:31:44.931 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:44.931 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-28T13:31:44.931 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:44.931 INFO:teuthology.orchestra.run.smithi196.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T13:31:44.931 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether da:e1:fd:d9:dc:8a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-28T13:31:44.931 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-28T13:31:44.931 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:44.931 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::d8e1:fdff:fed9:dc8a/64 scope link 2024-04-28T13:31:44.931 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:44.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:44.935 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-28T13:31:45.019 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:31:45.020 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:31:45.020 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:31:45.020 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:45.020 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-28T13:31:45.020 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:45.020 INFO:teuthology.orchestra.run.smithi196.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T13:31:45.020 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether da:e1:fd:d9:dc:8a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-28T13:31:45.020 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-28T13:31:45.020 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:45.020 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::d8e1:fdff:fed9:dc8a/64 scope link 2024-04-28T13:31:45.020 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:45.024 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-28T13:31:45.024 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:45.024 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-28T13:31:45.024 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-28T13:31:45.024 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-28T13:31:45.024 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-28T13:31:45.024 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-28T13:31:45.024 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-28T13:31:45.024 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-28T13:31:45.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:45.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:45.311 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:45.311 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-28T13:31:45.311 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.1 up 2024-04-28T13:31:45.311 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set dev brx.1 master ceph-brx 2024-04-28T13:31:45.311 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-28T13:31:45.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:45.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:45.523 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:31:45.523 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-28T13:31:45.565 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-28T13:31:45.566 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:31:45.566 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-28T13:31:45.657 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-28T13:31:45.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:45.657 DEBUG:teuthology.orchestra.run.smithi196:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-28T13:31:45.775 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-28T13:31:45.775 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-28T13:31:45.775 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:31:45.796 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-28T13:31:45.796 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-28T13:31:45.796 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:31:45.797 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:31:45.797 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:31:45.824 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-28T13:31:45.824 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-28T13:31:45.824 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:31:45.824 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:31:45.824 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:31:45.824 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-28T13:31:45.824 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-28T13:31:45.824 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=270aea32-2e16-4a97-85f7-aeae39bc8c76" will pass to kernel 2024-04-28T13:31:45.826 INFO:teuthology.orchestra.run.smithi196.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-28T13:31:45.826 INFO:teuthology.orchestra.run.smithi196.stdout: You just mounted a file system that supports labels which does not 2024-04-28T13:31:45.826 INFO:teuthology.orchestra.run.smithi196.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-28T13:31:45.826 INFO:teuthology.orchestra.run.smithi196.stdout: applications will generate AVC messages and not be allowed access to 2024-04-28T13:31:45.826 INFO:teuthology.orchestra.run.smithi196.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-28T13:31:45.830 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-28T13:31:45.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:45.830 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-28T13:31:45.901 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:31:46.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:31:46.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:46.017 INFO:teuthology.orchestra.run.smithi196.stdout:client4433 2024-04-28T13:31:46.017 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:31:46.018 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:31:46.067 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:31:46.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:31:46.219 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:46.228 INFO:teuthology.orchestra.run.smithi196.stdout:client4433 2024-04-28T13:31:46.228 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:31:46.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:31:46.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:46.345 INFO:teuthology.orchestra.run.smithi196.stdout:client4433 2024-04-28T13:31:46.346 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/270aea32-2e16-4a97-85f7-aeae39bc8c76.client4433/status') 2024-04-28T13:31:46.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:46.478 INFO:teuthology.orchestra.run.smithi196.stdout:instance: client.4433 (3)172.21.15.196:0/2450293512 2024-04-28T13:31:46.479 INFO:teuthology.orchestra.run.smithi196.stdout:blocklisted: false 2024-04-28T13:31:46.479 INFO:teuthology.orchestra.run.smithi196.stderr:0+1 records in 2024-04-28T13:31:46.479 INFO:teuthology.orchestra.run.smithi196.stderr:0+1 records out 2024-04-28T13:31:46.479 INFO:teuthology.orchestra.run.smithi196.stderr:71 bytes copied, 8.3341e-05 s, 852 kB/s 2024-04-28T13:31:46.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:46.492 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-28T13:31:46.492 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-28T13:31:46.492 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-28T13:31:46.493 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T13:31:46.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.689+0000 7ff5eb200640 1 Processor -- start 2024-04-28T13:31:46.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.689+0000 7ff5eb200640 1 -- start start 2024-04-28T13:31:46.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.691+0000 7ff5eb200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e40d1a90 0x7ff5e415f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:46.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.691+0000 7ff5eb200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff5e40d1e70 0x7ff5e415f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:46.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.691+0000 7ff5eb200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e40d5e60 con 0x7ff5e40b61b0 2024-04-28T13:31:46.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.691+0000 7ff5eb200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e40d5c10 con 0x7ff5e40d1e70 2024-04-28T13:31:46.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.691+0000 7ff5eb200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e40d38e0 con 0x7ff5e40d1a90 2024-04-28T13:31:46.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.692+0000 7ff5e9400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff5e40d1e70 0x7ff5e415f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:46.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.692+0000 7ff5e9e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e40d1a90 0x7ff5e415f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:46.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.692+0000 7ff5e9400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff5e40d1e70 0x7ff5e415f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:46060/0 (socket says 172.21.15.196:46060) 2024-04-28T13:31:46.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.692+0000 7ff5e9400640 1 -- 172.21.15.196:0/739609473 learned_addr learned my addr 172.21.15.196:0/739609473 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:46.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.692+0000 7ff5e9e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e40d1a90 0x7ff5e415f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:49104/0 (socket says 172.21.15.196:49104) 2024-04-28T13:31:46.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.692+0000 7ff5e9e00640 1 -- 172.21.15.196:0/739609473 >> v2:172.21.15.203:3300/0 conn(0x7ff5e40d1e70 msgr2=0x7ff5e415f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:46.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.692+0000 7ff5e9e00640 1 --2- 172.21.15.196:0/739609473 >> v2:172.21.15.203:3300/0 conn(0x7ff5e40d1e70 0x7ff5e415f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:46.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.692+0000 7ff5e9e00640 1 -- 172.21.15.196:0/739609473 >> v1:172.21.15.203:6789/0 conn(0x7ff5e40b61b0 legacy=0x7ff5e41632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:46.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.692+0000 7ff5e9e00640 1 -- 172.21.15.196:0/739609473 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e40d3b30 con 0x7ff5e40d1a90 2024-04-28T13:31:46.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.692+0000 7ff5e8a00640 1 -- 172.21.15.196:0/739609473 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1594086282 0 0) 0x7ff5e40d5e60 con 0x7ff5e40b61b0 2024-04-28T13:31:46.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.694+0000 7ff5e9e00640 1 --2- 172.21.15.196:0/739609473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e40d1a90 0x7ff5e415f200 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80097a0 tx=0x7ff5d802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dad1d31c9fcbf33 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:46.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.694+0000 7ff5e8a00640 1 -- 172.21.15.196:0/739609473 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8037070 con 0x7ff5e40d1a90 2024-04-28T13:31:46.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.694+0000 7ff5e8a00640 1 -- 172.21.15.196:0/739609473 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d8032440 con 0x7ff5e40d1a90 2024-04-28T13:31:46.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.694+0000 7ff5e8a00640 1 -- 172.21.15.196:0/739609473 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8032810 con 0x7ff5e40d1a90 2024-04-28T13:31:46.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.695+0000 7ff5eb200640 1 -- 172.21.15.196:0/739609473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e40d1a90 msgr2=0x7ff5e415f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:46.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.695+0000 7ff5eb200640 1 --2- 172.21.15.196:0/739609473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e40d1a90 0x7ff5e415f200 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80097a0 tx=0x7ff5d802a180 comp rx=0 tx=0).stop 2024-04-28T13:31:46.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.695+0000 7ff5eb200640 1 -- 172.21.15.196:0/739609473 shutdown_connections 2024-04-28T13:31:46.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.695+0000 7ff5eb200640 1 --2- 172.21.15.196:0/739609473 >> v2:172.21.15.203:3300/0 conn(0x7ff5e40d1e70 0x7ff5e415f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:46.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.695+0000 7ff5eb200640 1 --2- 172.21.15.196:0/739609473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e40d1a90 0x7ff5e415f200 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:46.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.695+0000 7ff5eb200640 1 -- 172.21.15.196:0/739609473 >> 172.21.15.196:0/739609473 conn(0x7ff5e40d50f0 msgr2=0x7ff5e40d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:46.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.695+0000 7ff5eb200640 1 -- 172.21.15.196:0/739609473 shutdown_connections 2024-04-28T13:31:46.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.695+0000 7ff5eb200640 1 -- 172.21.15.196:0/739609473 wait complete. 2024-04-28T13:31:46.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.697+0000 7ff5eb200640 1 Processor -- start 2024-04-28T13:31:46.697 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.697+0000 7ff5eb200640 1 -- start start 2024-04-28T13:31:46.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.698+0000 7ff5eb200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e40b61b0 0x7ff5e41535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:46.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.698+0000 7ff5eb200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e40d1a90 0x7ff5e41f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:46.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.698+0000 7ff5eb200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e40d1e70 0x7ff5e40b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:46.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.698+0000 7ff5eb200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e40d3ea0 con 0x7ff5e40d1e70 2024-04-28T13:31:46.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.698+0000 7ff5eb200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e40b6910 con 0x7ff5e40d1a90 2024-04-28T13:31:46.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.698+0000 7ff5eb200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e40b5c60 con 0x7ff5e40b61b0 2024-04-28T13:31:46.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.698+0000 7ff5ea800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e40d1e70 0x7ff5e40b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:46.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.698+0000 7ff5ea800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e40d1e70 0x7ff5e40b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43616/0 (socket says 172.21.15.196:43616) 2024-04-28T13:31:46.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.698+0000 7ff5ea800640 1 -- 172.21.15.196:0/4032647372 learned_addr learned my addr 172.21.15.196:0/4032647372 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:46.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.699+0000 7ff5e9400640 1 --2- 172.21.15.196:0/4032647372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e40d1a90 0x7ff5e41f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:46.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.699+0000 7ff5e9e00640 1 --2- 172.21.15.196:0/4032647372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e40b61b0 0x7ff5e41535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:46.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.699+0000 7ff5ea800640 1 -- 172.21.15.196:0/4032647372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e40b61b0 msgr2=0x7ff5e41535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:46.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.699+0000 7ff5ea800640 1 --2- 172.21.15.196:0/4032647372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e40b61b0 0x7ff5e41535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:46.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.699+0000 7ff5ea800640 1 -- 172.21.15.196:0/4032647372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e40d1a90 msgr2=0x7ff5e41f93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:46.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.699+0000 7ff5ea800640 1 --2- 172.21.15.196:0/4032647372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e40d1a90 0x7ff5e41f93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:46.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.699+0000 7ff5ea800640 1 -- 172.21.15.196:0/4032647372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e4153b20 con 0x7ff5e40d1e70 2024-04-28T13:31:46.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.699+0000 7ff5e9e00640 1 --2- 172.21.15.196:0/4032647372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e40b61b0 0x7ff5e41535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:31:46.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.699+0000 7ff5ea800640 1 --2- 172.21.15.196:0/4032647372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e40d1e70 0x7ff5e40b3380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000c780 tx=0x7ff5e000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:46.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.699+0000 7ff5de000640 1 -- 172.21.15.196:0/4032647372 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0003b70 con 0x7ff5e40d1e70 2024-04-28T13:31:46.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.700+0000 7ff5de000640 1 -- 172.21.15.196:0/4032647372 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e0003d10 con 0x7ff5e40d1e70 2024-04-28T13:31:46.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.700+0000 7ff5eb200640 1 -- 172.21.15.196:0/4032647372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e41567c0 con 0x7ff5e40d1e70 2024-04-28T13:31:46.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.700+0000 7ff5de000640 1 -- 172.21.15.196:0/4032647372 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0005270 con 0x7ff5e40d1e70 2024-04-28T13:31:46.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.700+0000 7ff5eb200640 1 -- 172.21.15.196:0/4032647372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e41578f0 con 0x7ff5e40d1e70 2024-04-28T13:31:46.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.701+0000 7ff5eb200640 1 -- 172.21.15.196:0/4032647372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e41588f0 con 0x7ff5e40d1e70 2024-04-28T13:31:46.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.702+0000 7ff5de000640 1 -- 172.21.15.196:0/4032647372 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5e0005410 con 0x7ff5e40d1e70 2024-04-28T13:31:46.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.703+0000 7ff5de000640 1 --2- 172.21.15.196:0/4032647372 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff5c806c8f0 0x7ff5c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:46.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.703+0000 7ff5de000640 1 -- 172.21.15.196:0/4032647372 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff5e008b930 con 0x7ff5e40d1e70 2024-04-28T13:31:46.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.703+0000 7ff5e9e00640 1 --2- 172.21.15.196:0/4032647372 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff5c806c8f0 0x7ff5c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:46.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.705+0000 7ff5e9e00640 1 --2- 172.21.15.196:0/4032647372 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff5c806c8f0 0x7ff5c806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8034000 tx=0x7ff5d802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:46.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.708+0000 7ff5de000640 1 -- 172.21.15.196:0/4032647372 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff5e41588f0 con 0x7ff5e40d1e70 2024-04-28T13:31:46.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.949+0000 7ff5eb200640 1 -- 172.21.15.196:0/4032647372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff5e40507a0 con 0x7ff5e40d1e70 2024-04-28T13:31:46.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.950+0000 7ff5de000640 1 -- 172.21.15.196:0/4032647372 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7ff5e4153b20 con 0x7ff5e40d1e70 2024-04-28T13:31:46.950 INFO:teuthology.orchestra.run.smithi196.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-28T13:31:46.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.953+0000 7ff5eb200640 1 -- 172.21.15.196:0/4032647372 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff5c806c8f0 msgr2=0x7ff5c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:46.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.953+0000 7ff5eb200640 1 --2- 172.21.15.196:0/4032647372 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff5c806c8f0 0x7ff5c806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8034000 tx=0x7ff5d802e000 comp rx=0 tx=0).stop 2024-04-28T13:31:46.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.953+0000 7ff5eb200640 1 -- 172.21.15.196:0/4032647372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e40d1e70 msgr2=0x7ff5e40b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:46.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.953+0000 7ff5eb200640 1 --2- 172.21.15.196:0/4032647372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e40d1e70 0x7ff5e40b3380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000c780 tx=0x7ff5e000cb80 comp rx=0 tx=0).stop 2024-04-28T13:31:46.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.953+0000 7ff5eb200640 1 -- 172.21.15.196:0/4032647372 shutdown_connections 2024-04-28T13:31:46.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.953+0000 7ff5eb200640 1 --2- 172.21.15.196:0/4032647372 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff5c806c8f0 0x7ff5c806edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:46.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.953+0000 7ff5eb200640 1 --2- 172.21.15.196:0/4032647372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e40d1e70 0x7ff5e40b3380 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:46.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.953+0000 7ff5eb200640 1 --2- 172.21.15.196:0/4032647372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e40d1a90 0x7ff5e41f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:46.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.953+0000 7ff5eb200640 1 --2- 172.21.15.196:0/4032647372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e40b61b0 0x7ff5e41535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:46.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.953+0000 7ff5eb200640 1 -- 172.21.15.196:0/4032647372 >> 172.21.15.196:0/4032647372 conn(0x7ff5e40d50f0 msgr2=0x7ff5e40d4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:46.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.953+0000 7ff5eb200640 1 -- 172.21.15.196:0/4032647372 shutdown_connections 2024-04-28T13:31:46.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:46.953+0000 7ff5eb200640 1 -- 172.21.15.196:0/4032647372 wait complete. 2024-04-28T13:31:46.973 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-28T13:31:46.973 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T13:31:46.973 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2024-04-28T13:31:46.973 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-28T13:31:46.973 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-28T13:31:46.973 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-28T13:31:46.973 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-28T13:31:46.973 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T13:31:46.973 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-28T13:31:46.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:46.974 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2024-04-28T13:31:47.033 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:31:47.033 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:31:47.034 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:31:47.034 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:47.034 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-28T13:31:47.034 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:47.034 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:31:47.034 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:47.034 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f0 2024-04-28T13:31:47.034 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:31:47.034 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:47.034 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f1 2024-04-28T13:31:47.034 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:31:47.034 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:47.034 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T13:31:47.035 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:47.035 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T13:31:47.035 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42612sec preferred_lft 42612sec 2024-04-28T13:31:47.035 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ec4:7aff:fe8f:4a23/64 scope link 2024-04-28T13:31:47.035 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:47.036 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-28T13:31:47.036 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:47.036 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-28T13:31:47.036 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add name ceph-brx type bridge 2024-04-28T13:31:47.036 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr flush dev ceph-brx 2024-04-28T13:31:47.036 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set ceph-brx up 2024-04-28T13:31:47.036 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-28T13:31:47.036 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-28T13:31:47.143 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:47.515 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:47.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:47.525 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T13:31:47.594 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-28T13:31:47.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:47.598 DEBUG:teuthology.orchestra.run.smithi203:> route 2024-04-28T13:31:47.670 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2024-04-28T13:31:47.670 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T13:31:47.670 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T13:31:47.670 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T13:31:47.671 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T13:31:47.671 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:47.671 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-28T13:31:47.671 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T13:31:47.671 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T13:31:47.671 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T13:31:47.671 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-28T13:31:47.775 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:48.436 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:48.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:48.447 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-28T13:31:48.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:48.502 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list-id 2024-04-28T13:31:48.554 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:48.554 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-28T13:31:48.554 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T13:31:48.554 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-28T13:31:48.554 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-28T13:31:48.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:48.730 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:48.739 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-28T13:31:48.739 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:48.740 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-28T13:31:48.740 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-28T13:31:48.740 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-28T13:31:48.740 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-28T13:31:48.740 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-28T13:31:48.740 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-28T13:31:48.740 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-28T13:31:48.842 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:49.080 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:49.089 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:49.090 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-28T13:31:49.090 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2024-04-28T13:31:49.090 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set dev brx.0 master ceph-brx 2024-04-28T13:31:49.090 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-28T13:31:49.181 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:49.279 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:49.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:31:49.288 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-28T13:31:49.336 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-28T13:31:49.336 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:31:49.336 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-28T13:31:49.426 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-28T13:31:49.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:49.426 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:31:49.544 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:31:49.545 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-28T13:31:49.545 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:31:50.244 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:31:50.244 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-28T13:31:50.245 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:31:50.245 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:31:50.245 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:31:50.279 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:31:50.279 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-28T13:31:50.279 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:31:50.279 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:31:50.280 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:31:50.280 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-28T13:31:50.280 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-28T13:31:50.280 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=270aea32-2e16-4a97-85f7-aeae39bc8c76" will pass to kernel 2024-04-28T13:31:50.284 INFO:teuthology.orchestra.run.smithi203.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-28T13:31:50.284 INFO:teuthology.orchestra.run.smithi203.stdout: You just mounted a file system that supports labels which does not 2024-04-28T13:31:50.284 INFO:teuthology.orchestra.run.smithi203.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-28T13:31:50.284 INFO:teuthology.orchestra.run.smithi203.stdout: applications will generate AVC messages and not be allowed access to 2024-04-28T13:31:50.284 INFO:teuthology.orchestra.run.smithi203.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-28T13:31:50.288 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-28T13:31:50.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:50.289 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-28T13:31:50.364 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:31:50.466 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:31:50.471 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:50.480 INFO:teuthology.orchestra.run.smithi203.stdout:client4531 2024-04-28T13:31:50.480 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:31:50.480 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:31:50.530 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:31:50.682 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:31:50.686 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:50.696 INFO:teuthology.orchestra.run.smithi203.stdout:client4531 2024-04-28T13:31:50.696 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:31:50.801 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:31:50.806 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:50.815 INFO:teuthology.orchestra.run.smithi203.stdout:client4531 2024-04-28T13:31:50.815 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/270aea32-2e16-4a97-85f7-aeae39bc8c76.client4531/status') 2024-04-28T13:31:50.914 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:50.944 INFO:teuthology.orchestra.run.smithi203.stdout:instance: client.4531 (3)192.168.0.1:0/119637442 2024-04-28T13:31:50.944 INFO:teuthology.orchestra.run.smithi203.stdout:blocklisted: false 2024-04-28T13:31:50.944 INFO:teuthology.orchestra.run.smithi203.stderr:0+1 records in 2024-04-28T13:31:50.944 INFO:teuthology.orchestra.run.smithi203.stderr:0+1 records out 2024-04-28T13:31:50.944 INFO:teuthology.orchestra.run.smithi203.stderr:68 bytes copied, 7.6925e-05 s, 884 kB/s 2024-04-28T13:31:50.947 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:50.956 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-28T13:31:50.956 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-28T13:31:50.956 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-28T13:31:50.957 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T13:31:51.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.167+0000 7f9156400640 1 Processor -- start 2024-04-28T13:31:51.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.167+0000 7f9156400640 1 -- start start 2024-04-28T13:31:51.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.169+0000 7f9156400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91500d1a90 0x7f915015f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:51.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.169+0000 7f9156400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91501640e0 con 0x7f91500d0df0 2024-04-28T13:31:51.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.169+0000 7f9156400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91500d5e60 con 0x7f91500d1e70 2024-04-28T13:31:51.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.169+0000 7f9156400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f91500d5c10 con 0x7f91500d1a90 2024-04-28T13:31:51.174 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.169+0000 7f9155a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f91500d0df0 0x7f91501639d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:59674/0 (socket says 172.21.15.196:59674) 2024-04-28T13:31:51.174 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.169+0000 7f9155a00640 1 -- 172.21.15.196:0/401583572 learned_addr learned my addr 172.21.15.196:0/401583572 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:51.174 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f9155000640 1 --2- 172.21.15.196:0/401583572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91500d1a90 0x7f915015f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:51.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f914f400640 1 -- 172.21.15.196:0/401583572 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4215567422 0 0) 0x7f91501640e0 con 0x7f91500d0df0 2024-04-28T13:31:51.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f914f400640 1 -- 172.21.15.196:0/401583572 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9138003520 con 0x7f91500d0df0 2024-04-28T13:31:51.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f914f400640 1 -- 172.21.15.196:0/401583572 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2715157182 0 0) 0x7f9138003520 con 0x7f91500d0df0 2024-04-28T13:31:51.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f914f400640 1 -- 172.21.15.196:0/401583572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91500d1a90 msgr2=0x7f915015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:51.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f914f400640 1 --2- 172.21.15.196:0/401583572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91500d1a90 0x7f915015f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:51.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f914f400640 1 -- 172.21.15.196:0/401583572 >> v1:172.21.15.203:6789/0 conn(0x7f91500d1e70 legacy=0x7f91501632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:51.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f914f400640 1 -- 172.21.15.196:0/401583572 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91500d3b30 con 0x7f91500d0df0 2024-04-28T13:31:51.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f914f400640 1 -- 172.21.15.196:0/401583572 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f91480019f0 con 0x7f91500d0df0 2024-04-28T13:31:51.176 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f914f400640 1 -- 172.21.15.196:0/401583572 >> v1:172.21.15.196:6789/0 conn(0x7f91500d0df0 legacy=0x7f91501639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:51.176 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f914f400640 1 --2- 172.21.15.196:0/401583572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91380039e0 0x7f9138003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:51.176 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f914f400640 1 --2- 172.21.15.196:0/401583572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9138004800 0x7f9138004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:51.176 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f914f400640 1 --2- 172.21.15.196:0/401583572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91500d1a90 0x7f91380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:51.176 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f914f400640 1 -- 172.21.15.196:0/401583572 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91500d5c10 con 0x7f9138004800 2024-04-28T13:31:51.176 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f914f400640 1 -- 172.21.15.196:0/401583572 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f913800ba70 con 0x7f91380039e0 2024-04-28T13:31:51.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.170+0000 7f914f400640 1 -- 172.21.15.196:0/401583572 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f913800bc70 con 0x7f91500d1a90 2024-04-28T13:31:51.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.171+0000 7f914fe00640 1 --2- 172.21.15.196:0/401583572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9138004800 0x7f9138004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:51.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.171+0000 7f9155000640 1 --2- 172.21.15.196:0/401583572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91380039e0 0x7f9138003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:51.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.171+0000 7f9155a00640 1 --2- 172.21.15.196:0/401583572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91500d1a90 0x7f91380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:51.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.171+0000 7f914fe00640 1 -- 172.21.15.196:0/401583572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91500d1a90 msgr2=0x7f91380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:51.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.171+0000 7f914fe00640 1 --2- 172.21.15.196:0/401583572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91500d1a90 0x7f91380051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:51.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.171+0000 7f914fe00640 1 -- 172.21.15.196:0/401583572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91380039e0 msgr2=0x7f9138003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:51.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.171+0000 7f914fe00640 1 --2- 172.21.15.196:0/401583572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91380039e0 0x7f9138003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:51.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.171+0000 7f914fe00640 1 -- 172.21.15.196:0/401583572 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9138003520 con 0x7f9138004800 2024-04-28T13:31:51.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.172+0000 7f914fe00640 1 --2- 172.21.15.196:0/401583572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9138004800 0x7f9138004c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9140001720 tx=0x7f9138005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4d30bfa2436cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:51.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.173+0000 7f914f400640 1 -- 172.21.15.196:0/401583572 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140001370 con 0x7f9138004800 2024-04-28T13:31:51.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.173+0000 7f914f400640 1 -- 172.21.15.196:0/401583572 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91400052d0 con 0x7f9138004800 2024-04-28T13:31:51.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.173+0000 7f914f400640 1 -- 172.21.15.196:0/401583572 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914002ab90 con 0x7f9138004800 2024-04-28T13:31:51.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.173+0000 7f9156400640 1 -- 172.21.15.196:0/401583572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9138004800 msgr2=0x7f9138004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:51.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.173+0000 7f9156400640 1 --2- 172.21.15.196:0/401583572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9138004800 0x7f9138004c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9140001720 tx=0x7f9138005830 comp rx=0 tx=0).stop 2024-04-28T13:31:51.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.173+0000 7f9155a00640 1 -- 172.21.15.196:0/401583572 reap_dead start 2024-04-28T13:31:51.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.173+0000 7f9156400640 1 -- 172.21.15.196:0/401583572 shutdown_connections 2024-04-28T13:31:51.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.173+0000 7f9156400640 1 -- 172.21.15.196:0/401583572 >> 172.21.15.196:0/401583572 conn(0x7f91500d50f0 msgr2=0x7f91500d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:51.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.173+0000 7f9156400640 1 -- 172.21.15.196:0/401583572 shutdown_connections 2024-04-28T13:31:51.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.173+0000 7f9156400640 1 -- 172.21.15.196:0/401583572 wait complete. 2024-04-28T13:31:51.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.175+0000 7f9156400640 1 Processor -- start 2024-04-28T13:31:51.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.175+0000 7f9156400640 1 -- start start 2024-04-28T13:31:51.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.176+0000 7f9156400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9138004800 0x7f915015c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:51.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.176+0000 7f9156400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91500b3f90 0x7f91501f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:51.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.176+0000 7f9156400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91501f8e30 0x7f91500b9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:51.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.176+0000 7f9156400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91500d38e0 con 0x7f91500b3f90 2024-04-28T13:31:51.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.176+0000 7f9156400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f91500b6910 con 0x7f91501f8e30 2024-04-28T13:31:51.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.176+0000 7f9156400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f91500d3ea0 con 0x7f9138004800 2024-04-28T13:31:51.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.176+0000 7f914fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91500b3f90 0x7f91501f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:51.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.176+0000 7f914fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91500b3f90 0x7f91501f88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43628/0 (socket says 172.21.15.196:43628) 2024-04-28T13:31:51.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.176+0000 7f914fe00640 1 -- 172.21.15.196:0/3192069391 learned_addr learned my addr 172.21.15.196:0/3192069391 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:51.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.177+0000 7f9155a00640 1 --2- 172.21.15.196:0/3192069391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91501f8e30 0x7f91500b9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:51.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.177+0000 7f9155000640 1 --2- 172.21.15.196:0/3192069391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9138004800 0x7f915015c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:51.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.177+0000 7f914fe00640 1 -- 172.21.15.196:0/3192069391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9138004800 msgr2=0x7f915015c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:51.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.177+0000 7f914fe00640 1 --2- 172.21.15.196:0/3192069391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9138004800 0x7f915015c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:51.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.177+0000 7f914fe00640 1 -- 172.21.15.196:0/3192069391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91501f8e30 msgr2=0x7f91500b9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:51.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.177+0000 7f914fe00640 1 --2- 172.21.15.196:0/3192069391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91501f8e30 0x7f91500b9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:51.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.177+0000 7f914fe00640 1 -- 172.21.15.196:0/3192069391 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915015cbb0 con 0x7f91500b3f90 2024-04-28T13:31:51.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.177+0000 7f9155a00640 1 --2- 172.21.15.196:0/3192069391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91501f8e30 0x7f91500b9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:51.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.177+0000 7f914fe00640 1 --2- 172.21.15.196:0/3192069391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91500b3f90 0x7f91501f88f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9140008fd0 tx=0x7f9140001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:51.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.177+0000 7f914cc00640 1 -- 172.21.15.196:0/3192069391 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140040020 con 0x7f91500b3f90 2024-04-28T13:31:51.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.177+0000 7f914cc00640 1 -- 172.21.15.196:0/3192069391 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f914002be40 con 0x7f91500b3f90 2024-04-28T13:31:51.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.177+0000 7f914cc00640 1 -- 172.21.15.196:0/3192069391 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140001370 con 0x7f91500b3f90 2024-04-28T13:31:51.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.177+0000 7f9156400640 1 -- 172.21.15.196:0/3192069391 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9150161990 con 0x7f91500b3f90 2024-04-28T13:31:51.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.178+0000 7f9156400640 1 -- 172.21.15.196:0/3192069391 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91501f9600 con 0x7f91500b3f90 2024-04-28T13:31:51.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.179+0000 7f9156400640 1 -- 172.21.15.196:0/3192069391 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91500a3030 con 0x7f91500b3f90 2024-04-28T13:31:51.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.179+0000 7f914cc00640 1 -- 172.21.15.196:0/3192069391 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9140043020 con 0x7f91500b3f90 2024-04-28T13:31:51.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.180+0000 7f914cc00640 1 --2- 172.21.15.196:0/3192069391 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f913406c920 0x7f913406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:51.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.180+0000 7f914cc00640 1 -- 172.21.15.196:0/3192069391 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f91400aba50 con 0x7f91500b3f90 2024-04-28T13:31:51.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.181+0000 7f9155000640 1 --2- 172.21.15.196:0/3192069391 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f913406c920 0x7f913406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:51.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.182+0000 7f9155000640 1 --2- 172.21.15.196:0/3192069391 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f913406c920 0x7f913406ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9144009950 tx=0x7f9144008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:51.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.185+0000 7f914cc00640 1 -- 172.21.15.196:0/3192069391 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f914004a030 con 0x7f91500b3f90 2024-04-28T13:31:51.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.425+0000 7f9156400640 1 -- 172.21.15.196:0/3192069391 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f91500507a0 con 0x7f91500b3f90 2024-04-28T13:31:51.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.425+0000 7f914cc00640 1 -- 172.21.15.196:0/3192069391 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f915015cbb0 con 0x7f91500b3f90 2024-04-28T13:31:51.425 INFO:teuthology.orchestra.run.smithi196.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-28T13:31:51.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.428+0000 7f9156400640 1 -- 172.21.15.196:0/3192069391 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f913406c920 msgr2=0x7f913406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:51.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.428+0000 7f9156400640 1 --2- 172.21.15.196:0/3192069391 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f913406c920 0x7f913406ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9144009950 tx=0x7f9144008040 comp rx=0 tx=0).stop 2024-04-28T13:31:51.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.428+0000 7f9156400640 1 -- 172.21.15.196:0/3192069391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91500b3f90 msgr2=0x7f91501f88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:51.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.428+0000 7f9156400640 1 --2- 172.21.15.196:0/3192069391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91500b3f90 0x7f91501f88f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9140008fd0 tx=0x7f9140001120 comp rx=0 tx=0).stop 2024-04-28T13:31:51.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.428+0000 7f9156400640 1 -- 172.21.15.196:0/3192069391 shutdown_connections 2024-04-28T13:31:51.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.428+0000 7f9156400640 1 --2- 172.21.15.196:0/3192069391 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f913406c920 0x7f913406ede0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:51.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.428+0000 7f9156400640 1 --2- 172.21.15.196:0/3192069391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91501f8e30 0x7f91500b9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:51.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.428+0000 7f9156400640 1 --2- 172.21.15.196:0/3192069391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91500b3f90 0x7f91501f88f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:51.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.428+0000 7f9156400640 1 --2- 172.21.15.196:0/3192069391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9138004800 0x7f915015c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:51.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.428+0000 7f9156400640 1 -- 172.21.15.196:0/3192069391 >> 172.21.15.196:0/3192069391 conn(0x7f91500d50f0 msgr2=0x7f91500cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:51.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.428+0000 7f9156400640 1 -- 172.21.15.196:0/3192069391 shutdown_connections 2024-04-28T13:31:51.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:51.428+0000 7f9156400640 1 -- 172.21.15.196:0/3192069391 wait complete. 2024-04-28T13:31:51.447 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-28T13:31:51.447 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T13:31:51.447 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2024-04-28T13:31:51.447 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-28T13:31:51.447 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-28T13:31:51.447 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-28T13:31:51.447 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-28T13:31:51.447 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T13:31:51.447 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-28T13:31:51.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:51.448 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2024-04-28T13:31:51.505 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:31:51.505 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:31:51.505 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:31:51.506 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:51.506 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-28T13:31:51.506 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:51.506 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:31:51.506 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:51.506 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f0 2024-04-28T13:31:51.506 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:31:51.506 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:51.506 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f1 2024-04-28T13:31:51.506 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:31:51.506 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:51.506 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T13:31:51.506 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:51.507 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T13:31:51.507 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42608sec preferred_lft 42608sec 2024-04-28T13:31:51.507 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ec4:7aff:fe8f:4a23/64 scope link 2024-04-28T13:31:51.507 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:51.507 INFO:teuthology.orchestra.run.smithi203.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T13:31:51.507 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 1e:04:67:22:4e:82 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:31:51.507 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-28T13:31:51.507 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:51.507 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::4e3:c8ff:fea0:8dce/64 scope link 2024-04-28T13:31:51.507 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:51.507 INFO:teuthology.orchestra.run.smithi203.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:31:51.507 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 1e:04:67:22:4e:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T13:31:51.507 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::1c04:67ff:fe22:4e82/64 scope link 2024-04-28T13:31:51.507 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:51.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:51.508 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T13:31:51.588 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-28T13:31:51.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:51.592 DEBUG:teuthology.orchestra.run.smithi203:> route 2024-04-28T13:31:51.647 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2024-04-28T13:31:51.647 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T13:31:51.647 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T13:31:51.647 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T13:31:51.647 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T13:31:51.647 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:51.647 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-28T13:31:51.647 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T13:31:51.648 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T13:31:51.648 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T13:31:51.648 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-28T13:31:51.752 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:51.852 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:51.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:51.862 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-28T13:31:51.908 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-28T13:31:51.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:51.909 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list-id 2024-04-28T13:31:52.005 INFO:teuthology.orchestra.run.smithi203.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-28T13:31:52.006 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:52.006 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-28T13:31:52.006 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-28T13:31:52.006 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-28T13:31:52.006 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-28T13:31:52.114 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:52.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:52.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:52.197 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-28T13:31:52.273 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:31:52.273 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:31:52.273 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:31:52.273 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:52.273 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-28T13:31:52.273 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:52.273 INFO:teuthology.orchestra.run.smithi203.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T13:31:52.274 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:c9:0f:35:42:11 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-28T13:31:52.274 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-28T13:31:52.274 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:52.274 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::b8c9:fff:fe35:4211/64 scope link 2024-04-28T13:31:52.274 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:52.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:52.277 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-28T13:31:52.362 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:31:52.362 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:31:52.362 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:31:52.362 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:52.362 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-28T13:31:52.363 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:52.363 INFO:teuthology.orchestra.run.smithi203.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T13:31:52.363 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:c9:0f:35:42:11 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-28T13:31:52.363 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-28T13:31:52.363 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:52.363 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::b8c9:fff:fe35:4211/64 scope link 2024-04-28T13:31:52.363 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:31:52.367 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-28T13:31:52.367 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:52.368 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-28T13:31:52.368 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-28T13:31:52.368 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-28T13:31:52.368 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-28T13:31:52.368 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-28T13:31:52.368 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-28T13:31:52.368 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-28T13:31:52.475 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:52.635 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:52.645 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:31:52.645 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-28T13:31:52.645 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.1 up 2024-04-28T13:31:52.645 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set dev brx.1 master ceph-brx 2024-04-28T13:31:52.645 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-28T13:31:52.744 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:52.843 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:52.853 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:31:52.853 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-28T13:31:52.898 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-28T13:31:52.898 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:31:52.898 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-28T13:31:52.991 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-28T13:31:52.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:52.992 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-28T13:31:53.122 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-28T13:31:53.122 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-28T13:31:53.122 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:31:53.145 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-28T13:31:53.145 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-28T13:31:53.145 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:31:53.145 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:31:53.145 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:31:53.170 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-28T13:31:53.171 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-28T13:31:53.171 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:31:53.171 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:31:53.171 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:31:53.171 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-28T13:31:53.171 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-28T13:31:53.171 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=270aea32-2e16-4a97-85f7-aeae39bc8c76" will pass to kernel 2024-04-28T13:31:53.173 INFO:teuthology.orchestra.run.smithi203.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-28T13:31:53.173 INFO:teuthology.orchestra.run.smithi203.stdout: You just mounted a file system that supports labels which does not 2024-04-28T13:31:53.173 INFO:teuthology.orchestra.run.smithi203.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-28T13:31:53.173 INFO:teuthology.orchestra.run.smithi203.stdout: applications will generate AVC messages and not be allowed access to 2024-04-28T13:31:53.173 INFO:teuthology.orchestra.run.smithi203.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-28T13:31:53.177 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-28T13:31:53.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:31:53.177 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-28T13:31:53.253 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:31:53.355 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:31:53.360 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:53.369 INFO:teuthology.orchestra.run.smithi203.stdout:client4445 2024-04-28T13:31:53.370 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:31:53.370 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:31:53.421 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:31:53.526 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:31:53.530 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:53.540 INFO:teuthology.orchestra.run.smithi203.stdout:client4445 2024-04-28T13:31:53.540 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:31:53.636 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:31:53.640 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:53.650 INFO:teuthology.orchestra.run.smithi203.stdout:client4445 2024-04-28T13:31:53.650 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/270aea32-2e16-4a97-85f7-aeae39bc8c76.client4445/status') 2024-04-28T13:31:53.754 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:31:53.783 INFO:teuthology.orchestra.run.smithi203.stdout:instance: client.4445 (3)192.168.0.2:0/485714222 2024-04-28T13:31:53.783 INFO:teuthology.orchestra.run.smithi203.stdout:blocklisted: false 2024-04-28T13:31:53.783 INFO:teuthology.orchestra.run.smithi203.stderr:0+1 records in 2024-04-28T13:31:53.783 INFO:teuthology.orchestra.run.smithi203.stderr:0+1 records out 2024-04-28T13:31:53.783 INFO:teuthology.orchestra.run.smithi203.stderr:68 bytes copied, 7.5248e-05 s, 904 kB/s 2024-04-28T13:31:53.786 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:31:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:31:53.796 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-28T13:31:53.812 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:54.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.022+0000 7f24f1000640 1 Processor -- start 2024-04-28T13:31:54.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.022+0000 7f24f1000640 1 -- start start 2024-04-28T13:31:54.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.024+0000 7f24f1000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ec163bf0 0x7f24ec163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.024+0000 7f24f1000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f24ec163570 0x7f24ec0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.024+0000 7f24f1000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24ec162da0 con 0x7f24ec1629c0 2024-04-28T13:31:54.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.024+0000 7f24f1000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f24ec0d3e20 con 0x7f24ec163570 2024-04-28T13:31:54.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.024+0000 7f24f1000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f24ec0b6190 con 0x7f24ec163bf0 2024-04-28T13:31:54.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.024+0000 7f24ebe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f24ec163570 0x7f24ec0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:54.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.024+0000 7f24eb400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f24ec1629c0 0x7f24ec0d3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:34264/0 (socket says 172.21.15.196:34264) 2024-04-28T13:31:54.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.024+0000 7f24eb400640 1 -- 172.21.15.196:0/1424548990 learned_addr learned my addr 172.21.15.196:0/1424548990 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:54.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.024+0000 7f24e2a00640 1 --2- 172.21.15.196:0/1424548990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ec163bf0 0x7f24ec163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:54.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.024+0000 7f24ebe00640 1 -- 172.21.15.196:0/1424548990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ec163bf0 msgr2=0x7f24ec163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:54.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.024+0000 7f24ebe00640 1 --2- 172.21.15.196:0/1424548990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ec163bf0 0x7f24ec163030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.025+0000 7f24ebe00640 1 -- 172.21.15.196:0/1424548990 >> v1:172.21.15.196:6789/0 conn(0x7f24ec1629c0 legacy=0x7f24ec0d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.025+0000 7f24ebe00640 1 -- 172.21.15.196:0/1424548990 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ec0b6470 con 0x7f24ec163570 2024-04-28T13:31:54.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.025+0000 7f24e2a00640 1 --2- 172.21.15.196:0/1424548990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ec163bf0 0x7f24ec163030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:54.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.025+0000 7f24eaa00640 1 -- 172.21.15.196:0/1424548990 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2835488747 0 0) 0x7f24ec162da0 con 0x7f24ec1629c0 2024-04-28T13:31:54.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.026+0000 7f24ebe00640 1 --2- 172.21.15.196:0/1424548990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24ec163570 0x7f24ec0b2380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009d90 tx=0x7f24e402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbed1c6d284ee85 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:54.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.026+0000 7f24eaa00640 1 -- 172.21.15.196:0/1424548990 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4007740 con 0x7f24ec163570 2024-04-28T13:31:54.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.026+0000 7f24eaa00640 1 -- 172.21.15.196:0/1424548990 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e40078e0 con 0x7f24ec163570 2024-04-28T13:31:54.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.026+0000 7f24eaa00640 1 -- 172.21.15.196:0/1424548990 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4007cb0 con 0x7f24ec163570 2024-04-28T13:31:54.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.026+0000 7f24f1000640 1 -- 172.21.15.196:0/1424548990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24ec163570 msgr2=0x7f24ec0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.026+0000 7f24f1000640 1 --2- 172.21.15.196:0/1424548990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24ec163570 0x7f24ec0b2380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009d90 tx=0x7f24e402a660 comp rx=0 tx=0).stop 2024-04-28T13:31:54.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.027+0000 7f24f1000640 1 -- 172.21.15.196:0/1424548990 shutdown_connections 2024-04-28T13:31:54.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.027+0000 7f24f1000640 1 --2- 172.21.15.196:0/1424548990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24ec163570 0x7f24ec0b2380 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.027+0000 7f24f1000640 1 --2- 172.21.15.196:0/1424548990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ec163bf0 0x7f24ec163030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.027+0000 7f24f1000640 1 -- 172.21.15.196:0/1424548990 >> 172.21.15.196:0/1424548990 conn(0x7f24ec09e570 msgr2=0x7f24ec0d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:54.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.027+0000 7f24f1000640 1 -- 172.21.15.196:0/1424548990 shutdown_connections 2024-04-28T13:31:54.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.027+0000 7f24f1000640 1 -- 172.21.15.196:0/1424548990 wait complete. 2024-04-28T13:31:54.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.028+0000 7f24f1000640 1 Processor -- start 2024-04-28T13:31:54.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.028+0000 7f24f1000640 1 -- start start 2024-04-28T13:31:54.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.029+0000 7f24f1000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24ec1629c0 0x7f24ec0b8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.029+0000 7f24f1000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24ec163570 0x7f24ec1f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.029+0000 7f24f1000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ec163bf0 0x7f24ec1f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.029+0000 7f24f1000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec0b1e60 con 0x7f24ec1629c0 2024-04-28T13:31:54.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.029+0000 7f24f1000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec0b6a30 con 0x7f24ec163570 2024-04-28T13:31:54.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.029+0000 7f24f1000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f24ec0b28c0 con 0x7f24ec163bf0 2024-04-28T13:31:54.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.029+0000 7f24eb400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24ec1629c0 0x7f24ec0b8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:54.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.029+0000 7f24eb400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24ec1629c0 0x7f24ec0b8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52270/0 (socket says 172.21.15.196:52270) 2024-04-28T13:31:54.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.029+0000 7f24eb400640 1 -- 172.21.15.196:0/2784770240 learned_addr learned my addr 172.21.15.196:0/2784770240 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:54.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.030+0000 7f24ebe00640 1 --2- 172.21.15.196:0/2784770240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ec163bf0 0x7f24ec1f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:54.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.030+0000 7f24eb400640 1 -- 172.21.15.196:0/2784770240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ec163bf0 msgr2=0x7f24ec1f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.030+0000 7f24eb400640 1 --2- 172.21.15.196:0/2784770240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ec163bf0 0x7f24ec1f5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.030+0000 7f24eb400640 1 -- 172.21.15.196:0/2784770240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24ec163570 msgr2=0x7f24ec1f53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.030+0000 7f24eb400640 1 --2- 172.21.15.196:0/2784770240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24ec163570 0x7f24ec1f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.030+0000 7f24eb400640 1 -- 172.21.15.196:0/2784770240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ec0b9410 con 0x7f24ec1629c0 2024-04-28T13:31:54.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.030+0000 7f24eb400640 1 --2- 172.21.15.196:0/2784770240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24ec1629c0 0x7f24ec0b8950 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00c4d0 tx=0x7f24dc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:54.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.031+0000 7f24e3e00640 1 -- 172.21.15.196:0/2784770240 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc018070 con 0x7f24ec1629c0 2024-04-28T13:31:54.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.031+0000 7f24e3e00640 1 -- 172.21.15.196:0/2784770240 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24dc0074e0 con 0x7f24ec1629c0 2024-04-28T13:31:54.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.031+0000 7f24e3e00640 1 -- 172.21.15.196:0/2784770240 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc004a20 con 0x7f24ec1629c0 2024-04-28T13:31:54.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.031+0000 7f24f1000640 1 -- 172.21.15.196:0/2784770240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ec0cc4a0 con 0x7f24ec1629c0 2024-04-28T13:31:54.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.031+0000 7f24f1000640 1 -- 172.21.15.196:0/2784770240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ec0ccf80 con 0x7f24ec1629c0 2024-04-28T13:31:54.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.032+0000 7f24f1000640 1 -- 172.21.15.196:0/2784770240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ec06a290 con 0x7f24ec1629c0 2024-04-28T13:31:54.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.033+0000 7f24e3e00640 1 -- 172.21.15.196:0/2784770240 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24dc01b070 con 0x7f24ec1629c0 2024-04-28T13:31:54.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.034+0000 7f24e3e00640 1 --2- 172.21.15.196:0/2784770240 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f24c806c8f0 0x7f24c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.034+0000 7f24e3e00640 1 -- 172.21.15.196:0/2784770240 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f24dc08af70 con 0x7f24ec1629c0 2024-04-28T13:31:54.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.035+0000 7f24e2a00640 1 --2- 172.21.15.196:0/2784770240 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f24c806c8f0 0x7f24c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:54.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.036+0000 7f24e2a00640 1 --2- 172.21.15.196:0/2784770240 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f24c806c8f0 0x7f24c806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f24ec1f82b0 tx=0x7f24d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:54.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.039+0000 7f24e3e00640 1 -- 172.21.15.196:0/2784770240 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f24dc059300 con 0x7f24ec1629c0 2024-04-28T13:31:54.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.241+0000 7f24f1000640 1 -- 172.21.15.196:0/2784770240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24ec0cd210 con 0x7f24ec1629c0 2024-04-28T13:31:54.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.243+0000 7f24e3e00640 1 -- 172.21.15.196:0/2784770240 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f24dc058c00 con 0x7f24ec1629c0 2024-04-28T13:31:54.243 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:54.244 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":20,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:22.624564+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:54.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.245+0000 7f24f1000640 1 -- 172.21.15.196:0/2784770240 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f24c806c8f0 msgr2=0x7f24c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.245+0000 7f24f1000640 1 --2- 172.21.15.196:0/2784770240 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f24c806c8f0 0x7f24c806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f24ec1f82b0 tx=0x7f24d4008040 comp rx=0 tx=0).stop 2024-04-28T13:31:54.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.245+0000 7f24f1000640 1 -- 172.21.15.196:0/2784770240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24ec1629c0 msgr2=0x7f24ec0b8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.245+0000 7f24f1000640 1 --2- 172.21.15.196:0/2784770240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24ec1629c0 0x7f24ec0b8950 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00c4d0 tx=0x7f24dc00c990 comp rx=0 tx=0).stop 2024-04-28T13:31:54.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.245+0000 7f24f1000640 1 -- 172.21.15.196:0/2784770240 shutdown_connections 2024-04-28T13:31:54.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.245+0000 7f24f1000640 1 --2- 172.21.15.196:0/2784770240 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f24c806c8f0 0x7f24c806edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.245+0000 7f24f1000640 1 --2- 172.21.15.196:0/2784770240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ec163bf0 0x7f24ec1f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.245+0000 7f24f1000640 1 --2- 172.21.15.196:0/2784770240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24ec163570 0x7f24ec1f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.245+0000 7f24f1000640 1 --2- 172.21.15.196:0/2784770240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24ec1629c0 0x7f24ec0b8950 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.246+0000 7f24f1000640 1 -- 172.21.15.196:0/2784770240 >> 172.21.15.196:0/2784770240 conn(0x7f24ec09e570 msgr2=0x7f24ec06dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:54.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.246+0000 7f24f1000640 1 -- 172.21.15.196:0/2784770240 shutdown_connections 2024-04-28T13:31:54.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.246+0000 7f24f1000640 1 -- 172.21.15.196:0/2784770240 wait complete. 2024-04-28T13:31:54.265 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:31:19.280975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:31:20.037066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:31:54.266 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:31:54.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.470+0000 7f01c6c00640 1 Processor -- start 2024-04-28T13:31:54.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.470+0000 7f01c6c00640 1 -- start start 2024-04-28T13:31:54.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.472+0000 7f01c6c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01c00b04c0 0x7f01c00af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.472+0000 7f01c6c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c01607d0 con 0x7f01c015fa10 2024-04-28T13:31:54.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.472+0000 7f01c6c00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c00aea50 con 0x7f01c0145960 2024-04-28T13:31:54.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.472+0000 7f01c6c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f01c01605b0 con 0x7f01c00b04c0 2024-04-28T13:31:54.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.473+0000 7f01c4c00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f01c015fa10 0x7f01c015fea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:34280/0 (socket says 172.21.15.196:34280) 2024-04-28T13:31:54.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.473+0000 7f01c4c00640 1 -- 172.21.15.196:0/3447981963 learned_addr learned my addr 172.21.15.196:0/3447981963 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:54.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.473+0000 7f01bf400640 1 --2- 172.21.15.196:0/3447981963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01c00b04c0 0x7f01c00af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:54.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.473+0000 7f01bea00640 1 -- 172.21.15.196:0/3447981963 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1658762570 0 0) 0x7f01c01607d0 con 0x7f01c015fa10 2024-04-28T13:31:54.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.473+0000 7f01bea00640 1 -- 172.21.15.196:0/3447981963 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01a4003520 con 0x7f01c015fa10 2024-04-28T13:31:54.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.474+0000 7f01bf400640 1 -- 172.21.15.196:0/3447981963 >> v1:172.21.15.203:6789/0 conn(0x7f01c0145960 legacy=0x7f01c00afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.474+0000 7f01bea00640 1 -- 172.21.15.196:0/3447981963 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3494741181 0 0) 0x7f01c00aea50 con 0x7f01c0145960 2024-04-28T13:31:54.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.474+0000 7f01bf400640 1 -- 172.21.15.196:0/3447981963 >> v1:172.21.15.196:6789/0 conn(0x7f01c015fa10 legacy=0x7f01c015fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.474+0000 7f01bf400640 1 -- 172.21.15.196:0/3447981963 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c00aed30 con 0x7f01c00b04c0 2024-04-28T13:31:54.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.475+0000 7f01bf400640 1 --2- 172.21.15.196:0/3447981963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01c00b04c0 0x7f01c00af530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f01a80097d0 tx=0x7f01a802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1c39ff89d6bbe93 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:54.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.476+0000 7f01bea00640 1 -- 172.21.15.196:0/3447981963 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8037070 con 0x7f01c00b04c0 2024-04-28T13:31:54.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.476+0000 7f01bea00640 1 -- 172.21.15.196:0/3447981963 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a8032440 con 0x7f01c00b04c0 2024-04-28T13:31:54.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.476+0000 7f01bea00640 1 -- 172.21.15.196:0/3447981963 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8032810 con 0x7f01c00b04c0 2024-04-28T13:31:54.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.477+0000 7f01c6c00640 1 -- 172.21.15.196:0/3447981963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01c00b04c0 msgr2=0x7f01c00af530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.477+0000 7f01c6c00640 1 --2- 172.21.15.196:0/3447981963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01c00b04c0 0x7f01c00af530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f01a80097d0 tx=0x7f01a802a210 comp rx=0 tx=0).stop 2024-04-28T13:31:54.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.477+0000 7f01c6c00640 1 -- 172.21.15.196:0/3447981963 shutdown_connections 2024-04-28T13:31:54.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.477+0000 7f01c6c00640 1 --2- 172.21.15.196:0/3447981963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01c00b04c0 0x7f01c00af530 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.477+0000 7f01c6c00640 1 -- 172.21.15.196:0/3447981963 >> 172.21.15.196:0/3447981963 conn(0x7f01c00cd8a0 msgr2=0x7f01c00cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:54.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.477+0000 7f01c6c00640 1 -- 172.21.15.196:0/3447981963 shutdown_connections 2024-04-28T13:31:54.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.477+0000 7f01c6c00640 1 -- 172.21.15.196:0/3447981963 wait complete. 2024-04-28T13:31:54.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.479+0000 7f01c6c00640 1 Processor -- start 2024-04-28T13:31:54.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.479+0000 7f01c6c00640 1 -- start start 2024-04-28T13:31:54.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.480+0000 7f01c6c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01c00b04c0 0x7f01c00b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.480+0000 7f01c6c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01c0145960 0x7f01c00cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.480+0000 7f01c6c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01c015fa10 0x7f01c015e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.480+0000 7f01c6c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f01c00734f0 con 0x7f01c0145960 2024-04-28T13:31:54.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.480+0000 7f01c6c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0073370 con 0x7f01c00b04c0 2024-04-28T13:31:54.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.480+0000 7f01c6c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f01c0073670 con 0x7f01c015fa10 2024-04-28T13:31:54.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.481+0000 7f01bf400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01c0145960 0x7f01c00cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:54.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.481+0000 7f01bf400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01c0145960 0x7f01c00cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52286/0 (socket says 172.21.15.196:52286) 2024-04-28T13:31:54.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.481+0000 7f01bf400640 1 -- 172.21.15.196:0/2607171121 learned_addr learned my addr 172.21.15.196:0/2607171121 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:54.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.481+0000 7f01bfe00640 1 --2- 172.21.15.196:0/2607171121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01c00b04c0 0x7f01c00b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:54.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.481+0000 7f01c4c00640 1 --2- 172.21.15.196:0/2607171121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01c015fa10 0x7f01c015e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:54.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.481+0000 7f01bf400640 1 -- 172.21.15.196:0/2607171121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01c015fa10 msgr2=0x7f01c015e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.481+0000 7f01bf400640 1 --2- 172.21.15.196:0/2607171121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01c015fa10 0x7f01c015e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.481+0000 7f01bf400640 1 -- 172.21.15.196:0/2607171121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01c00b04c0 msgr2=0x7f01c00b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.481+0000 7f01bf400640 1 --2- 172.21.15.196:0/2607171121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01c00b04c0 0x7f01c00b6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.481+0000 7f01bf400640 1 -- 172.21.15.196:0/2607171121 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c01370b0 con 0x7f01c0145960 2024-04-28T13:31:54.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.482+0000 7f01bf400640 1 --2- 172.21.15.196:0/2607171121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01c0145960 0x7f01c00cc230 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f01a80097d0 tx=0x7f01a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:54.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.482+0000 7f01b7e00640 1 -- 172.21.15.196:0/2607171121 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8037040 con 0x7f01c0145960 2024-04-28T13:31:54.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.482+0000 7f01b7e00640 1 -- 172.21.15.196:0/2607171121 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a8004230 con 0x7f01c0145960 2024-04-28T13:31:54.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.482+0000 7f01c6c00640 1 -- 172.21.15.196:0/2607171121 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c0137390 con 0x7f01c0145960 2024-04-28T13:31:54.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.482+0000 7f01b7e00640 1 -- 172.21.15.196:0/2607171121 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8003610 con 0x7f01c0145960 2024-04-28T13:31:54.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.482+0000 7f01c6c00640 1 -- 172.21.15.196:0/2607171121 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c0137950 con 0x7f01c0145960 2024-04-28T13:31:54.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.483+0000 7f01c6c00640 1 -- 172.21.15.196:0/2607171121 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c006a290 con 0x7f01c0145960 2024-04-28T13:31:54.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.485+0000 7f01b7e00640 1 -- 172.21.15.196:0/2607171121 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01a80037b0 con 0x7f01c0145960 2024-04-28T13:31:54.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.485+0000 7f01b7e00640 1 --2- 172.21.15.196:0/2607171121 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f019c06c920 0x7f019c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.486+0000 7f01b7e00640 1 -- 172.21.15.196:0/2607171121 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f01a80ac110 con 0x7f01c0145960 2024-04-28T13:31:54.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.486+0000 7f01bfe00640 1 --2- 172.21.15.196:0/2607171121 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f019c06c920 0x7f019c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:54.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.487+0000 7f01bfe00640 1 --2- 172.21.15.196:0/2607171121 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f019c06c920 0x7f019c06ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f01b0007fd0 tx=0x7f01b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:54.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.490+0000 7f01b7e00640 1 -- 172.21.15.196:0/2607171121 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f01a8041020 con 0x7f01c0145960 2024-04-28T13:31:54.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.692+0000 7f01c6c00640 1 -- 172.21.15.196:0/2607171121 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f01c00cd040 con 0x7f01c0145960 2024-04-28T13:31:54.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.693+0000 7f01b7e00640 1 -- 172.21.15.196:0/2607171121 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f01a807a530 con 0x7f01c0145960 2024-04-28T13:31:54.693 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:31:54.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.696+0000 7f01c6c00640 1 -- 172.21.15.196:0/2607171121 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f019c06c920 msgr2=0x7f019c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.696+0000 7f01c6c00640 1 --2- 172.21.15.196:0/2607171121 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f019c06c920 0x7f019c06ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f01b0007fd0 tx=0x7f01b0009040 comp rx=0 tx=0).stop 2024-04-28T13:31:54.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.696+0000 7f01c6c00640 1 -- 172.21.15.196:0/2607171121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01c0145960 msgr2=0x7f01c00cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.696+0000 7f01c6c00640 1 --2- 172.21.15.196:0/2607171121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01c0145960 0x7f01c00cc230 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f01a80097d0 tx=0x7f01a80046c0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.696+0000 7f01c6c00640 1 -- 172.21.15.196:0/2607171121 shutdown_connections 2024-04-28T13:31:54.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.696+0000 7f01c6c00640 1 --2- 172.21.15.196:0/2607171121 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f019c06c920 0x7f019c06ede0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.696+0000 7f01c6c00640 1 --2- 172.21.15.196:0/2607171121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01c015fa10 0x7f01c015e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.696+0000 7f01c6c00640 1 --2- 172.21.15.196:0/2607171121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01c0145960 0x7f01c00cc230 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.696+0000 7f01c6c00640 1 --2- 172.21.15.196:0/2607171121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01c00b04c0 0x7f01c00b6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.696+0000 7f01c6c00640 1 -- 172.21.15.196:0/2607171121 >> 172.21.15.196:0/2607171121 conn(0x7f01c00cd8a0 msgr2=0x7f01c00b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:54.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.696+0000 7f01c6c00640 1 -- 172.21.15.196:0/2607171121 shutdown_connections 2024-04-28T13:31:54.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.696+0000 7f01c6c00640 1 -- 172.21.15.196:0/2607171121 wait complete. 2024-04-28T13:31:54.714 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:31:54.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.920+0000 7f09cb200640 1 Processor -- start 2024-04-28T13:31:54.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.920+0000 7f09cb200640 1 -- start start 2024-04-28T13:31:54.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.921+0000 7f09cb200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09cc15be50 0x7f09cc0728d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.921+0000 7f09cb200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f09cc0af930 0x7f09cc0d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.921+0000 7f09cb200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f09cc072e10 0x7f09cc0d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.921+0000 7f09cb200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f09cc073ec0 con 0x7f09cc072e10 2024-04-28T13:31:54.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.921+0000 7f09cb200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f09cc0732b0 con 0x7f09cc0af930 2024-04-28T13:31:54.922 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.921+0000 7f09cb200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f09cc073590 con 0x7f09cc15be50 2024-04-28T13:31:54.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.922+0000 7f09ca800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f09cc072e10 0x7f09cc0d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:54.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.922+0000 7f09ca800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f09cc072e10 0x7f09cc0d15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52288/0 (socket says 172.21.15.196:52288) 2024-04-28T13:31:54.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.922+0000 7f09ca800640 1 -- 172.21.15.196:0/1184917619 learned_addr learned my addr 172.21.15.196:0/1184917619 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:54.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.922+0000 7f09c9400640 1 --2- 172.21.15.196:0/1184917619 >> v2:172.21.15.203:3300/0 conn(0x7f09cc0af930 0x7f09cc0d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:54.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.922+0000 7f09c9e00640 1 --2- 172.21.15.196:0/1184917619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09cc15be50 0x7f09cc0728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:54.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.922+0000 7f09ca800640 1 -- 172.21.15.196:0/1184917619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09cc15be50 msgr2=0x7f09cc0728d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:54.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.922+0000 7f09ca800640 1 --2- 172.21.15.196:0/1184917619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09cc15be50 0x7f09cc0728d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.922+0000 7f09ca800640 1 -- 172.21.15.196:0/1184917619 >> v2:172.21.15.203:3300/0 conn(0x7f09cc0af930 msgr2=0x7f09cc0d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:54.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.922+0000 7f09ca800640 1 --2- 172.21.15.196:0/1184917619 >> v2:172.21.15.203:3300/0 conn(0x7f09cc0af930 0x7f09cc0d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.922+0000 7f09ca800640 1 -- 172.21.15.196:0/1184917619 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09cc073870 con 0x7f09cc072e10 2024-04-28T13:31:54.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.924+0000 7f09ca800640 1 --2- 172.21.15.196:0/1184917619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09cc072e10 0x7f09cc0d15e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0737a0 tx=0x7f09c402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70764a808a53063f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:54.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.924+0000 7f09c8a00640 1 -- 172.21.15.196:0/1184917619 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4036070 con 0x7f09cc072e10 2024-04-28T13:31:54.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.924+0000 7f09c8a00640 1 -- 172.21.15.196:0/1184917619 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c4031600 con 0x7f09cc072e10 2024-04-28T13:31:54.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.924+0000 7f09c8a00640 1 -- 172.21.15.196:0/1184917619 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c40319d0 con 0x7f09cc072e10 2024-04-28T13:31:54.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.925+0000 7f09cb200640 1 -- 172.21.15.196:0/1184917619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09cc072e10 msgr2=0x7f09cc0d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.925+0000 7f09cb200640 1 --2- 172.21.15.196:0/1184917619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09cc072e10 0x7f09cc0d15e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0737a0 tx=0x7f09c402a680 comp rx=0 tx=0).stop 2024-04-28T13:31:54.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.925+0000 7f09cb200640 1 -- 172.21.15.196:0/1184917619 shutdown_connections 2024-04-28T13:31:54.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.925+0000 7f09cb200640 1 --2- 172.21.15.196:0/1184917619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09cc072e10 0x7f09cc0d15e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.925+0000 7f09cb200640 1 --2- 172.21.15.196:0/1184917619 >> v2:172.21.15.203:3300/0 conn(0x7f09cc0af930 0x7f09cc0d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.925+0000 7f09cb200640 1 --2- 172.21.15.196:0/1184917619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09cc15be50 0x7f09cc0728d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.925+0000 7f09cb200640 1 -- 172.21.15.196:0/1184917619 >> 172.21.15.196:0/1184917619 conn(0x7f09cc06c6c0 msgr2=0x7f09cc06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:54.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.925+0000 7f09cb200640 1 -- 172.21.15.196:0/1184917619 shutdown_connections 2024-04-28T13:31:54.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.925+0000 7f09cb200640 1 -- 172.21.15.196:0/1184917619 wait complete. 2024-04-28T13:31:54.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.927+0000 7f09cb200640 1 Processor -- start 2024-04-28T13:31:54.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.927+0000 7f09cb200640 1 -- start start 2024-04-28T13:31:54.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.928+0000 7f09cb200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09cc072e10 0x7f09cc0b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.928+0000 7f09cb200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09cc15be50 0x7f09cc1f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.928+0000 7f09cb200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09cc09c560 0x7f09cc0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.928+0000 7f09cb200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc0b3c90 con 0x7f09cc15be50 2024-04-28T13:31:54.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.928+0000 7f09cb200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc15c230 con 0x7f09cc09c560 2024-04-28T13:31:54.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.928+0000 7f09cb200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f09cc0b3e10 con 0x7f09cc072e10 2024-04-28T13:31:54.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.928+0000 7f09c9400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09cc15be50 0x7f09cc1f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:54.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.928+0000 7f09c9400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09cc15be50 0x7f09cc1f5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52304/0 (socket says 172.21.15.196:52304) 2024-04-28T13:31:54.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.928+0000 7f09c9400640 1 -- 172.21.15.196:0/2125642391 learned_addr learned my addr 172.21.15.196:0/2125642391 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:54.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.928+0000 7f09c9400640 1 -- 172.21.15.196:0/2125642391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09cc072e10 msgr2=0x7f09cc0b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.929+0000 7f09c9400640 1 --2- 172.21.15.196:0/2125642391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09cc072e10 0x7f09cc0b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.929+0000 7f09c9400640 1 -- 172.21.15.196:0/2125642391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09cc09c560 msgr2=0x7f09cc0b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:54.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.929+0000 7f09c9400640 1 --2- 172.21.15.196:0/2125642391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09cc09c560 0x7f09cc0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:54.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.929+0000 7f09c9400640 1 -- 172.21.15.196:0/2125642391 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09cc1f5d60 con 0x7f09cc15be50 2024-04-28T13:31:54.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.929+0000 7f09c9400640 1 --2- 172.21.15.196:0/2125642391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09cc15be50 0x7f09cc1f5820 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f09b400e9e0 tx=0x7f09b400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:54.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.929+0000 7f09c2000640 1 -- 172.21.15.196:0/2125642391 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4017070 con 0x7f09cc15be50 2024-04-28T13:31:54.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.929+0000 7f09c2000640 1 -- 172.21.15.196:0/2125642391 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09b400c9d0 con 0x7f09cc15be50 2024-04-28T13:31:54.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.929+0000 7f09c2000640 1 -- 172.21.15.196:0/2125642391 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b400cdc0 con 0x7f09cc15be50 2024-04-28T13:31:54.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.929+0000 7f09cb200640 1 -- 172.21.15.196:0/2125642391 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09cc0b35e0 con 0x7f09cc15be50 2024-04-28T13:31:54.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.929+0000 7f09cb200640 1 -- 172.21.15.196:0/2125642391 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09cc0cad70 con 0x7f09cc15be50 2024-04-28T13:31:54.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.931+0000 7f09cb200640 1 -- 172.21.15.196:0/2125642391 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09cc050730 con 0x7f09cc15be50 2024-04-28T13:31:54.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.932+0000 7f09c2000640 1 -- 172.21.15.196:0/2125642391 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09b401a070 con 0x7f09cc15be50 2024-04-28T13:31:54.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.932+0000 7f09c2000640 1 --2- 172.21.15.196:0/2125642391 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f09a806c8f0 0x7f09a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:54.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.933+0000 7f09c2000640 1 -- 172.21.15.196:0/2125642391 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f09b408bc40 con 0x7f09cc15be50 2024-04-28T13:31:54.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.933+0000 7f09c9e00640 1 --2- 172.21.15.196:0/2125642391 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f09a806c8f0 0x7f09a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:54.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.934+0000 7f09c9e00640 1 --2- 172.21.15.196:0/2125642391 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f09a806c8f0 0x7f09a806edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f09cc1570a0 tx=0x7f09bc009460 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:54.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:54.937+0000 7f09c2000640 1 -- 172.21.15.196:0/2125642391 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f09cc1f5d60 con 0x7f09cc15be50 2024-04-28T13:31:55.139 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.139+0000 7f09cb200640 1 -- 172.21.15.196:0/2125642391 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f09cc0b4c80 con 0x7f09cc15be50 2024-04-28T13:31:55.139 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.140+0000 7f09c2000640 1 -- 172.21.15.196:0/2125642391 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f09b405a060 con 0x7f09cc15be50 2024-04-28T13:31:55.140 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:31:55.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.142+0000 7f09cb200640 1 -- 172.21.15.196:0/2125642391 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f09a806c8f0 msgr2=0x7f09a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:55.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.142+0000 7f09cb200640 1 --2- 172.21.15.196:0/2125642391 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f09a806c8f0 0x7f09a806edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f09cc1570a0 tx=0x7f09bc009460 comp rx=0 tx=0).stop 2024-04-28T13:31:55.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.142+0000 7f09cb200640 1 -- 172.21.15.196:0/2125642391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09cc15be50 msgr2=0x7f09cc1f5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:55.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.142+0000 7f09cb200640 1 --2- 172.21.15.196:0/2125642391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09cc15be50 0x7f09cc1f5820 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f09b400e9e0 tx=0x7f09b400eea0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.142+0000 7f09cb200640 1 -- 172.21.15.196:0/2125642391 shutdown_connections 2024-04-28T13:31:55.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.142+0000 7f09cb200640 1 --2- 172.21.15.196:0/2125642391 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f09a806c8f0 0x7f09a806edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.142+0000 7f09cb200640 1 --2- 172.21.15.196:0/2125642391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09cc09c560 0x7f09cc0b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.142+0000 7f09cb200640 1 --2- 172.21.15.196:0/2125642391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09cc15be50 0x7f09cc1f5820 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.142+0000 7f09cb200640 1 --2- 172.21.15.196:0/2125642391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09cc072e10 0x7f09cc0b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.142+0000 7f09cb200640 1 -- 172.21.15.196:0/2125642391 >> 172.21.15.196:0/2125642391 conn(0x7f09cc06c6c0 msgr2=0x7f09cc06f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:55.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.142+0000 7f09cb200640 1 -- 172.21.15.196:0/2125642391 shutdown_connections 2024-04-28T13:31:55.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.142+0000 7f09cb200640 1 -- 172.21.15.196:0/2125642391 wait complete. 2024-04-28T13:31:55.160 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:31:55.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.361+0000 7f346e400640 1 Processor -- start 2024-04-28T13:31:55.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.361+0000 7f346e400640 1 -- start start 2024-04-28T13:31:55.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.363+0000 7f346e400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f34681628c0 0x7f34680d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:55.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.363+0000 7f346e400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34680d1ef0 0x7f34680d2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:55.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.363+0000 7f346e400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34680d2890 con 0x7f34680b83d0 2024-04-28T13:31:55.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.363+0000 7f346e400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f34680b7f40 con 0x7f34681628c0 2024-04-28T13:31:55.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.363+0000 7f346e400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f34680d0710 con 0x7f34680d1ef0 2024-04-28T13:31:55.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.363+0000 7f346da00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f34680b83d0 0x7f34680b8860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:34284/0 (socket says 172.21.15.196:34284) 2024-04-28T13:31:55.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.363+0000 7f346da00640 1 -- 172.21.15.196:0/969449493 learned_addr learned my addr 172.21.15.196:0/969449493 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:55.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.364+0000 7f3467e00640 1 --2- 172.21.15.196:0/969449493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34680d1ef0 0x7f34680d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:55.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.364+0000 7f346d000640 1 --2- 172.21.15.196:0/969449493 >> v2:172.21.15.203:3300/0 conn(0x7f34681628c0 0x7f34680d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:55.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.364+0000 7f3467400640 1 -- 172.21.15.196:0/969449493 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3420783538 0 0) 0x7f34680d2890 con 0x7f34680b83d0 2024-04-28T13:31:55.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.364+0000 7f3467400640 1 -- 172.21.15.196:0/969449493 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3450003520 con 0x7f34680b83d0 2024-04-28T13:31:55.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.364+0000 7f3467400640 1 -- 172.21.15.196:0/969449493 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3277950813 0 0) 0x7f3450003520 con 0x7f34680b83d0 2024-04-28T13:31:55.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.364+0000 7f346d000640 1 -- 172.21.15.196:0/969449493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34680d1ef0 msgr2=0x7f34680d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:55.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.364+0000 7f346d000640 1 --2- 172.21.15.196:0/969449493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34680d1ef0 0x7f34680d2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.364+0000 7f346d000640 1 -- 172.21.15.196:0/969449493 >> v1:172.21.15.196:6789/0 conn(0x7f34680b83d0 legacy=0x7f34680b8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:55.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.364+0000 7f346d000640 1 -- 172.21.15.196:0/969449493 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34680d09f0 con 0x7f34681628c0 2024-04-28T13:31:55.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.364+0000 7f3467e00640 1 --2- 172.21.15.196:0/969449493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34680d1ef0 0x7f34680d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:31:55.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.366+0000 7f346d000640 1 --2- 172.21.15.196:0/969449493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34681628c0 0x7f34680d19b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f345c0097a0 tx=0x7f345c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ed99ad7e523fdcb server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:55.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.366+0000 7f3467400640 1 -- 172.21.15.196:0/969449493 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c037070 con 0x7f34681628c0 2024-04-28T13:31:55.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.366+0000 7f3467400640 1 -- 172.21.15.196:0/969449493 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f345c02aba0 con 0x7f34681628c0 2024-04-28T13:31:55.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.366+0000 7f3467400640 1 -- 172.21.15.196:0/969449493 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34600019f0 con 0x7f34681628c0 2024-04-28T13:31:55.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.367+0000 7f346e400640 1 -- 172.21.15.196:0/969449493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34681628c0 msgr2=0x7f34680d19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:55.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.367+0000 7f346e400640 1 --2- 172.21.15.196:0/969449493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34681628c0 0x7f34680d19b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f345c0097a0 tx=0x7f345c02a110 comp rx=0 tx=0).stop 2024-04-28T13:31:55.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.367+0000 7f346e400640 1 -- 172.21.15.196:0/969449493 shutdown_connections 2024-04-28T13:31:55.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.367+0000 7f346e400640 1 --2- 172.21.15.196:0/969449493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34680d1ef0 0x7f34680d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.367+0000 7f346e400640 1 --2- 172.21.15.196:0/969449493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34681628c0 0x7f34680d19b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.367+0000 7f346e400640 1 -- 172.21.15.196:0/969449493 >> 172.21.15.196:0/969449493 conn(0x7f34680d4fc0 msgr2=0x7f34680d53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:55.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.367+0000 7f346e400640 1 -- 172.21.15.196:0/969449493 shutdown_connections 2024-04-28T13:31:55.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.367+0000 7f346e400640 1 -- 172.21.15.196:0/969449493 wait complete. 2024-04-28T13:31:55.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.369+0000 7f346e400640 1 Processor -- start 2024-04-28T13:31:55.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.369+0000 7f346e400640 1 -- start start 2024-04-28T13:31:55.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.370+0000 7f346e400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34680b83d0 0x7f34681553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:55.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.370+0000 7f346e400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34680d1ef0 0x7f34680cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:55.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.370+0000 7f346e400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34681628c0 0x7f34680b9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:55.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.370+0000 7f346e400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3468162ca0 con 0x7f34680d1ef0 2024-04-28T13:31:55.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.370+0000 7f346e400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f34680d0fb0 con 0x7f34681628c0 2024-04-28T13:31:55.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.370+0000 7f346e400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3468162e20 con 0x7f34680b83d0 2024-04-28T13:31:55.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.370+0000 7f3467e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34680d1ef0 0x7f34680cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:55.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.370+0000 7f3467e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34680d1ef0 0x7f34680cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52320/0 (socket says 172.21.15.196:52320) 2024-04-28T13:31:55.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.370+0000 7f3467e00640 1 -- 172.21.15.196:0/4148083935 learned_addr learned my addr 172.21.15.196:0/4148083935 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:55.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.371+0000 7f346d000640 1 --2- 172.21.15.196:0/4148083935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34680b83d0 0x7f34681553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:55.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.371+0000 7f346da00640 1 --2- 172.21.15.196:0/4148083935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34681628c0 0x7f34680b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:55.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.371+0000 7f3467e00640 1 -- 172.21.15.196:0/4148083935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34680b83d0 msgr2=0x7f34681553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:55.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.371+0000 7f3467e00640 1 --2- 172.21.15.196:0/4148083935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34680b83d0 0x7f34681553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.371+0000 7f3467e00640 1 -- 172.21.15.196:0/4148083935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34681628c0 msgr2=0x7f34680b9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:55.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.371+0000 7f3467e00640 1 --2- 172.21.15.196:0/4148083935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34681628c0 0x7f34680b9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.371+0000 7f3467e00640 1 -- 172.21.15.196:0/4148083935 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34680ce260 con 0x7f34680d1ef0 2024-04-28T13:31:55.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.371+0000 7f3467e00640 1 --2- 172.21.15.196:0/4148083935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34680d1ef0 0x7f34680cd7e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f345400ebe0 tx=0x7f345400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:55.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.371+0000 7f3464c00640 1 -- 172.21.15.196:0/4148083935 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454017070 con 0x7f34680d1ef0 2024-04-28T13:31:55.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.371+0000 7f3464c00640 1 -- 172.21.15.196:0/4148083935 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f345400cc30 con 0x7f34680d1ef0 2024-04-28T13:31:55.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.371+0000 7f346e400640 1 -- 172.21.15.196:0/4148083935 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34680ce540 con 0x7f34680d1ef0 2024-04-28T13:31:55.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.371+0000 7f3464c00640 1 -- 172.21.15.196:0/4148083935 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454013470 con 0x7f34680d1ef0 2024-04-28T13:31:55.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.371+0000 7f346e400640 1 -- 172.21.15.196:0/4148083935 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34680ceb00 con 0x7f34680d1ef0 2024-04-28T13:31:55.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.373+0000 7f346e400640 1 -- 172.21.15.196:0/4148083935 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3468050730 con 0x7f34680d1ef0 2024-04-28T13:31:55.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.374+0000 7f3464c00640 1 -- 172.21.15.196:0/4148083935 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f345401a070 con 0x7f34680d1ef0 2024-04-28T13:31:55.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.375+0000 7f3464c00640 1 --2- 172.21.15.196:0/4148083935 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f344806c8f0 0x7f344806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:55.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.375+0000 7f3464c00640 1 -- 172.21.15.196:0/4148083935 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f345408bf10 con 0x7f34680d1ef0 2024-04-28T13:31:55.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.375+0000 7f346d000640 1 --2- 172.21.15.196:0/4148083935 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f344806c8f0 0x7f344806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:55.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.376+0000 7f346d000640 1 --2- 172.21.15.196:0/4148083935 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f344806c8f0 0x7f344806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f345c034000 tx=0x7f345c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:55.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.379+0000 7f3464c00640 1 -- 172.21.15.196:0/4148083935 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f345405a2a0 con 0x7f34680d1ef0 2024-04-28T13:31:55.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.579+0000 7f346e400640 1 -- 172.21.15.196:0/4148083935 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f34680b60a0 con 0x7f34680d1ef0 2024-04-28T13:31:55.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.579+0000 7f3464c00640 1 -- 172.21.15.196:0/4148083935 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f3454059ba0 con 0x7f34680d1ef0 2024-04-28T13:31:55.579 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:31:55.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.582+0000 7f346e400640 1 -- 172.21.15.196:0/4148083935 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f344806c8f0 msgr2=0x7f344806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:55.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.582+0000 7f346e400640 1 --2- 172.21.15.196:0/4148083935 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f344806c8f0 0x7f344806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f345c034000 tx=0x7f345c02e000 comp rx=0 tx=0).stop 2024-04-28T13:31:55.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.582+0000 7f346e400640 1 -- 172.21.15.196:0/4148083935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34680d1ef0 msgr2=0x7f34680cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:55.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.582+0000 7f346e400640 1 --2- 172.21.15.196:0/4148083935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34680d1ef0 0x7f34680cd7e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f345400ebe0 tx=0x7f345400c520 comp rx=0 tx=0).stop 2024-04-28T13:31:55.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.582+0000 7f346e400640 1 -- 172.21.15.196:0/4148083935 shutdown_connections 2024-04-28T13:31:55.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.583+0000 7f346e400640 1 --2- 172.21.15.196:0/4148083935 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f344806c8f0 0x7f344806edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.583+0000 7f346e400640 1 --2- 172.21.15.196:0/4148083935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34681628c0 0x7f34680b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.583+0000 7f346e400640 1 --2- 172.21.15.196:0/4148083935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34680d1ef0 0x7f34680cd7e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.583+0000 7f346e400640 1 --2- 172.21.15.196:0/4148083935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34680b83d0 0x7f34681553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.583+0000 7f346e400640 1 -- 172.21.15.196:0/4148083935 >> 172.21.15.196:0/4148083935 conn(0x7f34680d4fc0 msgr2=0x7f34681603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:55.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.583+0000 7f346e400640 1 -- 172.21.15.196:0/4148083935 shutdown_connections 2024-04-28T13:31:55.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.583+0000 7f346e400640 1 -- 172.21.15.196:0/4148083935 wait complete. 2024-04-28T13:31:55.601 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:55.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.807+0000 7f80bb200640 1 Processor -- start 2024-04-28T13:31:55.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.807+0000 7f80bb200640 1 -- start start 2024-04-28T13:31:55.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.809+0000 7f80bb200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f80bc0d1760 0x7f80bc0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:55.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.809+0000 7f80bb200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f80bc09ec80 0x7f80bc06df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:55.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.809+0000 7f80bb200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80bc15e380 con 0x7f80bc0b5ee0 2024-04-28T13:31:55.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.809+0000 7f80bb200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f80bc0b4b20 con 0x7f80bc0d1760 2024-04-28T13:31:55.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.809+0000 7f80bb200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f80bc0b3f30 con 0x7f80bc09ec80 2024-04-28T13:31:55.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.809+0000 7f80b9400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f80bc0d1760 0x7f80bc0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:55.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.809+0000 7f80b9400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f80bc0d1760 0x7f80bc0b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52334/0 (socket says 172.21.15.196:52334) 2024-04-28T13:31:55.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.809+0000 7f80b9400640 1 -- 172.21.15.196:0/2555252716 learned_addr learned my addr 172.21.15.196:0/2555252716 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:55.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.810+0000 7f80ba800640 1 --2- 172.21.15.196:0/2555252716 >> v2:172.21.15.203:3300/0 conn(0x7f80bc09ec80 0x7f80bc06df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:55.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.810+0000 7f80b9400640 1 -- 172.21.15.196:0/2555252716 >> v2:172.21.15.203:3300/0 conn(0x7f80bc09ec80 msgr2=0x7f80bc06df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:55.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.810+0000 7f80b9400640 1 --2- 172.21.15.196:0/2555252716 >> v2:172.21.15.203:3300/0 conn(0x7f80bc09ec80 0x7f80bc06df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.810+0000 7f80b9400640 1 -- 172.21.15.196:0/2555252716 >> v1:172.21.15.196:6789/0 conn(0x7f80bc0b5ee0 legacy=0x7f80bc0d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:55.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.810+0000 7f80b9400640 1 -- 172.21.15.196:0/2555252716 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc0b41f0 con 0x7f80bc0d1760 2024-04-28T13:31:55.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.811+0000 7f80b9400640 1 --2- 172.21.15.196:0/2555252716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc0d1760 0x7f80bc0b4f40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f80bc074240 tx=0x7f80a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35660ed429177f5a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:55.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.811+0000 7f80b8a00640 1 -- 172.21.15.196:0/2555252716 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8037070 con 0x7f80bc0d1760 2024-04-28T13:31:55.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.811+0000 7f80b8a00640 1 -- 172.21.15.196:0/2555252716 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a8032440 con 0x7f80bc0d1760 2024-04-28T13:31:55.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.811+0000 7f80b8a00640 1 -- 172.21.15.196:0/2555252716 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8032810 con 0x7f80bc0d1760 2024-04-28T13:31:55.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.812+0000 7f80bb200640 1 -- 172.21.15.196:0/2555252716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc0d1760 msgr2=0x7f80bc0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:55.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.812+0000 7f80bb200640 1 --2- 172.21.15.196:0/2555252716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc0d1760 0x7f80bc0b4f40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f80bc074240 tx=0x7f80a802a270 comp rx=0 tx=0).stop 2024-04-28T13:31:55.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.812+0000 7f80bb200640 1 -- 172.21.15.196:0/2555252716 shutdown_connections 2024-04-28T13:31:55.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.812+0000 7f80bb200640 1 --2- 172.21.15.196:0/2555252716 >> v2:172.21.15.203:3300/0 conn(0x7f80bc09ec80 0x7f80bc06df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.812+0000 7f80bb200640 1 --2- 172.21.15.196:0/2555252716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc0d1760 0x7f80bc0b4f40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.812+0000 7f80bb200640 1 -- 172.21.15.196:0/2555252716 >> 172.21.15.196:0/2555252716 conn(0x7f80bc06c6c0 msgr2=0x7f80bc06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:55.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.812+0000 7f80bb200640 1 -- 172.21.15.196:0/2555252716 shutdown_connections 2024-04-28T13:31:55.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.812+0000 7f80bb200640 1 -- 172.21.15.196:0/2555252716 wait complete. 2024-04-28T13:31:55.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.813+0000 7f80bb200640 1 Processor -- start 2024-04-28T13:31:55.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.814+0000 7f80bb200640 1 -- start start 2024-04-28T13:31:55.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.814+0000 7f80bb200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc09ec80 0x7f80bc142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:55.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.815+0000 7f80bb200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80bc0b5ee0 0x7f80bc1536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:55.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.815+0000 7f80bb200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0d1760 0x7f80bc0b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:55.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.815+0000 7f80bb200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc0b3c20 con 0x7f80bc09ec80 2024-04-28T13:31:55.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.815+0000 7f80bb200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc0b47b0 con 0x7f80bc0b5ee0 2024-04-28T13:31:55.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.815+0000 7f80bb200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f80bc0add70 con 0x7f80bc0d1760 2024-04-28T13:31:55.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.815+0000 7f80b9e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc09ec80 0x7f80bc142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:55.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.815+0000 7f80b9e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc09ec80 0x7f80bc142ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52344/0 (socket says 172.21.15.196:52344) 2024-04-28T13:31:55.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.815+0000 7f80b9e00640 1 -- 172.21.15.196:0/3164696146 learned_addr learned my addr 172.21.15.196:0/3164696146 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:55.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.815+0000 7f80b9400640 1 --2- 172.21.15.196:0/3164696146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80bc0b5ee0 0x7f80bc1536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:55.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.815+0000 7f80b9e00640 1 -- 172.21.15.196:0/3164696146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0d1760 msgr2=0x7f80bc0b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:55.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.815+0000 7f80b9e00640 1 --2- 172.21.15.196:0/3164696146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0d1760 0x7f80bc0b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.815+0000 7f80b9e00640 1 -- 172.21.15.196:0/3164696146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80bc0b5ee0 msgr2=0x7f80bc1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:55.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.815+0000 7f80b9e00640 1 --2- 172.21.15.196:0/3164696146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80bc0b5ee0 0x7f80bc1536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:55.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.815+0000 7f80b9e00640 1 -- 172.21.15.196:0/3164696146 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc0b1210 con 0x7f80bc09ec80 2024-04-28T13:31:55.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.816+0000 7f80b9e00640 1 --2- 172.21.15.196:0/3164696146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc09ec80 0x7f80bc142ff0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c430 tx=0x7f80ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:55.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.816+0000 7f80b2000640 1 -- 172.21.15.196:0/3164696146 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac0052d0 con 0x7f80bc09ec80 2024-04-28T13:31:55.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.816+0000 7f80b2000640 1 -- 172.21.15.196:0/3164696146 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80ac005490 con 0x7f80bc09ec80 2024-04-28T13:31:55.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.816+0000 7f80b2000640 1 -- 172.21.15.196:0/3164696146 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac007260 con 0x7f80bc09ec80 2024-04-28T13:31:55.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.816+0000 7f80bb200640 1 -- 172.21.15.196:0/3164696146 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80bc153c30 con 0x7f80bc09ec80 2024-04-28T13:31:55.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.816+0000 7f80bb200640 1 -- 172.21.15.196:0/3164696146 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80bc156950 con 0x7f80bc09ec80 2024-04-28T13:31:55.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.817+0000 7f80bb200640 1 -- 172.21.15.196:0/3164696146 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80bc06a290 con 0x7f80bc09ec80 2024-04-28T13:31:55.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.819+0000 7f80b2000640 1 -- 172.21.15.196:0/3164696146 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80ac007400 con 0x7f80bc09ec80 2024-04-28T13:31:55.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.819+0000 7f80b2000640 1 --2- 172.21.15.196:0/3164696146 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f809c06c8f0 0x7f809c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:55.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.820+0000 7f80b2000640 1 -- 172.21.15.196:0/3164696146 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f80ac08b140 con 0x7f80bc09ec80 2024-04-28T13:31:55.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.820+0000 7f80b9400640 1 --2- 172.21.15.196:0/3164696146 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f809c06c8f0 0x7f809c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:55.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.821+0000 7f80b9400640 1 --2- 172.21.15.196:0/3164696146 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f809c06c8f0 0x7f809c06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f80a8034000 tx=0x7f80a802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:55.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:55.824+0000 7f80b2000640 1 -- 172.21.15.196:0/3164696146 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f80ac0594e0 con 0x7f80bc09ec80 2024-04-28T13:31:56.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.032+0000 7f80bb200640 1 -- 172.21.15.196:0/3164696146 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80bc06ee60 con 0x7f80bc09ec80 2024-04-28T13:31:56.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.035+0000 7f80b2000640 1 -- 172.21.15.196:0/3164696146 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f80ac058de0 con 0x7f80bc09ec80 2024-04-28T13:31:56.035 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:56.035 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":20,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:22.624564+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:56.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.038+0000 7f80bb200640 1 -- 172.21.15.196:0/3164696146 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f809c06c8f0 msgr2=0x7f809c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:56.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.038+0000 7f80bb200640 1 --2- 172.21.15.196:0/3164696146 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f809c06c8f0 0x7f809c06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f80a8034000 tx=0x7f80a802e000 comp rx=0 tx=0).stop 2024-04-28T13:31:56.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.038+0000 7f80bb200640 1 -- 172.21.15.196:0/3164696146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc09ec80 msgr2=0x7f80bc142ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:56.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.038+0000 7f80bb200640 1 --2- 172.21.15.196:0/3164696146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc09ec80 0x7f80bc142ff0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c430 tx=0x7f80ac00c8f0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.038+0000 7f80bb200640 1 -- 172.21.15.196:0/3164696146 shutdown_connections 2024-04-28T13:31:56.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.038+0000 7f80bb200640 1 --2- 172.21.15.196:0/3164696146 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f809c06c8f0 0x7f809c06edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.038+0000 7f80bb200640 1 --2- 172.21.15.196:0/3164696146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0d1760 0x7f80bc0b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.038+0000 7f80bb200640 1 --2- 172.21.15.196:0/3164696146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80bc0b5ee0 0x7f80bc1536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.038+0000 7f80bb200640 1 --2- 172.21.15.196:0/3164696146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc09ec80 0x7f80bc142ff0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.038+0000 7f80bb200640 1 -- 172.21.15.196:0/3164696146 >> 172.21.15.196:0/3164696146 conn(0x7f80bc06c6c0 msgr2=0x7f80bc09ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:56.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.038+0000 7f80bb200640 1 -- 172.21.15.196:0/3164696146 shutdown_connections 2024-04-28T13:31:56.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.038+0000 7f80bb200640 1 -- 172.21.15.196:0/3164696146 wait complete. 2024-04-28T13:31:56.057 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:31:19.280975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:31:20.037066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:31:56.058 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:31:56.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.245+0000 7fb92fa00640 1 Processor -- start 2024-04-28T13:31:56.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.245+0000 7fb92fa00640 1 -- start start 2024-04-28T13:31:56.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.246+0000 7fb92fa00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb928159cf0 0x7fb928070db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.246+0000 7fb92fa00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9280712f0 0x7fb928073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.246+0000 7fb92fa00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb92806f4d0 con 0x7fb928073690 2024-04-28T13:31:56.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.246+0000 7fb92fa00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb92806f7b0 con 0x7fb928159cf0 2024-04-28T13:31:56.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.246+0000 7fb92fa00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb92806fa90 con 0x7fb9280712f0 2024-04-28T13:31:56.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.247+0000 7fb92da00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fb928073690 0x7fb92815fae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:34292/0 (socket says 172.21.15.196:34292) 2024-04-28T13:31:56.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.247+0000 7fb92da00640 1 -- 172.21.15.196:0/4100548439 learned_addr learned my addr 172.21.15.196:0/4100548439 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:56.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.247+0000 7fb92d000640 1 --2- 172.21.15.196:0/4100548439 >> v2:172.21.15.203:3300/0 conn(0x7fb928159cf0 0x7fb928070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.247+0000 7fb927e00640 1 --2- 172.21.15.196:0/4100548439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9280712f0 0x7fb928073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.247+0000 7fb927400640 1 -- 172.21.15.196:0/4100548439 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003767207 0 0) 0x7fb92806f4d0 con 0x7fb928073690 2024-04-28T13:31:56.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.248+0000 7fb927400640 1 -- 172.21.15.196:0/4100548439 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb90c003520 con 0x7fb928073690 2024-04-28T13:31:56.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.248+0000 7fb927400640 1 -- 172.21.15.196:0/4100548439 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 566113724 0 0) 0x7fb90c003520 con 0x7fb928073690 2024-04-28T13:31:56.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.248+0000 7fb927400640 1 -- 172.21.15.196:0/4100548439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9280712f0 msgr2=0x7fb928073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:56.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.248+0000 7fb927400640 1 --2- 172.21.15.196:0/4100548439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9280712f0 0x7fb928073150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.248+0000 7fb927400640 1 -- 172.21.15.196:0/4100548439 >> v2:172.21.15.203:3300/0 conn(0x7fb928159cf0 msgr2=0x7fb928070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:56.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.248+0000 7fb927400640 1 --2- 172.21.15.196:0/4100548439 >> v2:172.21.15.203:3300/0 conn(0x7fb928159cf0 0x7fb928070db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.248+0000 7fb927400640 1 -- 172.21.15.196:0/4100548439 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92806fd70 con 0x7fb928073690 2024-04-28T13:31:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.248+0000 7fb927400640 1 -- 172.21.15.196:0/4100548439 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fb9200019f0 con 0x7fb928073690 2024-04-28T13:31:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.248+0000 7fb927400640 1 -- 172.21.15.196:0/4100548439 >> v1:172.21.15.196:6789/0 conn(0x7fb928073690 legacy=0x7fb92815fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.248+0000 7fb927400640 1 --2- 172.21.15.196:0/4100548439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9280712f0 0x7fb90c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.248+0000 7fb927400640 1 --2- 172.21.15.196:0/4100548439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c003f80 0x7fb90c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.248+0000 7fb927400640 1 --2- 172.21.15.196:0/4100548439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb90c004d40 0x7fb90c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.248+0000 7fb927400640 1 -- 172.21.15.196:0/4100548439 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb92806f7b0 con 0x7fb90c004d40 2024-04-28T13:31:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.248+0000 7fb927400640 1 -- 172.21.15.196:0/4100548439 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb92806fa90 con 0x7fb90c003f80 2024-04-28T13:31:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.248+0000 7fb927400640 1 -- 172.21.15.196:0/4100548439 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb90c00ba70 con 0x7fb9280712f0 2024-04-28T13:31:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.249+0000 7fb92da00640 1 --2- 172.21.15.196:0/4100548439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb90c004d40 0x7fb90c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.249+0000 7fb927e00640 1 --2- 172.21.15.196:0/4100548439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c003f80 0x7fb90c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.249+0000 7fb92d000640 1 --2- 172.21.15.196:0/4100548439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9280712f0 0x7fb90c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.249+0000 7fb92da00640 1 -- 172.21.15.196:0/4100548439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9280712f0 msgr2=0x7fb90c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.249+0000 7fb92da00640 1 --2- 172.21.15.196:0/4100548439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9280712f0 0x7fb90c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.249+0000 7fb92da00640 1 -- 172.21.15.196:0/4100548439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c003f80 msgr2=0x7fb90c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:56.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.249+0000 7fb92da00640 1 --2- 172.21.15.196:0/4100548439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c003f80 0x7fb90c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.249+0000 7fb92da00640 1 -- 172.21.15.196:0/4100548439 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90c003520 con 0x7fb90c004d40 2024-04-28T13:31:56.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.249+0000 7fb927e00640 1 --2- 172.21.15.196:0/4100548439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c003f80 0x7fb90c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:31:56.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.250+0000 7fb92da00640 1 --2- 172.21.15.196:0/4100548439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb90c004d40 0x7fb90c0051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb92000a7a0 tx=0x7fb90c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbef4cfdc7960ffb server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:56.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.251+0000 7fb927400640 1 -- 172.21.15.196:0/4100548439 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92000a3f0 con 0x7fb90c004d40 2024-04-28T13:31:56.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.251+0000 7fb927400640 1 -- 172.21.15.196:0/4100548439 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb92002f070 con 0x7fb90c004d40 2024-04-28T13:31:56.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.251+0000 7fb927400640 1 -- 172.21.15.196:0/4100548439 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92002bc60 con 0x7fb90c004d40 2024-04-28T13:31:56.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.251+0000 7fb92fa00640 1 -- 172.21.15.196:0/4100548439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb90c004d40 msgr2=0x7fb90c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:56.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.251+0000 7fb92fa00640 1 --2- 172.21.15.196:0/4100548439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb90c004d40 0x7fb90c0051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb92000a7a0 tx=0x7fb90c005830 comp rx=0 tx=0).stop 2024-04-28T13:31:56.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.251+0000 7fb92da00640 1 -- 172.21.15.196:0/4100548439 reap_dead start 2024-04-28T13:31:56.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.251+0000 7fb92fa00640 1 -- 172.21.15.196:0/4100548439 shutdown_connections 2024-04-28T13:31:56.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.251+0000 7fb92fa00640 1 -- 172.21.15.196:0/4100548439 >> 172.21.15.196:0/4100548439 conn(0x7fb9280d1e90 msgr2=0x7fb9280d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:56.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.251+0000 7fb92fa00640 1 -- 172.21.15.196:0/4100548439 shutdown_connections 2024-04-28T13:31:56.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.251+0000 7fb92fa00640 1 -- 172.21.15.196:0/4100548439 wait complete. 2024-04-28T13:31:56.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.253+0000 7fb92fa00640 1 Processor -- start 2024-04-28T13:31:56.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.253+0000 7fb92fa00640 1 -- start start 2024-04-28T13:31:56.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.254+0000 7fb92fa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c004d40 0x7fb928160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.254+0000 7fb92fa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb92815c0e0 0x7fb92815c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.254+0000 7fb92fa00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb928144fc0 0x7fb928145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.254+0000 7fb92fa00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9280b6780 con 0x7fb92815c0e0 2024-04-28T13:31:56.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.254+0000 7fb92fa00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb92815d6b0 con 0x7fb90c004d40 2024-04-28T13:31:56.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.254+0000 7fb92fa00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb9280b6900 con 0x7fb928144fc0 2024-04-28T13:31:56.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.254+0000 7fb927e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb92815c0e0 0x7fb92815c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.255+0000 7fb927e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb92815c0e0 0x7fb92815c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52352/0 (socket says 172.21.15.196:52352) 2024-04-28T13:31:56.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.255+0000 7fb927e00640 1 -- 172.21.15.196:0/2346889790 learned_addr learned my addr 172.21.15.196:0/2346889790 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:56.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.255+0000 7fb92d000640 1 --2- 172.21.15.196:0/2346889790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c004d40 0x7fb928160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.255+0000 7fb92da00640 1 --2- 172.21.15.196:0/2346889790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb928144fc0 0x7fb928145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.255+0000 7fb927e00640 1 -- 172.21.15.196:0/2346889790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb928144fc0 msgr2=0x7fb928145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:56.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.255+0000 7fb927e00640 1 --2- 172.21.15.196:0/2346889790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb928144fc0 0x7fb928145440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.255+0000 7fb927e00640 1 -- 172.21.15.196:0/2346889790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c004d40 msgr2=0x7fb928160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:56.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.255+0000 7fb927e00640 1 --2- 172.21.15.196:0/2346889790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c004d40 0x7fb928160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.255+0000 7fb927e00640 1 -- 172.21.15.196:0/2346889790 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9281f8e20 con 0x7fb92815c0e0 2024-04-28T13:31:56.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.255+0000 7fb92da00640 1 --2- 172.21.15.196:0/2346889790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb928144fc0 0x7fb928145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:56.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.255+0000 7fb927e00640 1 --2- 172.21.15.196:0/2346889790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb92815c0e0 0x7fb92815c540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb910011630 tx=0x7fb910011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:56.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.256+0000 7fb924c00640 1 -- 172.21.15.196:0/2346889790 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91000f920 con 0x7fb92815c0e0 2024-04-28T13:31:56.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.256+0000 7fb924c00640 1 -- 172.21.15.196:0/2346889790 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91000fac0 con 0x7fb92815c0e0 2024-04-28T13:31:56.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.256+0000 7fb92fa00640 1 -- 172.21.15.196:0/2346889790 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9281f9100 con 0x7fb92815c0e0 2024-04-28T13:31:56.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.256+0000 7fb924c00640 1 -- 172.21.15.196:0/2346889790 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91000fe90 con 0x7fb92815c0e0 2024-04-28T13:31:56.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.256+0000 7fb92fa00640 1 -- 172.21.15.196:0/2346889790 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9281f96c0 con 0x7fb92815c0e0 2024-04-28T13:31:56.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.257+0000 7fb92fa00640 1 -- 172.21.15.196:0/2346889790 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9280693d0 con 0x7fb92815c0e0 2024-04-28T13:31:56.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.258+0000 7fb924c00640 1 -- 172.21.15.196:0/2346889790 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb910015460 con 0x7fb92815c0e0 2024-04-28T13:31:56.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.259+0000 7fb924c00640 1 --2- 172.21.15.196:0/2346889790 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb90406ca90 0x7fb90406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.259+0000 7fb924c00640 1 -- 172.21.15.196:0/2346889790 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb91008c4b0 con 0x7fb92815c0e0 2024-04-28T13:31:56.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.259+0000 7fb92d000640 1 --2- 172.21.15.196:0/2346889790 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb90406ca90 0x7fb90406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.261+0000 7fb92d000640 1 --2- 172.21.15.196:0/2346889790 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb90406ca90 0x7fb90406ef50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb918009b80 tx=0x7fb918008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:56.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.263+0000 7fb924c00640 1 -- 172.21.15.196:0/2346889790 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb910019030 con 0x7fb92815c0e0 2024-04-28T13:31:56.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.449+0000 7fb92fa00640 1 -- 172.21.15.196:0/2346889790 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb9280d1070 con 0x7fb92815c0e0 2024-04-28T13:31:56.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.450+0000 7fb924c00640 1 -- 172.21.15.196:0/2346889790 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fb91005a8e0 con 0x7fb92815c0e0 2024-04-28T13:31:56.450 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:31:56.453 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.453+0000 7fb92fa00640 1 -- 172.21.15.196:0/2346889790 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb90406ca90 msgr2=0x7fb90406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:56.453 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.453+0000 7fb92fa00640 1 --2- 172.21.15.196:0/2346889790 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb90406ca90 0x7fb90406ef50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb918009b80 tx=0x7fb918008040 comp rx=0 tx=0).stop 2024-04-28T13:31:56.453 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.453+0000 7fb92fa00640 1 -- 172.21.15.196:0/2346889790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb92815c0e0 msgr2=0x7fb92815c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:56.454 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.453+0000 7fb92fa00640 1 --2- 172.21.15.196:0/2346889790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb92815c0e0 0x7fb92815c540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb910011630 tx=0x7fb910011af0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.454 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.453+0000 7fb92fa00640 1 -- 172.21.15.196:0/2346889790 shutdown_connections 2024-04-28T13:31:56.454 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.453+0000 7fb92fa00640 1 --2- 172.21.15.196:0/2346889790 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb90406ca90 0x7fb90406ef50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.454 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.453+0000 7fb92fa00640 1 --2- 172.21.15.196:0/2346889790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb928144fc0 0x7fb928145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.454 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.453+0000 7fb92fa00640 1 --2- 172.21.15.196:0/2346889790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb92815c0e0 0x7fb92815c540 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.454 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.453+0000 7fb92fa00640 1 --2- 172.21.15.196:0/2346889790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb90c004d40 0x7fb928160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.454 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.453+0000 7fb92fa00640 1 -- 172.21.15.196:0/2346889790 >> 172.21.15.196:0/2346889790 conn(0x7fb9280d1e90 msgr2=0x7fb928156ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:56.454 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.453+0000 7fb92fa00640 1 -- 172.21.15.196:0/2346889790 shutdown_connections 2024-04-28T13:31:56.454 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.453+0000 7fb92fa00640 1 -- 172.21.15.196:0/2346889790 wait complete. 2024-04-28T13:31:56.472 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:31:56.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.675+0000 7fa4d4000640 1 Processor -- start 2024-04-28T13:31:56.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.675+0000 7fa4d4000640 1 -- start start 2024-04-28T13:31:56.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.676+0000 7fa4d4000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa4cc0b4eb0 0x7fa4cc0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.676+0000 7fa4d4000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4cc15dca0 con 0x7fa4cc0ceff0 2024-04-28T13:31:56.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.676+0000 7fa4d4000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4cc15df80 con 0x7fa4cc143840 2024-04-28T13:31:56.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.676+0000 7fa4d4000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa4cc143c20 con 0x7fa4cc0b4eb0 2024-04-28T13:31:56.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.677+0000 7fa4d2000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa4cc0ceff0 0x7fa4cc0cf480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:34304/0 (socket says 172.21.15.196:34304) 2024-04-28T13:31:56.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.677+0000 7fa4d2000640 1 -- 172.21.15.196:0/3764197766 learned_addr learned my addr 172.21.15.196:0/3764197766 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:56.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.677+0000 7fa4d0c00640 1 --2- 172.21.15.196:0/3764197766 >> v2:172.21.15.203:3300/0 conn(0x7fa4cc0b4eb0 0x7fa4cc0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.677+0000 7fa4cbe00640 1 -- 172.21.15.196:0/3764197766 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1196046475 0 0) 0x7fa4cc15dca0 con 0x7fa4cc0ceff0 2024-04-28T13:31:56.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.677+0000 7fa4cbe00640 1 -- 172.21.15.196:0/3764197766 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4b4003520 con 0x7fa4cc0ceff0 2024-04-28T13:31:56.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.678+0000 7fa4cbe00640 1 -- 172.21.15.196:0/3764197766 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 729985244 0 0) 0x7fa4b4003520 con 0x7fa4cc0ceff0 2024-04-28T13:31:56.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.678+0000 7fa4cbe00640 1 -- 172.21.15.196:0/3764197766 >> v2:172.21.15.203:3300/0 conn(0x7fa4cc0b4eb0 msgr2=0x7fa4cc0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:56.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.678+0000 7fa4cbe00640 1 --2- 172.21.15.196:0/3764197766 >> v2:172.21.15.203:3300/0 conn(0x7fa4cc0b4eb0 0x7fa4cc0b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.678+0000 7fa4cbe00640 1 -- 172.21.15.196:0/3764197766 >> v1:172.21.15.203:6789/0 conn(0x7fa4cc143840 legacy=0x7fa4cc0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:56.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.678+0000 7fa4cbe00640 1 -- 172.21.15.196:0/3764197766 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc15e260 con 0x7fa4cc0ceff0 2024-04-28T13:31:56.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.678+0000 7fa4cbe00640 1 -- 172.21.15.196:0/3764197766 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fa4c40019f0 con 0x7fa4cc0ceff0 2024-04-28T13:31:56.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.678+0000 7fa4cbe00640 1 -- 172.21.15.196:0/3764197766 >> v1:172.21.15.196:6789/0 conn(0x7fa4cc0ceff0 legacy=0x7fa4cc0cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:56.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.678+0000 7fa4cbe00640 1 --2- 172.21.15.196:0/3764197766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b40039b0 0x7fa4b4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.678+0000 7fa4cbe00640 1 --2- 172.21.15.196:0/3764197766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4b4004800 0x7fa4b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.678+0000 7fa4cbe00640 1 --2- 172.21.15.196:0/3764197766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b40051c0 0x7fa4b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.678+0000 7fa4cbe00640 1 -- 172.21.15.196:0/3764197766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc143c20 con 0x7fa4b4004800 2024-04-28T13:31:56.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.678+0000 7fa4cbe00640 1 -- 172.21.15.196:0/3764197766 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b400c090 con 0x7fa4b40051c0 2024-04-28T13:31:56.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.678+0000 7fa4cbe00640 1 -- 172.21.15.196:0/3764197766 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b400c290 con 0x7fa4b40039b0 2024-04-28T13:31:56.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.678+0000 7fa4d0c00640 1 --2- 172.21.15.196:0/3764197766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4b4004800 0x7fa4b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.679+0000 7fa4d2000640 1 --2- 172.21.15.196:0/3764197766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b40051c0 0x7fa4b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.679+0000 7fa4d1600640 1 --2- 172.21.15.196:0/3764197766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b40039b0 0x7fa4b4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.679+0000 7fa4d0c00640 1 -- 172.21.15.196:0/3764197766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b40039b0 msgr2=0x7fa4b4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:56.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.679+0000 7fa4d0c00640 1 --2- 172.21.15.196:0/3764197766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b40039b0 0x7fa4b4003df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.679+0000 7fa4d0c00640 1 -- 172.21.15.196:0/3764197766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b40051c0 msgr2=0x7fa4b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:56.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.679+0000 7fa4d0c00640 1 --2- 172.21.15.196:0/3764197766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b40051c0 0x7fa4b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.679+0000 7fa4d0c00640 1 -- 172.21.15.196:0/3764197766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b4003520 con 0x7fa4b4004800 2024-04-28T13:31:56.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.679+0000 7fa4d2000640 1 --2- 172.21.15.196:0/3764197766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b40051c0 0x7fa4b400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:56.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.679+0000 7fa4d2000640 1 -- 172.21.15.196:0/3764197766 reap_dead start 2024-04-28T13:31:56.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.680+0000 7fa4d0c00640 1 --2- 172.21.15.196:0/3764197766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4b4004800 0x7fa4b4004c80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009ac0 tx=0x7fa4b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587130bb7caab25 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:56.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.680+0000 7fa4cbe00640 1 -- 172.21.15.196:0/3764197766 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8036070 con 0x7fa4b4004800 2024-04-28T13:31:56.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.680+0000 7fa4cbe00640 1 -- 172.21.15.196:0/3764197766 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b8031400 con 0x7fa4b4004800 2024-04-28T13:31:56.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.680+0000 7fa4cbe00640 1 -- 172.21.15.196:0/3764197766 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8009690 con 0x7fa4b4004800 2024-04-28T13:31:56.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.680+0000 7fa4d4000640 1 -- 172.21.15.196:0/3764197766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4b4004800 msgr2=0x7fa4b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:56.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.680+0000 7fa4d4000640 1 --2- 172.21.15.196:0/3764197766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4b4004800 0x7fa4b4004c80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009ac0 tx=0x7fa4b4005830 comp rx=0 tx=0).stop 2024-04-28T13:31:56.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.681+0000 7fa4d4000640 1 -- 172.21.15.196:0/3764197766 shutdown_connections 2024-04-28T13:31:56.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.681+0000 7fa4d4000640 1 --2- 172.21.15.196:0/3764197766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4b4004800 0x7fa4b4004c80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.681+0000 7fa4d4000640 1 -- 172.21.15.196:0/3764197766 >> 172.21.15.196:0/3764197766 conn(0x7fa4cc0cd8a0 msgr2=0x7fa4cc0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:56.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.681+0000 7fa4d4000640 1 -- 172.21.15.196:0/3764197766 shutdown_connections 2024-04-28T13:31:56.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.681+0000 7fa4d4000640 1 -- 172.21.15.196:0/3764197766 wait complete. 2024-04-28T13:31:56.683 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.682+0000 7fa4d4000640 1 Processor -- start 2024-04-28T13:31:56.683 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.683+0000 7fa4d4000640 1 -- start start 2024-04-28T13:31:56.684 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.683+0000 7fa4d4000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b4004800 0x7fa4cc140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.684 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.683+0000 7fa4d4000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4cc1f4f10 0x7fa4cc1f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.684 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.684+0000 7fa4d4000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4cc13e150 0x7fa4cc13e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.684 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.684+0000 7fa4d4000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc15da40 con 0x7fa4cc13e150 2024-04-28T13:31:56.684 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.684+0000 7fa4d4000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc15d8c0 con 0x7fa4cc1f4f10 2024-04-28T13:31:56.684 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.684+0000 7fa4d4000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc0b5330 con 0x7fa4b4004800 2024-04-28T13:31:56.684 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.684+0000 7fa4d2000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4cc13e150 0x7fa4cc13e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.684 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.684+0000 7fa4d2000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4cc13e150 0x7fa4cc13e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52362/0 (socket says 172.21.15.196:52362) 2024-04-28T13:31:56.684 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.684+0000 7fa4d2000640 1 -- 172.21.15.196:0/1636023915 learned_addr learned my addr 172.21.15.196:0/1636023915 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:56.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.684+0000 7fa4d1600640 1 --2- 172.21.15.196:0/1636023915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b4004800 0x7fa4cc140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.684+0000 7fa4d2000640 1 -- 172.21.15.196:0/1636023915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b4004800 msgr2=0x7fa4cc140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:56.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.684+0000 7fa4d2000640 1 --2- 172.21.15.196:0/1636023915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b4004800 0x7fa4cc140820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.684+0000 7fa4d2000640 1 -- 172.21.15.196:0/1636023915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4cc1f4f10 msgr2=0x7fa4cc1f5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:56.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.684+0000 7fa4d2000640 1 --2- 172.21.15.196:0/1636023915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4cc1f4f10 0x7fa4cc1f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.684+0000 7fa4d2000640 1 -- 172.21.15.196:0/1636023915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc1f5c90 con 0x7fa4cc13e150 2024-04-28T13:31:56.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.684+0000 7fa4d1600640 1 --2- 172.21.15.196:0/1636023915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b4004800 0x7fa4cc140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:56.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.685+0000 7fa4d2000640 1 --2- 172.21.15.196:0/1636023915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4cc13e150 0x7fa4cc13e5d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc15b440 tx=0x7fa4c400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:56.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.685+0000 7fa4c9600640 1 -- 172.21.15.196:0/1636023915 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4004d60 con 0x7fa4cc13e150 2024-04-28T13:31:56.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.685+0000 7fa4c9600640 1 -- 172.21.15.196:0/1636023915 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c400bb00 con 0x7fa4cc13e150 2024-04-28T13:31:56.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.685+0000 7fa4c9600640 1 -- 172.21.15.196:0/1636023915 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4007110 con 0x7fa4cc13e150 2024-04-28T13:31:56.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.685+0000 7fa4d4000640 1 -- 172.21.15.196:0/1636023915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4cc1f8710 con 0x7fa4cc13e150 2024-04-28T13:31:56.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.685+0000 7fa4d4000640 1 -- 172.21.15.196:0/1636023915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4cc1f58d0 con 0x7fa4cc13e150 2024-04-28T13:31:56.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.686+0000 7fa4d4000640 1 -- 172.21.15.196:0/1636023915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4cc1f8500 con 0x7fa4cc13e150 2024-04-28T13:31:56.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.687+0000 7fa4c9600640 1 -- 172.21.15.196:0/1636023915 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4c40072b0 con 0x7fa4cc13e150 2024-04-28T13:31:56.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.688+0000 7fa4c9600640 1 --2- 172.21.15.196:0/1636023915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa4ac06c8f0 0x7fa4ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:56.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.688+0000 7fa4c9600640 1 -- 172.21.15.196:0/1636023915 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa4c408baf0 con 0x7fa4cc13e150 2024-04-28T13:31:56.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.688+0000 7fa4d1600640 1 --2- 172.21.15.196:0/1636023915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa4ac06c8f0 0x7fa4ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:56.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.690+0000 7fa4d1600640 1 --2- 172.21.15.196:0/1636023915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa4ac06c8f0 0x7fa4ac06edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0001630 tx=0x7fa4c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:56.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.692+0000 7fa4c9600640 1 -- 172.21.15.196:0/1636023915 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa4c4059f10 con 0x7fa4cc13e150 2024-04-28T13:31:56.893 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.893+0000 7fa4d4000640 1 -- 172.21.15.196:0/1636023915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa4cc0ce420 con 0x7fa4cc13e150 2024-04-28T13:31:56.893 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.894+0000 7fa4c9600640 1 -- 172.21.15.196:0/1636023915 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa4c4059810 con 0x7fa4cc13e150 2024-04-28T13:31:56.894 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:31:56.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.896+0000 7fa4d4000640 1 -- 172.21.15.196:0/1636023915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa4ac06c8f0 msgr2=0x7fa4ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:56.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.896+0000 7fa4d4000640 1 --2- 172.21.15.196:0/1636023915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa4ac06c8f0 0x7fa4ac06edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0001630 tx=0x7fa4c0009040 comp rx=0 tx=0).stop 2024-04-28T13:31:56.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.896+0000 7fa4d4000640 1 -- 172.21.15.196:0/1636023915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4cc13e150 msgr2=0x7fa4cc13e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:56.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.896+0000 7fa4d4000640 1 --2- 172.21.15.196:0/1636023915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4cc13e150 0x7fa4cc13e5d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc15b440 tx=0x7fa4c400b4a0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.896+0000 7fa4d4000640 1 -- 172.21.15.196:0/1636023915 shutdown_connections 2024-04-28T13:31:56.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.896+0000 7fa4d4000640 1 --2- 172.21.15.196:0/1636023915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa4ac06c8f0 0x7fa4ac06edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.896+0000 7fa4d4000640 1 --2- 172.21.15.196:0/1636023915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4cc13e150 0x7fa4cc13e5d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.896+0000 7fa4d4000640 1 --2- 172.21.15.196:0/1636023915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4cc1f4f10 0x7fa4cc1f5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.896+0000 7fa4d4000640 1 --2- 172.21.15.196:0/1636023915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b4004800 0x7fa4cc140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:56.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.896+0000 7fa4d4000640 1 -- 172.21.15.196:0/1636023915 >> 172.21.15.196:0/1636023915 conn(0x7fa4cc0cd8a0 msgr2=0x7fa4cc0d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:56.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.897+0000 7fa4d4000640 1 -- 172.21.15.196:0/1636023915 shutdown_connections 2024-04-28T13:31:56.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:56.897+0000 7fa4d4000640 1 -- 172.21.15.196:0/1636023915 wait complete. 2024-04-28T13:31:56.915 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:31:57.115 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.115+0000 7fb8e5400640 1 Processor -- start 2024-04-28T13:31:57.115 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.115+0000 7fb8e5400640 1 -- start start 2024-04-28T13:31:57.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.117+0000 7fb8e5400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8e00b4eb0 0x7fb8e00b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.117+0000 7fb8e5400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8e015dca0 con 0x7fb8e00ceff0 2024-04-28T13:31:57.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.117+0000 7fb8e5400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8e015df80 con 0x7fb8e0143840 2024-04-28T13:31:57.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.117+0000 7fb8e5400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e0143c20 con 0x7fb8e00b4eb0 2024-04-28T13:31:57.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.117+0000 7fb8df200640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fb8e00ceff0 0x7fb8e00cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:34318/0 (socket says 172.21.15.196:34318) 2024-04-28T13:31:57.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.117+0000 7fb8df200640 1 -- 172.21.15.196:0/2221155109 learned_addr learned my addr 172.21.15.196:0/2221155109 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:57.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8d7e00640 1 --2- 172.21.15.196:0/2221155109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8e00b4eb0 0x7fb8e00b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8dde00640 1 -- 172.21.15.196:0/2221155109 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1370474890 0 0) 0x7fb8e015dca0 con 0x7fb8e00ceff0 2024-04-28T13:31:57.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8dde00640 1 -- 172.21.15.196:0/2221155109 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8c4003520 con 0x7fb8e00ceff0 2024-04-28T13:31:57.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8dde00640 1 -- 172.21.15.196:0/2221155109 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2028268609 0 0) 0x7fb8c4003520 con 0x7fb8e00ceff0 2024-04-28T13:31:57.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8dde00640 1 -- 172.21.15.196:0/2221155109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8e00b4eb0 msgr2=0x7fb8e00b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:57.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8dde00640 1 --2- 172.21.15.196:0/2221155109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8e00b4eb0 0x7fb8e00b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8dde00640 1 -- 172.21.15.196:0/2221155109 >> v1:172.21.15.203:6789/0 conn(0x7fb8e0143840 legacy=0x7fb8e00b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:57.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8dde00640 1 -- 172.21.15.196:0/2221155109 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e015e260 con 0x7fb8e00ceff0 2024-04-28T13:31:57.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8dde00640 1 -- 172.21.15.196:0/2221155109 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fb8d80019f0 con 0x7fb8e00ceff0 2024-04-28T13:31:57.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8dde00640 1 -- 172.21.15.196:0/2221155109 >> v1:172.21.15.196:6789/0 conn(0x7fb8e00ceff0 legacy=0x7fb8e00cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:57.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8dde00640 1 --2- 172.21.15.196:0/2221155109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8e00b4eb0 0x7fb8c40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8dde00640 1 --2- 172.21.15.196:0/2221155109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8c4003f60 0x7fb8c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8dde00640 1 --2- 172.21.15.196:0/2221155109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8c4004d40 0x7fb8c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8dde00640 1 -- 172.21.15.196:0/2221155109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e0143c20 con 0x7fb8c4004d40 2024-04-28T13:31:57.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8dde00640 1 -- 172.21.15.196:0/2221155109 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c400baa0 con 0x7fb8c4003f60 2024-04-28T13:31:57.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.118+0000 7fb8dde00640 1 -- 172.21.15.196:0/2221155109 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c400bca0 con 0x7fb8e00b4eb0 2024-04-28T13:31:57.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.119+0000 7fb8df200640 1 --2- 172.21.15.196:0/2221155109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8c4004d40 0x7fb8c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.119+0000 7fb8de800640 1 --2- 172.21.15.196:0/2221155109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8e00b4eb0 0x7fb8c40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.119+0000 7fb8d7e00640 1 --2- 172.21.15.196:0/2221155109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8c4003f60 0x7fb8c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.119+0000 7fb8df200640 1 -- 172.21.15.196:0/2221155109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8e00b4eb0 msgr2=0x7fb8c40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:57.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.119+0000 7fb8df200640 1 --2- 172.21.15.196:0/2221155109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8e00b4eb0 0x7fb8c40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.119+0000 7fb8df200640 1 -- 172.21.15.196:0/2221155109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8c4003f60 msgr2=0x7fb8c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:57.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.119+0000 7fb8df200640 1 --2- 172.21.15.196:0/2221155109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8c4003f60 0x7fb8c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.119+0000 7fb8df200640 1 -- 172.21.15.196:0/2221155109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c4003520 con 0x7fb8c4004d40 2024-04-28T13:31:57.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.119+0000 7fb8d7e00640 1 --2- 172.21.15.196:0/2221155109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8c4003f60 0x7fb8c4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:57.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.120+0000 7fb8df200640 1 --2- 172.21.15.196:0/2221155109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8c4004d40 0x7fb8c40051c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800a850 tx=0x7fb8c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b08b424a4b51a35 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:57.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.120+0000 7fb8dde00640 1 -- 172.21.15.196:0/2221155109 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d800a4a0 con 0x7fb8c4004d40 2024-04-28T13:31:57.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.120+0000 7fb8dde00640 1 -- 172.21.15.196:0/2221155109 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d802f070 con 0x7fb8c4004d40 2024-04-28T13:31:57.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.120+0000 7fb8dde00640 1 -- 172.21.15.196:0/2221155109 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d802bd20 con 0x7fb8c4004d40 2024-04-28T13:31:57.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.121+0000 7fb8e5400640 1 -- 172.21.15.196:0/2221155109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8c4004d40 msgr2=0x7fb8c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:57.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.121+0000 7fb8e5400640 1 --2- 172.21.15.196:0/2221155109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8c4004d40 0x7fb8c40051c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800a850 tx=0x7fb8c4005830 comp rx=0 tx=0).stop 2024-04-28T13:31:57.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.121+0000 7fb8df200640 1 -- 172.21.15.196:0/2221155109 reap_dead start 2024-04-28T13:31:57.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.121+0000 7fb8e5400640 1 -- 172.21.15.196:0/2221155109 shutdown_connections 2024-04-28T13:31:57.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.121+0000 7fb8e5400640 1 -- 172.21.15.196:0/2221155109 >> 172.21.15.196:0/2221155109 conn(0x7fb8e00cd8a0 msgr2=0x7fb8e00cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:57.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.121+0000 7fb8e5400640 1 -- 172.21.15.196:0/2221155109 shutdown_connections 2024-04-28T13:31:57.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.121+0000 7fb8e5400640 1 -- 172.21.15.196:0/2221155109 wait complete. 2024-04-28T13:31:57.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.123+0000 7fb8e5400640 1 Processor -- start 2024-04-28T13:31:57.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.123+0000 7fb8e5400640 1 -- start start 2024-04-28T13:31:57.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.124+0000 7fb8e5400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e015c6a0 0x7fb8e00af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.124+0000 7fb8e5400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8e00afcd0 0x7fb8e00732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.124+0000 7fb8e5400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8e0073820 0x7fb8e013e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.124+0000 7fb8e5400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e015d8c0 con 0x7fb8e015c6a0 2024-04-28T13:31:57.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.124+0000 7fb8e5400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e00b46f0 con 0x7fb8e00afcd0 2024-04-28T13:31:57.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.124+0000 7fb8e5400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e015da40 con 0x7fb8e0073820 2024-04-28T13:31:57.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.124+0000 7fb8de800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e015c6a0 0x7fb8e00af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.124+0000 7fb8de800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e015c6a0 0x7fb8e00af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52376/0 (socket says 172.21.15.196:52376) 2024-04-28T13:31:57.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.124+0000 7fb8de800640 1 -- 172.21.15.196:0/2057682825 learned_addr learned my addr 172.21.15.196:0/2057682825 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:57.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.125+0000 7fb8d7e00640 1 --2- 172.21.15.196:0/2057682825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8e00afcd0 0x7fb8e00732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.125+0000 7fb8df200640 1 --2- 172.21.15.196:0/2057682825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8e0073820 0x7fb8e013e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.125+0000 7fb8de800640 1 -- 172.21.15.196:0/2057682825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8e0073820 msgr2=0x7fb8e013e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:57.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.125+0000 7fb8de800640 1 --2- 172.21.15.196:0/2057682825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8e0073820 0x7fb8e013e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.125+0000 7fb8de800640 1 -- 172.21.15.196:0/2057682825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8e00afcd0 msgr2=0x7fb8e00732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:57.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.125+0000 7fb8de800640 1 --2- 172.21.15.196:0/2057682825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8e00afcd0 0x7fb8e00732e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.125+0000 7fb8de800640 1 -- 172.21.15.196:0/2057682825 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e00b0220 con 0x7fb8e015c6a0 2024-04-28T13:31:57.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.125+0000 7fb8df200640 1 --2- 172.21.15.196:0/2057682825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8e0073820 0x7fb8e013e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:31:57.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.125+0000 7fb8de800640 1 --2- 172.21.15.196:0/2057682825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e015c6a0 0x7fb8e00af790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000e5b0 tx=0x7fb8d000e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:57.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.125+0000 7fb8d6a00640 1 -- 172.21.15.196:0/2057682825 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00052e0 con 0x7fb8e015c6a0 2024-04-28T13:31:57.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.126+0000 7fb8d6a00640 1 -- 172.21.15.196:0/2057682825 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d0018070 con 0x7fb8e015c6a0 2024-04-28T13:31:57.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.126+0000 7fb8d6a00640 1 -- 172.21.15.196:0/2057682825 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0003810 con 0x7fb8e015c6a0 2024-04-28T13:31:57.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.126+0000 7fb8e5400640 1 -- 172.21.15.196:0/2057682825 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e013d3f0 con 0x7fb8e015c6a0 2024-04-28T13:31:57.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.126+0000 7fb8e5400640 1 -- 172.21.15.196:0/2057682825 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e013d9b0 con 0x7fb8e015c6a0 2024-04-28T13:31:57.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.127+0000 7fb8e5400640 1 -- 172.21.15.196:0/2057682825 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e006a290 con 0x7fb8e015c6a0 2024-04-28T13:31:57.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.128+0000 7fb8d6a00640 1 -- 172.21.15.196:0/2057682825 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8d00039b0 con 0x7fb8e015c6a0 2024-04-28T13:31:57.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.129+0000 7fb8d6a00640 1 --2- 172.21.15.196:0/2057682825 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb8bc06c8f0 0x7fb8bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.129+0000 7fb8d6a00640 1 -- 172.21.15.196:0/2057682825 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb8d008b750 con 0x7fb8e015c6a0 2024-04-28T13:31:57.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.129+0000 7fb8d7e00640 1 --2- 172.21.15.196:0/2057682825 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb8bc06c8f0 0x7fb8bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.131+0000 7fb8d7e00640 1 --2- 172.21.15.196:0/2057682825 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb8bc06c8f0 0x7fb8bc06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb8e01f88c0 tx=0x7fb8c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:57.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.134+0000 7fb8d6a00640 1 -- 172.21.15.196:0/2057682825 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8d0059b70 con 0x7fb8e015c6a0 2024-04-28T13:31:57.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.332+0000 7fb8e5400640 1 -- 172.21.15.196:0/2057682825 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb8e0073c00 con 0x7fb8e015c6a0 2024-04-28T13:31:57.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.333+0000 7fb8d6a00640 1 -- 172.21.15.196:0/2057682825 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fb8d0059470 con 0x7fb8e015c6a0 2024-04-28T13:31:57.333 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:31:57.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.335+0000 7fb8e5400640 1 -- 172.21.15.196:0/2057682825 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb8bc06c8f0 msgr2=0x7fb8bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:57.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.335+0000 7fb8e5400640 1 --2- 172.21.15.196:0/2057682825 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb8bc06c8f0 0x7fb8bc06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb8e01f88c0 tx=0x7fb8c8008040 comp rx=0 tx=0).stop 2024-04-28T13:31:57.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.335+0000 7fb8e5400640 1 -- 172.21.15.196:0/2057682825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e015c6a0 msgr2=0x7fb8e00af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:57.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.335+0000 7fb8e5400640 1 --2- 172.21.15.196:0/2057682825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e015c6a0 0x7fb8e00af790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000e5b0 tx=0x7fb8d000e9b0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.335+0000 7fb8e5400640 1 -- 172.21.15.196:0/2057682825 shutdown_connections 2024-04-28T13:31:57.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.335+0000 7fb8e5400640 1 --2- 172.21.15.196:0/2057682825 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb8bc06c8f0 0x7fb8bc06edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.335+0000 7fb8e5400640 1 --2- 172.21.15.196:0/2057682825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8e0073820 0x7fb8e013e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.335+0000 7fb8e5400640 1 --2- 172.21.15.196:0/2057682825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8e00afcd0 0x7fb8e00732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.335+0000 7fb8e5400640 1 --2- 172.21.15.196:0/2057682825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e015c6a0 0x7fb8e00af790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.335+0000 7fb8e5400640 1 -- 172.21.15.196:0/2057682825 >> 172.21.15.196:0/2057682825 conn(0x7fb8e00cd8a0 msgr2=0x7fb8e00b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:57.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.336+0000 7fb8e5400640 1 -- 172.21.15.196:0/2057682825 shutdown_connections 2024-04-28T13:31:57.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.336+0000 7fb8e5400640 1 -- 172.21.15.196:0/2057682825 wait complete. 2024-04-28T13:31:57.352 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:57.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.549+0000 7f006a600640 1 Processor -- start 2024-04-28T13:31:57.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.549+0000 7f006a600640 1 -- start start 2024-04-28T13:31:57.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.550+0000 7f006a600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f00641469f0 0x7f00640ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.550+0000 7f006a600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00640cd420 0x7f00640d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.551+0000 7f006a600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0064160f40 con 0x7f00640d1240 2024-04-28T13:31:57.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.551+0000 7f006a600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f00640b5960 con 0x7f00641469f0 2024-04-28T13:31:57.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.551+0000 7f006a600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0064161ac0 con 0x7f00640cd420 2024-04-28T13:31:57.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.551+0000 7f0063e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f00640d1240 0x7f006406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:34334/0 (socket says 172.21.15.196:34334) 2024-04-28T13:31:57.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.551+0000 7f0063e00640 1 -- 172.21.15.196:0/1946553436 learned_addr learned my addr 172.21.15.196:0/1946553436 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:57.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.551+0000 7f005aa00640 1 --2- 172.21.15.196:0/1946553436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00640cd420 0x7f00640d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.551+0000 7f0063400640 1 --2- 172.21.15.196:0/1946553436 >> v2:172.21.15.203:3300/0 conn(0x7f00641469f0 0x7f00640ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.551+0000 7f0062a00640 1 -- 172.21.15.196:0/1946553436 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3759889887 0 0) 0x7f0064160f40 con 0x7f00640d1240 2024-04-28T13:31:57.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.551+0000 7f0062a00640 1 -- 172.21.15.196:0/1946553436 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0048003520 con 0x7f00640d1240 2024-04-28T13:31:57.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0062a00640 1 -- 172.21.15.196:0/1946553436 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2402643895 0 0) 0x7f0048003520 con 0x7f00640d1240 2024-04-28T13:31:57.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0062a00640 1 -- 172.21.15.196:0/1946553436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00640cd420 msgr2=0x7f00640d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:57.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0062a00640 1 --2- 172.21.15.196:0/1946553436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00640cd420 0x7f00640d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0062a00640 1 -- 172.21.15.196:0/1946553436 >> v2:172.21.15.203:3300/0 conn(0x7f00641469f0 msgr2=0x7f00640ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:57.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0062a00640 1 --2- 172.21.15.196:0/1946553436 >> v2:172.21.15.203:3300/0 conn(0x7f00641469f0 0x7f00640ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0062a00640 1 -- 172.21.15.196:0/1946553436 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0064161220 con 0x7f00640d1240 2024-04-28T13:31:57.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0062a00640 1 -- 172.21.15.196:0/1946553436 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f005c0019f0 con 0x7f00640d1240 2024-04-28T13:31:57.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0062a00640 1 -- 172.21.15.196:0/1946553436 >> v1:172.21.15.196:6789/0 conn(0x7f00640d1240 legacy=0x7f006406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:57.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0062a00640 1 --2- 172.21.15.196:0/1946553436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00480039b0 0x7f0048003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0062a00640 1 --2- 172.21.15.196:0/1946553436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00640cd420 0x7f0048004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0062a00640 1 --2- 172.21.15.196:0/1946553436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0048004d40 0x7f00480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0062a00640 1 -- 172.21.15.196:0/1946553436 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0064161ac0 con 0x7f00480039b0 2024-04-28T13:31:57.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0062a00640 1 -- 172.21.15.196:0/1946553436 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f004800b9c0 con 0x7f0048004d40 2024-04-28T13:31:57.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0062a00640 1 -- 172.21.15.196:0/1946553436 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f004800bb40 con 0x7f00640cd420 2024-04-28T13:31:57.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0063400640 1 --2- 172.21.15.196:0/1946553436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00480039b0 0x7f0048003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.552+0000 7f0063e00640 1 --2- 172.21.15.196:0/1946553436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0048004d40 0x7f00480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.553+0000 7f0063400640 1 -- 172.21.15.196:0/1946553436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00640cd420 msgr2=0x7f0048004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:57.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.553+0000 7f0063400640 1 --2- 172.21.15.196:0/1946553436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00640cd420 0x7f0048004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.553+0000 7f0063400640 1 -- 172.21.15.196:0/1946553436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0048004d40 msgr2=0x7f00480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:57.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.553+0000 7f0063400640 1 --2- 172.21.15.196:0/1946553436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0048004d40 0x7f00480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.553+0000 7f0063400640 1 -- 172.21.15.196:0/1946553436 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0048003520 con 0x7f00480039b0 2024-04-28T13:31:57.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.553+0000 7f0063e00640 1 --2- 172.21.15.196:0/1946553436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0048004d40 0x7f00480051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:57.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.553+0000 7f0063400640 1 --2- 172.21.15.196:0/1946553436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00480039b0 0x7f0048003dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0054009af0 tx=0x7f0048005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6bdd4cf56f1b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:57.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.554+0000 7f0062a00640 1 -- 172.21.15.196:0/1946553436 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054036070 con 0x7f00480039b0 2024-04-28T13:31:57.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.554+0000 7f0062a00640 1 -- 172.21.15.196:0/1946553436 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0054031400 con 0x7f00480039b0 2024-04-28T13:31:57.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.554+0000 7f0062a00640 1 -- 172.21.15.196:0/1946553436 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054009690 con 0x7f00480039b0 2024-04-28T13:31:57.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.554+0000 7f006a600640 1 -- 172.21.15.196:0/1946553436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00480039b0 msgr2=0x7f0048003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:57.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.554+0000 7f006a600640 1 --2- 172.21.15.196:0/1946553436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00480039b0 0x7f0048003dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0054009af0 tx=0x7f0048005830 comp rx=0 tx=0).stop 2024-04-28T13:31:57.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.554+0000 7f0063e00640 1 -- 172.21.15.196:0/1946553436 reap_dead start 2024-04-28T13:31:57.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.554+0000 7f006a600640 1 -- 172.21.15.196:0/1946553436 shutdown_connections 2024-04-28T13:31:57.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.554+0000 7f006a600640 1 -- 172.21.15.196:0/1946553436 >> 172.21.15.196:0/1946553436 conn(0x7f00640cd830 msgr2=0x7f00640cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:57.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.555+0000 7f006a600640 1 -- 172.21.15.196:0/1946553436 shutdown_connections 2024-04-28T13:31:57.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.555+0000 7f006a600640 1 -- 172.21.15.196:0/1946553436 wait complete. 2024-04-28T13:31:57.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.556+0000 7f006a600640 1 Processor -- start 2024-04-28T13:31:57.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.557+0000 7f006a600640 1 -- start start 2024-04-28T13:31:57.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.557+0000 7f006a600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00480039b0 0x7f00641396b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.557+0000 7f006a600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00641436f0 0x7f00641f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.558+0000 7f006a600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0064143b50 0x7f00641f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.558+0000 7f006a600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0064072020 con 0x7f00480039b0 2024-04-28T13:31:57.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.558+0000 7f006a600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f00641617e0 con 0x7f00641436f0 2024-04-28T13:31:57.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.558+0000 7f006a600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f00640b6de0 con 0x7f0064143b50 2024-04-28T13:31:57.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.558+0000 7f0063400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00480039b0 0x7f00641396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.558+0000 7f0063400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00480039b0 0x7f00641396b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52390/0 (socket says 172.21.15.196:52390) 2024-04-28T13:31:57.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.558+0000 7f0063400640 1 -- 172.21.15.196:0/3580753634 learned_addr learned my addr 172.21.15.196:0/3580753634 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:57.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.558+0000 7f0063400640 1 -- 172.21.15.196:0/3580753634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0064143b50 msgr2=0x7f00641f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:57.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.558+0000 7f0063400640 1 --2- 172.21.15.196:0/3580753634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0064143b50 0x7f00641f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.558+0000 7f0063400640 1 -- 172.21.15.196:0/3580753634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00641436f0 msgr2=0x7f00641f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:57.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.558+0000 7f0063400640 1 --2- 172.21.15.196:0/3580753634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00641436f0 0x7f00641f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.558+0000 7f0063400640 1 -- 172.21.15.196:0/3580753634 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00641440d0 con 0x7f00480039b0 2024-04-28T13:31:57.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.558+0000 7f0063400640 1 --2- 172.21.15.196:0/3580753634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00480039b0 0x7f00641396b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0054009af0 tx=0x7f0054004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:57.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.559+0000 7f005be00640 1 -- 172.21.15.196:0/3580753634 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054041020 con 0x7f00480039b0 2024-04-28T13:31:57.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.559+0000 7f005be00640 1 -- 172.21.15.196:0/3580753634 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f005403c470 con 0x7f00480039b0 2024-04-28T13:31:57.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.559+0000 7f005be00640 1 -- 172.21.15.196:0/3580753634 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054036040 con 0x7f00480039b0 2024-04-28T13:31:57.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.559+0000 7f006a600640 1 -- 172.21.15.196:0/3580753634 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00641443b0 con 0x7f00480039b0 2024-04-28T13:31:57.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.559+0000 7f006a600640 1 -- 172.21.15.196:0/3580753634 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0064144970 con 0x7f00480039b0 2024-04-28T13:31:57.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.560+0000 7f006a600640 1 -- 172.21.15.196:0/3580753634 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f006406a290 con 0x7f00480039b0 2024-04-28T13:31:57.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.561+0000 7f005be00640 1 -- 172.21.15.196:0/3580753634 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0054044020 con 0x7f00480039b0 2024-04-28T13:31:57.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.562+0000 7f005be00640 1 --2- 172.21.15.196:0/3580753634 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f004006c920 0x7f004006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.562+0000 7f005be00640 1 -- 172.21.15.196:0/3580753634 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f00540abab0 con 0x7f00480039b0 2024-04-28T13:31:57.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.562+0000 7f005aa00640 1 --2- 172.21.15.196:0/3580753634 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f004006c920 0x7f004006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.563+0000 7f005aa00640 1 --2- 172.21.15.196:0/3580753634 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f004006c920 0x7f004006ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f004c006a90 tx=0x7f004c006a20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:57.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.566+0000 7f005be00640 1 -- 172.21.15.196:0/3580753634 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0054079e50 con 0x7f00480039b0 2024-04-28T13:31:57.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.764+0000 7f006a600640 1 -- 172.21.15.196:0/3580753634 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00640cd200 con 0x7f00480039b0 2024-04-28T13:31:57.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.767+0000 7f005be00640 1 -- 172.21.15.196:0/3580753634 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f0054079750 con 0x7f00480039b0 2024-04-28T13:31:57.767 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:57.767 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":20,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:22.624564+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:57.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.769+0000 7f006a600640 1 -- 172.21.15.196:0/3580753634 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f004006c920 msgr2=0x7f004006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:57.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.769+0000 7f006a600640 1 --2- 172.21.15.196:0/3580753634 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f004006c920 0x7f004006ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f004c006a90 tx=0x7f004c006a20 comp rx=0 tx=0).stop 2024-04-28T13:31:57.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.769+0000 7f006a600640 1 -- 172.21.15.196:0/3580753634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00480039b0 msgr2=0x7f00641396b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:57.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.769+0000 7f006a600640 1 --2- 172.21.15.196:0/3580753634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00480039b0 0x7f00641396b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0054009af0 tx=0x7f0054004730 comp rx=0 tx=0).stop 2024-04-28T13:31:57.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.769+0000 7f006a600640 1 -- 172.21.15.196:0/3580753634 shutdown_connections 2024-04-28T13:31:57.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.769+0000 7f006a600640 1 --2- 172.21.15.196:0/3580753634 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f004006c920 0x7f004006ede0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.769+0000 7f006a600640 1 --2- 172.21.15.196:0/3580753634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0064143b50 0x7f00641f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.769+0000 7f006a600640 1 --2- 172.21.15.196:0/3580753634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00641436f0 0x7f00641f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.769+0000 7f006a600640 1 --2- 172.21.15.196:0/3580753634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00480039b0 0x7f00641396b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.769+0000 7f006a600640 1 -- 172.21.15.196:0/3580753634 >> 172.21.15.196:0/3580753634 conn(0x7f00640cd830 msgr2=0x7f00640b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:57.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.770+0000 7f006a600640 1 -- 172.21.15.196:0/3580753634 shutdown_connections 2024-04-28T13:31:57.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.770+0000 7f006a600640 1 -- 172.21.15.196:0/3580753634 wait complete. 2024-04-28T13:31:57.789 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:31:19.280975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:31:20.037066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:31:57.789 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:31:57.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.993+0000 7fcf71400640 1 Processor -- start 2024-04-28T13:31:57.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.993+0000 7fcf71400640 1 -- start start 2024-04-28T13:31:57.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.995+0000 7fcf71400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcf6c0d17b0 0x7fcf6c06f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.995+0000 7fcf71400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf6c0b5df0 0x7fcf6c072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.995+0000 7fcf71400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf6c06df10 con 0x7fcf6c15f150 2024-04-28T13:31:57.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.995+0000 7fcf71400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fcf6c06fda0 con 0x7fcf6c0d17b0 2024-04-28T13:31:57.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.995+0000 7fcf71400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcf6c06e1f0 con 0x7fcf6c0b5df0 2024-04-28T13:31:57.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.995+0000 7fcf6be00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fcf6c15f150 0x7fcf6c0d10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:34346/0 (socket says 172.21.15.196:34346) 2024-04-28T13:31:57.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.995+0000 7fcf6be00640 1 -- 172.21.15.196:0/3587361927 learned_addr learned my addr 172.21.15.196:0/3587361927 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:57.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6b400640 1 --2- 172.21.15.196:0/3587361927 >> v2:172.21.15.203:3300/0 conn(0x7fcf6c0d17b0 0x7fcf6c06f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf70a00640 1 --2- 172.21.15.196:0/3587361927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf6c0b5df0 0x7fcf6c072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 -- 172.21.15.196:0/3587361927 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1753618428 0 0) 0x7fcf6c06df10 con 0x7fcf6c15f150 2024-04-28T13:31:57.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 -- 172.21.15.196:0/3587361927 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf50003520 con 0x7fcf6c15f150 2024-04-28T13:31:57.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 -- 172.21.15.196:0/3587361927 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 89396907 0 0) 0x7fcf50003520 con 0x7fcf6c15f150 2024-04-28T13:31:57.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 -- 172.21.15.196:0/3587361927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf6c0b5df0 msgr2=0x7fcf6c072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:57.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 --2- 172.21.15.196:0/3587361927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf6c0b5df0 0x7fcf6c072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 -- 172.21.15.196:0/3587361927 >> v2:172.21.15.203:3300/0 conn(0x7fcf6c0d17b0 msgr2=0x7fcf6c06f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:57.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 --2- 172.21.15.196:0/3587361927 >> v2:172.21.15.203:3300/0 conn(0x7fcf6c0d17b0 0x7fcf6c06f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 -- 172.21.15.196:0/3587361927 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6c06e4d0 con 0x7fcf6c15f150 2024-04-28T13:31:57.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 -- 172.21.15.196:0/3587361927 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fcf5c001670 con 0x7fcf6c15f150 2024-04-28T13:31:57.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 -- 172.21.15.196:0/3587361927 >> v1:172.21.15.196:6789/0 conn(0x7fcf6c15f150 legacy=0x7fcf6c0d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:57.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 --2- 172.21.15.196:0/3587361927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf500039b0 0x7fcf50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 --2- 172.21.15.196:0/3587361927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf6c0b5df0 0x7fcf50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 --2- 172.21.15.196:0/3587361927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf50004d40 0x7fcf500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 -- 172.21.15.196:0/3587361927 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c06fda0 con 0x7fcf500039b0 2024-04-28T13:31:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 -- 172.21.15.196:0/3587361927 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c06e1f0 con 0x7fcf50004d40 2024-04-28T13:31:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.996+0000 7fcf6aa00640 1 -- 172.21.15.196:0/3587361927 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcf5000b9c0 con 0x7fcf6c0b5df0 2024-04-28T13:31:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.997+0000 7fcf6be00640 1 --2- 172.21.15.196:0/3587361927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf500039b0 0x7fcf50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.997+0000 7fcf6b400640 1 --2- 172.21.15.196:0/3587361927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf6c0b5df0 0x7fcf50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.997+0000 7fcf70a00640 1 --2- 172.21.15.196:0/3587361927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf50004d40 0x7fcf500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.997+0000 7fcf6be00640 1 -- 172.21.15.196:0/3587361927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf6c0b5df0 msgr2=0x7fcf50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.997+0000 7fcf6be00640 1 --2- 172.21.15.196:0/3587361927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf6c0b5df0 0x7fcf50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.997+0000 7fcf6be00640 1 -- 172.21.15.196:0/3587361927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf50004d40 msgr2=0x7fcf500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.997+0000 7fcf6be00640 1 --2- 172.21.15.196:0/3587361927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf50004d40 0x7fcf500051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.997+0000 7fcf6be00640 1 -- 172.21.15.196:0/3587361927 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf50003520 con 0x7fcf500039b0 2024-04-28T13:31:58.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.998+0000 7fcf6be00640 1 --2- 172.21.15.196:0/3587361927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf500039b0 0x7fcf50003dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00c240 tx=0x7fcf50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de150ba2f71882f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:58.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.998+0000 7fcf6aa00640 1 -- 172.21.15.196:0/3587361927 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c02e070 con 0x7fcf500039b0 2024-04-28T13:31:58.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.998+0000 7fcf6aa00640 1 -- 172.21.15.196:0/3587361927 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf5c005af0 con 0x7fcf500039b0 2024-04-28T13:31:58.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.998+0000 7fcf6aa00640 1 -- 172.21.15.196:0/3587361927 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c007d30 con 0x7fcf500039b0 2024-04-28T13:31:58.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.999+0000 7fcf71400640 1 -- 172.21.15.196:0/3587361927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf500039b0 msgr2=0x7fcf50003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:58.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.999+0000 7fcf71400640 1 --2- 172.21.15.196:0/3587361927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf500039b0 0x7fcf50003dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00c240 tx=0x7fcf50005830 comp rx=0 tx=0).stop 2024-04-28T13:31:58.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.999+0000 7fcf70a00640 1 -- 172.21.15.196:0/3587361927 reap_dead start 2024-04-28T13:31:58.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.999+0000 7fcf71400640 1 -- 172.21.15.196:0/3587361927 shutdown_connections 2024-04-28T13:31:58.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.999+0000 7fcf71400640 1 -- 172.21.15.196:0/3587361927 >> 172.21.15.196:0/3587361927 conn(0x7fcf6c06c6c0 msgr2=0x7fcf6c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:58.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.999+0000 7fcf71400640 1 -- 172.21.15.196:0/3587361927 shutdown_connections 2024-04-28T13:31:58.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:57.999+0000 7fcf71400640 1 -- 172.21.15.196:0/3587361927 wait complete. 2024-04-28T13:31:58.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.001+0000 7fcf71400640 1 Processor -- start 2024-04-28T13:31:58.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.001+0000 7fcf71400640 1 -- start start 2024-04-28T13:31:58.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.002+0000 7fcf71400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf500039b0 0x7fcf6c14b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.002+0000 7fcf71400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf6c155420 0x7fcf6c0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.002+0000 7fcf71400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf6c15d610 0x7fcf6c15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.002+0000 7fcf71400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c15f530 con 0x7fcf6c155420 2024-04-28T13:31:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.002+0000 7fcf71400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c06ea90 con 0x7fcf6c15d610 2024-04-28T13:31:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.002+0000 7fcf71400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcf6c072150 con 0x7fcf500039b0 2024-04-28T13:31:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.002+0000 7fcf6b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf6c155420 0x7fcf6c0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.002+0000 7fcf6b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf6c155420 0x7fcf6c0b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52410/0 (socket says 172.21.15.196:52410) 2024-04-28T13:31:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.002+0000 7fcf6b400640 1 -- 172.21.15.196:0/3417387618 learned_addr learned my addr 172.21.15.196:0/3417387618 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.003+0000 7fcf6be00640 1 --2- 172.21.15.196:0/3417387618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf500039b0 0x7fcf6c14b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.003+0000 7fcf70a00640 1 --2- 172.21.15.196:0/3417387618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf6c15d610 0x7fcf6c15dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.003+0000 7fcf6b400640 1 -- 172.21.15.196:0/3417387618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf500039b0 msgr2=0x7fcf6c14b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.003+0000 7fcf6b400640 1 --2- 172.21.15.196:0/3417387618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf500039b0 0x7fcf6c14b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.003+0000 7fcf6b400640 1 -- 172.21.15.196:0/3417387618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf6c15d610 msgr2=0x7fcf6c15dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.003+0000 7fcf6b400640 1 --2- 172.21.15.196:0/3417387618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf6c15d610 0x7fcf6c15dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.003+0000 7fcf6b400640 1 -- 172.21.15.196:0/3417387618 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6c0b20c0 con 0x7fcf6c155420 2024-04-28T13:31:58.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.003+0000 7fcf6b400640 1 --2- 172.21.15.196:0/3417387618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf6c155420 0x7fcf6c0b3160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcf58011630 tx=0x7fcf58011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:58.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.003+0000 7fcf63e00640 1 -- 172.21.15.196:0/3417387618 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58015870 con 0x7fcf6c155420 2024-04-28T13:31:58.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.004+0000 7fcf63e00640 1 -- 172.21.15.196:0/3417387618 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf58015a10 con 0x7fcf6c155420 2024-04-28T13:31:58.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.004+0000 7fcf71400640 1 -- 172.21.15.196:0/3417387618 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf6c0b23a0 con 0x7fcf6c155420 2024-04-28T13:31:58.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.004+0000 7fcf63e00640 1 -- 172.21.15.196:0/3417387618 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58015dc0 con 0x7fcf6c155420 2024-04-28T13:31:58.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.004+0000 7fcf71400640 1 -- 172.21.15.196:0/3417387618 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf6c0b2960 con 0x7fcf6c155420 2024-04-28T13:31:58.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.005+0000 7fcf71400640 1 -- 172.21.15.196:0/3417387618 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf6c050730 con 0x7fcf6c155420 2024-04-28T13:31:58.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.006+0000 7fcf63e00640 1 -- 172.21.15.196:0/3417387618 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf58013430 con 0x7fcf6c155420 2024-04-28T13:31:58.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.007+0000 7fcf63e00640 1 --2- 172.21.15.196:0/3417387618 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcf4c06c920 0x7fcf4c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.007+0000 7fcf63e00640 1 -- 172.21.15.196:0/3417387618 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcf5808c360 con 0x7fcf6c155420 2024-04-28T13:31:58.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.007+0000 7fcf6be00640 1 --2- 172.21.15.196:0/3417387618 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcf4c06c920 0x7fcf4c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.008+0000 7fcf6be00640 1 --2- 172.21.15.196:0/3417387618 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcf4c06c920 0x7fcf4c06ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c033000 tx=0x7fcf5c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:58.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.011+0000 7fcf63e00640 1 -- 172.21.15.196:0/3417387618 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf58019030 con 0x7fcf6c155420 2024-04-28T13:31:58.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.208+0000 7fcf71400640 1 -- 172.21.15.196:0/3417387618 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf6c072f60 con 0x7fcf6c155420 2024-04-28T13:31:58.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.209+0000 7fcf63e00640 1 -- 172.21.15.196:0/3417387618 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fcf5805a780 con 0x7fcf6c155420 2024-04-28T13:31:58.209 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:31:58.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.211+0000 7fcf71400640 1 -- 172.21.15.196:0/3417387618 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcf4c06c920 msgr2=0x7fcf4c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:58.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.211+0000 7fcf71400640 1 --2- 172.21.15.196:0/3417387618 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcf4c06c920 0x7fcf4c06ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c033000 tx=0x7fcf5c009030 comp rx=0 tx=0).stop 2024-04-28T13:31:58.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.211+0000 7fcf71400640 1 -- 172.21.15.196:0/3417387618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf6c155420 msgr2=0x7fcf6c0b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:58.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.211+0000 7fcf71400640 1 --2- 172.21.15.196:0/3417387618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf6c155420 0x7fcf6c0b3160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcf58011630 tx=0x7fcf58011af0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.212+0000 7fcf71400640 1 -- 172.21.15.196:0/3417387618 shutdown_connections 2024-04-28T13:31:58.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.212+0000 7fcf71400640 1 --2- 172.21.15.196:0/3417387618 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcf4c06c920 0x7fcf4c06ede0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.212+0000 7fcf71400640 1 --2- 172.21.15.196:0/3417387618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf6c15d610 0x7fcf6c15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.212+0000 7fcf71400640 1 --2- 172.21.15.196:0/3417387618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf6c155420 0x7fcf6c0b3160 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.212+0000 7fcf71400640 1 --2- 172.21.15.196:0/3417387618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf500039b0 0x7fcf6c14b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.212+0000 7fcf71400640 1 -- 172.21.15.196:0/3417387618 >> 172.21.15.196:0/3417387618 conn(0x7fcf6c06c6c0 msgr2=0x7fcf6c0b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:58.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.212+0000 7fcf71400640 1 -- 172.21.15.196:0/3417387618 shutdown_connections 2024-04-28T13:31:58.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.212+0000 7fcf71400640 1 -- 172.21.15.196:0/3417387618 wait complete. 2024-04-28T13:31:58.229 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:31:58.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.424+0000 7f10ee400640 1 Processor -- start 2024-04-28T13:31:58.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.425+0000 7f10ee400640 1 -- start start 2024-04-28T13:31:58.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.426+0000 7f10ee400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e8074200 0x7f10e8166670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.426+0000 7f10ee400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e815c530 con 0x7f10e815b930 2024-04-28T13:31:58.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.426+0000 7f10ee400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e815c810 con 0x7f10e815f4f0 2024-04-28T13:31:58.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.426+0000 7f10ee400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f10e815caf0 con 0x7f10e8074200 2024-04-28T13:31:58.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.427+0000 7f10e7400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f10e815b930 0x7f10e815bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:34358/0 (socket says 172.21.15.196:34358) 2024-04-28T13:31:58.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.427+0000 7f10e7400640 1 -- 172.21.15.196:0/4019453320 learned_addr learned my addr 172.21.15.196:0/4019453320 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:58.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.427+0000 7f10e7e00640 1 --2- 172.21.15.196:0/4019453320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e8074200 0x7f10e8166670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.427+0000 7f10e6000640 1 -- 172.21.15.196:0/4019453320 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2679193662 0 0) 0x7f10e815c530 con 0x7f10e815b930 2024-04-28T13:31:58.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.427+0000 7f10e6000640 1 -- 172.21.15.196:0/4019453320 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10cc003440 con 0x7f10e815b930 2024-04-28T13:31:58.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.427+0000 7f10e6000640 1 -- 172.21.15.196:0/4019453320 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 715056228 0 0) 0x7f10cc003440 con 0x7f10e815b930 2024-04-28T13:31:58.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.427+0000 7f10e6000640 1 -- 172.21.15.196:0/4019453320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e8074200 msgr2=0x7f10e8166670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:58.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.427+0000 7f10e6000640 1 --2- 172.21.15.196:0/4019453320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e8074200 0x7f10e8166670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.427+0000 7f10e6000640 1 -- 172.21.15.196:0/4019453320 >> v1:172.21.15.203:6789/0 conn(0x7f10e815f4f0 legacy=0x7f10e8073af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:58.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.427+0000 7f10e6000640 1 -- 172.21.15.196:0/4019453320 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e815cdd0 con 0x7f10e815b930 2024-04-28T13:31:58.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e6000640 1 -- 172.21.15.196:0/4019453320 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f10d8001670 con 0x7f10e815b930 2024-04-28T13:31:58.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e6000640 1 -- 172.21.15.196:0/4019453320 >> v1:172.21.15.196:6789/0 conn(0x7f10e815b930 legacy=0x7f10e815bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:58.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e6000640 1 --2- 172.21.15.196:0/4019453320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e8074200 0x7f10cc003ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e6000640 1 --2- 172.21.15.196:0/4019453320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10cc004050 0x7f10cc00a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e6000640 1 --2- 172.21.15.196:0/4019453320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10cc00aff0 0x7f10cc00d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e6000640 1 -- 172.21.15.196:0/4019453320 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10e815caf0 con 0x7f10cc004050 2024-04-28T13:31:58.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e6000640 1 -- 172.21.15.196:0/4019453320 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc00dcc0 con 0x7f10cc00aff0 2024-04-28T13:31:58.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e6000640 1 -- 172.21.15.196:0/4019453320 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc00dec0 con 0x7f10e8074200 2024-04-28T13:31:58.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e6a00640 1 --2- 172.21.15.196:0/4019453320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10cc004050 0x7f10cc00a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e7400640 1 --2- 172.21.15.196:0/4019453320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e8074200 0x7f10cc003ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e7e00640 1 --2- 172.21.15.196:0/4019453320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10cc00aff0 0x7f10cc00d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e6a00640 1 -- 172.21.15.196:0/4019453320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e8074200 msgr2=0x7f10cc003ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:58.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e6a00640 1 --2- 172.21.15.196:0/4019453320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e8074200 0x7f10cc003ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e6a00640 1 -- 172.21.15.196:0/4019453320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10cc00aff0 msgr2=0x7f10cc00d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:58.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e6a00640 1 --2- 172.21.15.196:0/4019453320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10cc00aff0 0x7f10cc00d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e6a00640 1 -- 172.21.15.196:0/4019453320 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc003440 con 0x7f10cc004050 2024-04-28T13:31:58.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.428+0000 7f10e7e00640 1 --2- 172.21.15.196:0/4019453320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10cc00aff0 0x7f10cc00d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:58.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.429+0000 7f10e6a00640 1 --2- 172.21.15.196:0/4019453320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10cc004050 0x7f10cc00a860 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f10d4001720 tx=0x7f10cc003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1904fd56c9151b77 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:58.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.430+0000 7f10e6000640 1 -- 172.21.15.196:0/4019453320 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4001370 con 0x7f10cc004050 2024-04-28T13:31:58.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.430+0000 7f10e6000640 1 -- 172.21.15.196:0/4019453320 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d40052d0 con 0x7f10cc004050 2024-04-28T13:31:58.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.430+0000 7f10e6000640 1 -- 172.21.15.196:0/4019453320 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d402abc0 con 0x7f10cc004050 2024-04-28T13:31:58.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.430+0000 7f10ee400640 1 -- 172.21.15.196:0/4019453320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10cc004050 msgr2=0x7f10cc00a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:58.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.430+0000 7f10ee400640 1 --2- 172.21.15.196:0/4019453320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10cc004050 0x7f10cc00a860 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f10d4001720 tx=0x7f10cc003730 comp rx=0 tx=0).stop 2024-04-28T13:31:58.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.430+0000 7f10e7e00640 1 -- 172.21.15.196:0/4019453320 reap_dead start 2024-04-28T13:31:58.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.430+0000 7f10ee400640 1 -- 172.21.15.196:0/4019453320 shutdown_connections 2024-04-28T13:31:58.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.430+0000 7f10ee400640 1 -- 172.21.15.196:0/4019453320 >> 172.21.15.196:0/4019453320 conn(0x7f10e80cd8a0 msgr2=0x7f10e80cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:58.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.430+0000 7f10ee400640 1 -- 172.21.15.196:0/4019453320 shutdown_connections 2024-04-28T13:31:58.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.430+0000 7f10ee400640 1 -- 172.21.15.196:0/4019453320 wait complete. 2024-04-28T13:31:58.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.432+0000 7f10ee400640 1 Processor -- start 2024-04-28T13:31:58.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.432+0000 7f10ee400640 1 -- start start 2024-04-28T13:31:58.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.433+0000 7f10ee400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10cc004050 0x7f10e81ea420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.433+0000 7f10ee400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10e814d200 0x7f10e814d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.433+0000 7f10ee400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e81e6370 0x7f10e81e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.433+0000 7f10ee400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10e80d0270 con 0x7f10e81e6370 2024-04-28T13:31:58.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.433+0000 7f10ee400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f10e806fdd0 con 0x7f10e814d200 2024-04-28T13:31:58.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.433+0000 7f10ee400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f10e80d03f0 con 0x7f10cc004050 2024-04-28T13:31:58.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.434+0000 7f10e7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e81e6370 0x7f10e81e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.434+0000 7f10e7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e81e6370 0x7f10e81e67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52432/0 (socket says 172.21.15.196:52432) 2024-04-28T13:31:58.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.434+0000 7f10e7e00640 1 -- 172.21.15.196:0/1325553054 learned_addr learned my addr 172.21.15.196:0/1325553054 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:58.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.434+0000 7f10e7400640 1 --2- 172.21.15.196:0/1325553054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10cc004050 0x7f10e81ea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.434+0000 7f10e6a00640 1 --2- 172.21.15.196:0/1325553054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10e814d200 0x7f10e814d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.434+0000 7f10e7e00640 1 -- 172.21.15.196:0/1325553054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10cc004050 msgr2=0x7f10e81ea420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:58.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.434+0000 7f10e7e00640 1 --2- 172.21.15.196:0/1325553054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10cc004050 0x7f10e81ea420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.434+0000 7f10e7e00640 1 -- 172.21.15.196:0/1325553054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10e814d200 msgr2=0x7f10e814d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:58.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.434+0000 7f10e7e00640 1 --2- 172.21.15.196:0/1325553054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10e814d200 0x7f10e814d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.434+0000 7f10e7e00640 1 -- 172.21.15.196:0/1325553054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e814dbc0 con 0x7f10e81e6370 2024-04-28T13:31:58.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.435+0000 7f10e7e00640 1 --2- 172.21.15.196:0/1325553054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e81e6370 0x7f10e81e67f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f10e815f8d0 tx=0x7f10e000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:58.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.435+0000 7f10df400640 1 -- 172.21.15.196:0/1325553054 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e001c070 con 0x7f10e81e6370 2024-04-28T13:31:58.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.435+0000 7f10df400640 1 -- 172.21.15.196:0/1325553054 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10e0017400 con 0x7f10e81e6370 2024-04-28T13:31:58.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.435+0000 7f10df400640 1 -- 172.21.15.196:0/1325553054 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e00177f0 con 0x7f10e81e6370 2024-04-28T13:31:58.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.435+0000 7f10ee400640 1 -- 172.21.15.196:0/1325553054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e81e70e0 con 0x7f10e81e6370 2024-04-28T13:31:58.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.435+0000 7f10ee400640 1 -- 172.21.15.196:0/1325553054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e81e7db0 con 0x7f10e81e6370 2024-04-28T13:31:58.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.436+0000 7f10ee400640 1 -- 172.21.15.196:0/1325553054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e806fa50 con 0x7f10e81e6370 2024-04-28T13:31:58.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.437+0000 7f10df400640 1 -- 172.21.15.196:0/1325553054 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10e0005d10 con 0x7f10e81e6370 2024-04-28T13:31:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.438+0000 7f10df400640 1 --2- 172.21.15.196:0/1325553054 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f10c806c920 0x7f10c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.438+0000 7f10df400640 1 -- 172.21.15.196:0/1325553054 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f10e008bde0 con 0x7f10e81e6370 2024-04-28T13:31:58.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.439+0000 7f10e7400640 1 --2- 172.21.15.196:0/1325553054 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f10c806c920 0x7f10c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.440+0000 7f10e7400640 1 --2- 172.21.15.196:0/1325553054 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f10c806c920 0x7f10c806ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f10d8001cd0 tx=0x7f10d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:58.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.443+0000 7f10df400640 1 -- 172.21.15.196:0/1325553054 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f10e806fa50 con 0x7f10e81e6370 2024-04-28T13:31:58.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.634+0000 7f10ee400640 1 -- 172.21.15.196:0/1325553054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f10e806a290 con 0x7f10e81e6370 2024-04-28T13:31:58.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.636+0000 7f10df400640 1 -- 172.21.15.196:0/1325553054 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f10e0012070 con 0x7f10e81e6370 2024-04-28T13:31:58.635 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:31:58.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.638+0000 7f10ee400640 1 -- 172.21.15.196:0/1325553054 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f10c806c920 msgr2=0x7f10c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:58.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.638+0000 7f10ee400640 1 --2- 172.21.15.196:0/1325553054 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f10c806c920 0x7f10c806ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f10d8001cd0 tx=0x7f10d8009040 comp rx=0 tx=0).stop 2024-04-28T13:31:58.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.638+0000 7f10ee400640 1 -- 172.21.15.196:0/1325553054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e81e6370 msgr2=0x7f10e81e67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:58.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.638+0000 7f10ee400640 1 --2- 172.21.15.196:0/1325553054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e81e6370 0x7f10e81e67f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f10e815f8d0 tx=0x7f10e000f4a0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.638+0000 7f10ee400640 1 -- 172.21.15.196:0/1325553054 shutdown_connections 2024-04-28T13:31:58.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.638+0000 7f10ee400640 1 --2- 172.21.15.196:0/1325553054 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f10c806c920 0x7f10c806ede0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.638+0000 7f10ee400640 1 --2- 172.21.15.196:0/1325553054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e81e6370 0x7f10e81e67f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.638+0000 7f10ee400640 1 --2- 172.21.15.196:0/1325553054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10e814d200 0x7f10e814d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.638+0000 7f10ee400640 1 --2- 172.21.15.196:0/1325553054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10cc004050 0x7f10e81ea420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.638+0000 7f10ee400640 1 -- 172.21.15.196:0/1325553054 >> 172.21.15.196:0/1325553054 conn(0x7f10e80cd8a0 msgr2=0x7f10e815bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:58.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.638+0000 7f10ee400640 1 -- 172.21.15.196:0/1325553054 shutdown_connections 2024-04-28T13:31:58.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.638+0000 7f10ee400640 1 -- 172.21.15.196:0/1325553054 wait complete. 2024-04-28T13:31:58.656 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:31:58.854 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.853+0000 7f73ecc00640 1 Processor -- start 2024-04-28T13:31:58.854 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.854+0000 7f73ecc00640 1 -- start start 2024-04-28T13:31:58.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.855+0000 7f73ecc00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e815f680 0x7f73e80d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.855+0000 7f73ecc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f73e8163aa0 0x7f73e80b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.855+0000 7f73ecc00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f73e80b60b0 0x7f73e80d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.855+0000 7f73ecc00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f73e815fa60 con 0x7f73e8163aa0 2024-04-28T13:31:58.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.855+0000 7f73ecc00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f73e80d3f50 con 0x7f73e80b60b0 2024-04-28T13:31:58.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.855+0000 7f73ecc00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f73e80b5090 con 0x7f73e815f680 2024-04-28T13:31:58.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.856+0000 7f73e5400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f73e8163aa0 0x7f73e80b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.856+0000 7f73e5400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f73e8163aa0 0x7f73e80b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52446/0 (socket says 172.21.15.196:52446) 2024-04-28T13:31:58.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.856+0000 7f73e5400640 1 -- 172.21.15.196:0/750845300 learned_addr learned my addr 172.21.15.196:0/750845300 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:58.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.856+0000 7f73e5400640 1 -- 172.21.15.196:0/750845300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e815f680 msgr2=0x7f73e80d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:58.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.856+0000 7f73e5e00640 1 --2- 172.21.15.196:0/750845300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e815f680 0x7f73e80d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.856+0000 7f73e6800640 1 --2- 172.21.15.196:0/750845300 >> v2:172.21.15.203:3300/0 conn(0x7f73e80b60b0 0x7f73e80d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.856+0000 7f73e5400640 1 --2- 172.21.15.196:0/750845300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e815f680 0x7f73e80d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.856+0000 7f73e5400640 1 -- 172.21.15.196:0/750845300 >> v2:172.21.15.203:3300/0 conn(0x7f73e80b60b0 msgr2=0x7f73e80d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:58.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.856+0000 7f73e5400640 1 --2- 172.21.15.196:0/750845300 >> v2:172.21.15.203:3300/0 conn(0x7f73e80b60b0 0x7f73e80d4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.856+0000 7f73e5400640 1 -- 172.21.15.196:0/750845300 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e80b5370 con 0x7f73e8163aa0 2024-04-28T13:31:58.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.856+0000 7f73e5e00640 1 --2- 172.21.15.196:0/750845300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e815f680 0x7f73e80d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:58.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.857+0000 7f73e5400640 1 --2- 172.21.15.196:0/750845300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e8163aa0 0x7f73e80b5b70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f73e80b4b10 tx=0x7f73d402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f791a6b7186da5fa server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:58.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.858+0000 7f73e4a00640 1 -- 172.21.15.196:0/750845300 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4037070 con 0x7f73e8163aa0 2024-04-28T13:31:58.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.858+0000 7f73e4a00640 1 -- 172.21.15.196:0/750845300 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73d4032440 con 0x7f73e8163aa0 2024-04-28T13:31:58.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.858+0000 7f73e4a00640 1 -- 172.21.15.196:0/750845300 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4032810 con 0x7f73e8163aa0 2024-04-28T13:31:58.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.858+0000 7f73ecc00640 1 -- 172.21.15.196:0/750845300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e8163aa0 msgr2=0x7f73e80b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:58.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.858+0000 7f73ecc00640 1 --2- 172.21.15.196:0/750845300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e8163aa0 0x7f73e80b5b70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f73e80b4b10 tx=0x7f73d402a1e0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.859+0000 7f73ecc00640 1 -- 172.21.15.196:0/750845300 shutdown_connections 2024-04-28T13:31:58.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.859+0000 7f73ecc00640 1 --2- 172.21.15.196:0/750845300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e8163aa0 0x7f73e80b5b70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.859+0000 7f73ecc00640 1 --2- 172.21.15.196:0/750845300 >> v2:172.21.15.203:3300/0 conn(0x7f73e80b60b0 0x7f73e80d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.859+0000 7f73ecc00640 1 --2- 172.21.15.196:0/750845300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e815f680 0x7f73e80d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.859+0000 7f73ecc00640 1 -- 172.21.15.196:0/750845300 >> 172.21.15.196:0/750845300 conn(0x7f73e808eca0 msgr2=0x7f73e809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:58.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.859+0000 7f73ecc00640 1 -- 172.21.15.196:0/750845300 shutdown_connections 2024-04-28T13:31:58.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.859+0000 7f73ecc00640 1 -- 172.21.15.196:0/750845300 wait complete. 2024-04-28T13:31:58.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.860+0000 7f73ecc00640 1 Processor -- start 2024-04-28T13:31:58.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.860+0000 7f73ecc00640 1 -- start start 2024-04-28T13:31:58.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.861+0000 7f73ecc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e80b60b0 0x7f73e80cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.861+0000 7f73ecc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73e815f680 0x7f73e80d1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.861+0000 7f73ecc00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e8163aa0 0x7f73e80c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.861+0000 7f73ecc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73e80d55e0 con 0x7f73e80b60b0 2024-04-28T13:31:58.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.861+0000 7f73ecc00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f73e80d5460 con 0x7f73e815f680 2024-04-28T13:31:58.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.861+0000 7f73ecc00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f73e80d5760 con 0x7f73e8163aa0 2024-04-28T13:31:58.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.862+0000 7f73e5e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e80b60b0 0x7f73e80cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.862+0000 7f73e5e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e80b60b0 0x7f73e80cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52452/0 (socket says 172.21.15.196:52452) 2024-04-28T13:31:58.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.862+0000 7f73e5e00640 1 -- 172.21.15.196:0/2773463219 learned_addr learned my addr 172.21.15.196:0/2773463219 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:58.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.862+0000 7f73e6800640 1 --2- 172.21.15.196:0/2773463219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e8163aa0 0x7f73e80c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.862+0000 7f73e5400640 1 --2- 172.21.15.196:0/2773463219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73e815f680 0x7f73e80d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.862+0000 7f73e5e00640 1 -- 172.21.15.196:0/2773463219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e8163aa0 msgr2=0x7f73e80c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:58.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.862+0000 7f73e5e00640 1 --2- 172.21.15.196:0/2773463219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e8163aa0 0x7f73e80c8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.862+0000 7f73e5e00640 1 -- 172.21.15.196:0/2773463219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73e815f680 msgr2=0x7f73e80d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:58.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.862+0000 7f73e5e00640 1 --2- 172.21.15.196:0/2773463219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73e815f680 0x7f73e80d1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:58.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.862+0000 7f73e5e00640 1 -- 172.21.15.196:0/2773463219 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e80d08f0 con 0x7f73e80b60b0 2024-04-28T13:31:58.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.862+0000 7f73e5e00640 1 --2- 172.21.15.196:0/2773463219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e80b60b0 0x7f73e80cabc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f73d800c7f0 tx=0x7f73d800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:58.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.863+0000 7f73de000640 1 -- 172.21.15.196:0/2773463219 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8015070 con 0x7f73e80b60b0 2024-04-28T13:31:58.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.863+0000 7f73de000640 1 -- 172.21.15.196:0/2773463219 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73d8007a30 con 0x7f73e80b60b0 2024-04-28T13:31:58.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.863+0000 7f73de000640 1 -- 172.21.15.196:0/2773463219 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8007e00 con 0x7f73e80b60b0 2024-04-28T13:31:58.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.863+0000 7f73ecc00640 1 -- 172.21.15.196:0/2773463219 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e80cc990 con 0x7f73e80b60b0 2024-04-28T13:31:58.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.863+0000 7f73ecc00640 1 -- 172.21.15.196:0/2773463219 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e80d0b90 con 0x7f73e80b60b0 2024-04-28T13:31:58.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.864+0000 7f73ecc00640 1 -- 172.21.15.196:0/2773463219 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e806a290 con 0x7f73e80b60b0 2024-04-28T13:31:58.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.865+0000 7f73de000640 1 -- 172.21.15.196:0/2773463219 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73d8018070 con 0x7f73e80b60b0 2024-04-28T13:31:58.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.866+0000 7f73de000640 1 --2- 172.21.15.196:0/2773463219 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73c806c8f0 0x7f73c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:58.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.866+0000 7f73de000640 1 -- 172.21.15.196:0/2773463219 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f73d808bb20 con 0x7f73e80b60b0 2024-04-28T13:31:58.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.867+0000 7f73e5400640 1 --2- 172.21.15.196:0/2773463219 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73c806c8f0 0x7f73c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:58.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.868+0000 7f73e5400640 1 --2- 172.21.15.196:0/2773463219 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73c806c8f0 0x7f73c806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f73d4034000 tx=0x7f73d402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:58.870 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:58.871+0000 7f73de000640 1 -- 172.21.15.196:0/2773463219 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f73d8059f40 con 0x7f73e80b60b0 2024-04-28T13:31:59.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.071+0000 7f73ecc00640 1 -- 172.21.15.196:0/2773463219 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f73e80b4cd0 con 0x7f73e80b60b0 2024-04-28T13:31:59.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.072+0000 7f73de000640 1 -- 172.21.15.196:0/2773463219 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f73d8059840 con 0x7f73e80b60b0 2024-04-28T13:31:59.072 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:31:59.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.075+0000 7f73ecc00640 1 -- 172.21.15.196:0/2773463219 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73c806c8f0 msgr2=0x7f73c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:59.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.075+0000 7f73ecc00640 1 --2- 172.21.15.196:0/2773463219 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73c806c8f0 0x7f73c806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f73d4034000 tx=0x7f73d402e000 comp rx=0 tx=0).stop 2024-04-28T13:31:59.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.075+0000 7f73ecc00640 1 -- 172.21.15.196:0/2773463219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e80b60b0 msgr2=0x7f73e80cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:59.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.075+0000 7f73ecc00640 1 --2- 172.21.15.196:0/2773463219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e80b60b0 0x7f73e80cabc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f73d800c7f0 tx=0x7f73d800cbf0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.075+0000 7f73ecc00640 1 -- 172.21.15.196:0/2773463219 shutdown_connections 2024-04-28T13:31:59.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.075+0000 7f73ecc00640 1 --2- 172.21.15.196:0/2773463219 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73c806c8f0 0x7f73c806edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.075+0000 7f73ecc00640 1 --2- 172.21.15.196:0/2773463219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e8163aa0 0x7f73e80c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.075+0000 7f73ecc00640 1 --2- 172.21.15.196:0/2773463219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73e815f680 0x7f73e80d1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.075+0000 7f73ecc00640 1 --2- 172.21.15.196:0/2773463219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e80b60b0 0x7f73e80cabc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.075+0000 7f73ecc00640 1 -- 172.21.15.196:0/2773463219 >> 172.21.15.196:0/2773463219 conn(0x7f73e808eca0 msgr2=0x7f73e80d49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:59.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.075+0000 7f73ecc00640 1 -- 172.21.15.196:0/2773463219 shutdown_connections 2024-04-28T13:31:59.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.075+0000 7f73ecc00640 1 -- 172.21.15.196:0/2773463219 wait complete. 2024-04-28T13:31:59.094 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:31:59.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.295+0000 7f3f7f000640 1 Processor -- start 2024-04-28T13:31:59.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.296+0000 7f3f7f000640 1 -- start start 2024-04-28T13:31:59.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.297+0000 7f3f7f000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3f780d1b80 0x7f3f780d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:59.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.297+0000 7f3f7f000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3f780cfda0 0x7f3f780d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:59.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.297+0000 7f3f7f000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f780d0220 con 0x7f3f780d3fa0 2024-04-28T13:31:59.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.297+0000 7f3f7f000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3f780d0500 con 0x7f3f780d1b80 2024-04-28T13:31:59.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.297+0000 7f3f7f000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3f780d07e0 con 0x7f3f780cfda0 2024-04-28T13:31:59.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.297+0000 7f3f77e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3f780d1b80 0x7f3f780d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:59.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.298+0000 7f3f7d000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f3f780d3fa0 0x7f3f780b60a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:34366/0 (socket says 172.21.15.196:34366) 2024-04-28T13:31:59.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.298+0000 7f3f7d000640 1 -- 172.21.15.196:0/1489565066 learned_addr learned my addr 172.21.15.196:0/1489565066 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:59.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.298+0000 7f3f77e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3f780d1b80 0x7f3f780d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52468/0 (socket says 172.21.15.196:52468) 2024-04-28T13:31:59.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.298+0000 7f3f77400640 1 --2- 172.21.15.196:0/1489565066 >> v2:172.21.15.203:3300/0 conn(0x7f3f780cfda0 0x7f3f780d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:59.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.298+0000 7f3f77e00640 1 -- 172.21.15.196:0/1489565066 >> v2:172.21.15.203:3300/0 conn(0x7f3f780cfda0 msgr2=0x7f3f780d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:59.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.298+0000 7f3f77e00640 1 --2- 172.21.15.196:0/1489565066 >> v2:172.21.15.203:3300/0 conn(0x7f3f780cfda0 0x7f3f780d3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.298+0000 7f3f77e00640 1 -- 172.21.15.196:0/1489565066 >> v1:172.21.15.196:6789/0 conn(0x7f3f780d3fa0 legacy=0x7f3f780b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:59.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.298+0000 7f3f77e00640 1 -- 172.21.15.196:0/1489565066 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f780d0ac0 con 0x7f3f780d1b80 2024-04-28T13:31:59.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.298+0000 7f3f76a00640 1 -- 172.21.15.196:0/1489565066 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4177831490 0 0) 0x7f3f780d0220 con 0x7f3f780d3fa0 2024-04-28T13:31:59.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.298+0000 7f3f77400640 1 --2- 172.21.15.196:0/1489565066 >> v2:172.21.15.203:3300/0 conn(0x7f3f780cfda0 0x7f3f780d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:31:59.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.299+0000 7f3f77e00640 1 --2- 172.21.15.196:0/1489565066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780d1b80 0x7f3f780d1f60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3f7808ddb0 tx=0x7f3f6802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f94a0f64f566574 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:59.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.299+0000 7f3f76a00640 1 -- 172.21.15.196:0/1489565066 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68037070 con 0x7f3f780d1b80 2024-04-28T13:31:59.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.299+0000 7f3f76a00640 1 -- 172.21.15.196:0/1489565066 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f68032440 con 0x7f3f780d1b80 2024-04-28T13:31:59.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.299+0000 7f3f76a00640 1 -- 172.21.15.196:0/1489565066 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68032810 con 0x7f3f780d1b80 2024-04-28T13:31:59.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.300+0000 7f3f7f000640 1 -- 172.21.15.196:0/1489565066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780d1b80 msgr2=0x7f3f780d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:59.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.300+0000 7f3f7f000640 1 --2- 172.21.15.196:0/1489565066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780d1b80 0x7f3f780d1f60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3f7808ddb0 tx=0x7f3f6802a1a0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.300+0000 7f3f7f000640 1 -- 172.21.15.196:0/1489565066 shutdown_connections 2024-04-28T13:31:59.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.300+0000 7f3f7f000640 1 --2- 172.21.15.196:0/1489565066 >> v2:172.21.15.203:3300/0 conn(0x7f3f780cfda0 0x7f3f780d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.300+0000 7f3f7f000640 1 --2- 172.21.15.196:0/1489565066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780d1b80 0x7f3f780d1f60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.300+0000 7f3f7f000640 1 -- 172.21.15.196:0/1489565066 >> 172.21.15.196:0/1489565066 conn(0x7f3f7809e540 msgr2=0x7f3f780d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:59.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.300+0000 7f3f7f000640 1 -- 172.21.15.196:0/1489565066 shutdown_connections 2024-04-28T13:31:59.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.300+0000 7f3f7f000640 1 -- 172.21.15.196:0/1489565066 wait complete. 2024-04-28T13:31:59.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.302+0000 7f3f7f000640 1 Processor -- start 2024-04-28T13:31:59.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.302+0000 7f3f7f000640 1 -- start start 2024-04-28T13:31:59.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.303+0000 7f3f7f000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780cfda0 0x7f3f781f60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:59.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.303+0000 7f3f7f000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f780d1b80 0x7f3f78153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:59.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.303+0000 7f3f7f000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f780d3fa0 0x7f3f781f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:59.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.303+0000 7f3f7f000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f780b6930 con 0x7f3f780cfda0 2024-04-28T13:31:59.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.303+0000 7f3f7f000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3f780b67b0 con 0x7f3f780d1b80 2024-04-28T13:31:59.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.303+0000 7f3f7f000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3f780b6ab0 con 0x7f3f780d3fa0 2024-04-28T13:31:59.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.303+0000 7f3f77e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780cfda0 0x7f3f781f60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:59.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.303+0000 7f3f77e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780cfda0 0x7f3f781f60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52480/0 (socket says 172.21.15.196:52480) 2024-04-28T13:31:59.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.303+0000 7f3f77e00640 1 -- 172.21.15.196:0/2344545348 learned_addr learned my addr 172.21.15.196:0/2344545348 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:59.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.303+0000 7f3f77e00640 1 -- 172.21.15.196:0/2344545348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f780d3fa0 msgr2=0x7f3f781f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:59.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.303+0000 7f3f77e00640 1 --2- 172.21.15.196:0/2344545348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f780d3fa0 0x7f3f781f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.303+0000 7f3f77e00640 1 -- 172.21.15.196:0/2344545348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f780d1b80 msgr2=0x7f3f78153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:59.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.303+0000 7f3f77e00640 1 --2- 172.21.15.196:0/2344545348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f780d1b80 0x7f3f78153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.303+0000 7f3f77e00640 1 -- 172.21.15.196:0/2344545348 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7815b9e0 con 0x7f3f780cfda0 2024-04-28T13:31:59.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.304+0000 7f3f77e00640 1 --2- 172.21.15.196:0/2344545348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780cfda0 0x7f3f781f60f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3f68000c00 tx=0x7f3f680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:59.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.304+0000 7f3f6fe00640 1 -- 172.21.15.196:0/2344545348 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68037040 con 0x7f3f780cfda0 2024-04-28T13:31:59.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.304+0000 7f3f6fe00640 1 -- 172.21.15.196:0/2344545348 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f68042070 con 0x7f3f780cfda0 2024-04-28T13:31:59.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.304+0000 7f3f7f000640 1 -- 172.21.15.196:0/2344545348 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f780cc370 con 0x7f3f780cfda0 2024-04-28T13:31:59.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.304+0000 7f3f6fe00640 1 -- 172.21.15.196:0/2344545348 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68030bb0 con 0x7f3f780cfda0 2024-04-28T13:31:59.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.304+0000 7f3f7f000640 1 -- 172.21.15.196:0/2344545348 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f780cc930 con 0x7f3f780cfda0 2024-04-28T13:31:59.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.305+0000 7f3f7f000640 1 -- 172.21.15.196:0/2344545348 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f7806a290 con 0x7f3f780cfda0 2024-04-28T13:31:59.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.306+0000 7f3f6fe00640 1 -- 172.21.15.196:0/2344545348 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f68047460 con 0x7f3f780cfda0 2024-04-28T13:31:59.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.307+0000 7f3f6fe00640 1 --2- 172.21.15.196:0/2344545348 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3f5406c920 0x7f3f5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:59.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.307+0000 7f3f6fe00640 1 -- 172.21.15.196:0/2344545348 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3f680ac800 con 0x7f3f780cfda0 2024-04-28T13:31:59.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.308+0000 7f3f77400640 1 --2- 172.21.15.196:0/2344545348 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3f5406c920 0x7f3f5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:59.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.309+0000 7f3f77400640 1 --2- 172.21.15.196:0/2344545348 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3f5406c920 0x7f3f5406ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3f60006fd0 tx=0x7f3f60008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:59.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.312+0000 7f3f6fe00640 1 -- 172.21.15.196:0/2344545348 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f68032e20 con 0x7f3f780cfda0 2024-04-28T13:31:59.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.517+0000 7f3f7f000640 1 -- 172.21.15.196:0/2344545348 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f780d27b0 con 0x7f3f780cfda0 2024-04-28T13:31:59.520 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.520+0000 7f3f6fe00640 1 -- 172.21.15.196:0/2344545348 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f3f68045360 con 0x7f3f780cfda0 2024-04-28T13:31:59.520 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:31:59.521 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":20,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:22.624564+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:31:59.523 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.523+0000 7f3f7f000640 1 -- 172.21.15.196:0/2344545348 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3f5406c920 msgr2=0x7f3f5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:59.523 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.523+0000 7f3f7f000640 1 --2- 172.21.15.196:0/2344545348 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3f5406c920 0x7f3f5406ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3f60006fd0 tx=0x7f3f60008040 comp rx=0 tx=0).stop 2024-04-28T13:31:59.523 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.523+0000 7f3f7f000640 1 -- 172.21.15.196:0/2344545348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780cfda0 msgr2=0x7f3f781f60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:59.523 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.523+0000 7f3f7f000640 1 --2- 172.21.15.196:0/2344545348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780cfda0 0x7f3f781f60f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3f68000c00 tx=0x7f3f680046c0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.523 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.523+0000 7f3f7f000640 1 -- 172.21.15.196:0/2344545348 shutdown_connections 2024-04-28T13:31:59.523 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.523+0000 7f3f7f000640 1 --2- 172.21.15.196:0/2344545348 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3f5406c920 0x7f3f5406ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.523 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.523+0000 7f3f7f000640 1 --2- 172.21.15.196:0/2344545348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f780d3fa0 0x7f3f781f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.524 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.523+0000 7f3f7f000640 1 --2- 172.21.15.196:0/2344545348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f780d1b80 0x7f3f78153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.524 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.523+0000 7f3f7f000640 1 --2- 172.21.15.196:0/2344545348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f780cfda0 0x7f3f781f60f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.524 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.523+0000 7f3f7f000640 1 -- 172.21.15.196:0/2344545348 >> 172.21.15.196:0/2344545348 conn(0x7f3f7809e540 msgr2=0x7f3f780b9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:59.524 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.523+0000 7f3f7f000640 1 -- 172.21.15.196:0/2344545348 shutdown_connections 2024-04-28T13:31:59.524 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.523+0000 7f3f7f000640 1 -- 172.21.15.196:0/2344545348 wait complete. 2024-04-28T13:31:59.542 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:31:19.280975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:31:20.037066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:31:59.542 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:31:59.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.744+0000 7fd62e400640 1 Processor -- start 2024-04-28T13:31:59.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.745+0000 7fd62e400640 1 -- start start 2024-04-28T13:31:59.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.746+0000 7fd62e400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd6280d3c10 0x7fd6280b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:59.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.746+0000 7fd62e400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6280cfb80 con 0x7fd6280d3830 2024-04-28T13:31:59.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.746+0000 7fd62e400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6280d2ee0 con 0x7fd6280b8370 2024-04-28T13:31:59.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.746+0000 7fd62e400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd6280d31c0 con 0x7fd6280d3c10 2024-04-28T13:31:59.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.746+0000 7fd62d000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fd6280d3830 0x7fd6280d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:34374/0 (socket says 172.21.15.196:34374) 2024-04-28T13:31:59.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.746+0000 7fd62d000640 1 -- 172.21.15.196:0/4159179206 learned_addr learned my addr 172.21.15.196:0/4159179206 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:31:59.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.746+0000 7fd627e00640 1 --2- 172.21.15.196:0/4159179206 >> v2:172.21.15.196:3300/0 conn(0x7fd6280d3c10 0x7fd6280b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:59.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.747+0000 7fd627e00640 1 -- 172.21.15.196:0/4159179206 >> v1:172.21.15.203:6789/0 conn(0x7fd6280b8370 legacy=0x7fd6280d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:31:59.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.747+0000 7fd627e00640 1 -- 172.21.15.196:0/4159179206 >> v1:172.21.15.196:6789/0 conn(0x7fd6280d3830 legacy=0x7fd6280d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:59.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.747+0000 7fd627e00640 1 -- 172.21.15.196:0/4159179206 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6280cfe00 con 0x7fd6280d3c10 2024-04-28T13:31:59.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.748+0000 7fd627e00640 1 --2- 172.21.15.196:0/4159179206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6280d3c10 0x7fd6280b7e30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd610009d90 tx=0x7fd61002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1cd5879f9b676a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:59.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.748+0000 7fd627400640 1 -- 172.21.15.196:0/4159179206 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610007740 con 0x7fd6280d3c10 2024-04-28T13:31:59.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.748+0000 7fd627400640 1 -- 172.21.15.196:0/4159179206 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6100078e0 con 0x7fd6280d3c10 2024-04-28T13:31:59.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.748+0000 7fd627400640 1 -- 172.21.15.196:0/4159179206 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610007cb0 con 0x7fd6280d3c10 2024-04-28T13:31:59.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.749+0000 7fd62e400640 1 -- 172.21.15.196:0/4159179206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6280d3c10 msgr2=0x7fd6280b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:59.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.749+0000 7fd62e400640 1 --2- 172.21.15.196:0/4159179206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6280d3c10 0x7fd6280b7e30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd610009d90 tx=0x7fd61002a6a0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.749+0000 7fd62e400640 1 -- 172.21.15.196:0/4159179206 shutdown_connections 2024-04-28T13:31:59.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.749+0000 7fd62e400640 1 --2- 172.21.15.196:0/4159179206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6280d3c10 0x7fd6280b7e30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.749+0000 7fd62e400640 1 -- 172.21.15.196:0/4159179206 >> 172.21.15.196:0/4159179206 conn(0x7fd6280d4fe0 msgr2=0x7fd6280d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:59.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.749+0000 7fd62e400640 1 -- 172.21.15.196:0/4159179206 shutdown_connections 2024-04-28T13:31:59.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.749+0000 7fd62e400640 1 -- 172.21.15.196:0/4159179206 wait complete. 2024-04-28T13:31:59.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.751+0000 7fd62e400640 1 Processor -- start 2024-04-28T13:31:59.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.751+0000 7fd62e400640 1 -- start start 2024-04-28T13:31:59.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.752+0000 7fd62e400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6280b8370 0x7fd62814c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:59.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.752+0000 7fd62e400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6280d3830 0x7fd62815eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:59.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.752+0000 7fd62e400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6280d3c10 0x7fd62815efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:59.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.752+0000 7fd62e400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6280d2bb0 con 0x7fd6280b8370 2024-04-28T13:31:59.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.752+0000 7fd62e400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd6280d5d90 con 0x7fd6280d3830 2024-04-28T13:31:59.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.752+0000 7fd62e400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd6280d0d10 con 0x7fd6280d3c10 2024-04-28T13:31:59.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.752+0000 7fd62d000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6280b8370 0x7fd62814c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:59.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.752+0000 7fd62d000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6280b8370 0x7fd62814c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52500/0 (socket says 172.21.15.196:52500) 2024-04-28T13:31:59.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.752+0000 7fd62d000640 1 -- 172.21.15.196:0/3757444641 learned_addr learned my addr 172.21.15.196:0/3757444641 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:31:59.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.752+0000 7fd62d000640 1 -- 172.21.15.196:0/3757444641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6280d3c10 msgr2=0x7fd62815efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:59.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.752+0000 7fd62d000640 1 --2- 172.21.15.196:0/3757444641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6280d3c10 0x7fd62815efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.752+0000 7fd62d000640 1 -- 172.21.15.196:0/3757444641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6280d3830 msgr2=0x7fd62815eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:59.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.752+0000 7fd62d000640 1 --2- 172.21.15.196:0/3757444641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6280d3830 0x7fd62815eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.752+0000 7fd62d000640 1 -- 172.21.15.196:0/3757444641 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6280ced10 con 0x7fd6280b8370 2024-04-28T13:31:59.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.753+0000 7fd62d000640 1 --2- 172.21.15.196:0/3757444641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6280b8370 0x7fd62814c460 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd61800c430 tx=0x7fd61800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:59.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.753+0000 7fd624c00640 1 -- 172.21.15.196:0/3757444641 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618017070 con 0x7fd6280b8370 2024-04-28T13:31:59.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.753+0000 7fd624c00640 1 -- 172.21.15.196:0/3757444641 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd618005ce0 con 0x7fd6280b8370 2024-04-28T13:31:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.753+0000 7fd624c00640 1 -- 172.21.15.196:0/3757444641 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6180072f0 con 0x7fd6280b8370 2024-04-28T13:31:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.753+0000 7fd62e400640 1 -- 172.21.15.196:0/3757444641 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6281f6140 con 0x7fd6280b8370 2024-04-28T13:31:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.753+0000 7fd62e400640 1 -- 172.21.15.196:0/3757444641 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd62815a7e0 con 0x7fd6280b8370 2024-04-28T13:31:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.754+0000 7fd62e400640 1 -- 172.21.15.196:0/3757444641 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd628050730 con 0x7fd6280b8370 2024-04-28T13:31:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.755+0000 7fd624c00640 1 -- 172.21.15.196:0/3757444641 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd61801a070 con 0x7fd6280b8370 2024-04-28T13:31:59.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.756+0000 7fd624c00640 1 --2- 172.21.15.196:0/3757444641 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd60406c8f0 0x7fd60406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:31:59.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.756+0000 7fd624c00640 1 -- 172.21.15.196:0/3757444641 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd61808af70 con 0x7fd6280b8370 2024-04-28T13:31:59.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.757+0000 7fd627e00640 1 --2- 172.21.15.196:0/3757444641 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd60406c8f0 0x7fd60406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:31:59.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.758+0000 7fd627e00640 1 --2- 172.21.15.196:0/3757444641 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd60406c8f0 0x7fd60406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd610034000 tx=0x7fd61002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:31:59.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.760+0000 7fd624c00640 1 -- 172.21.15.196:0/3757444641 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd6280cfb80 con 0x7fd6280b8370 2024-04-28T13:31:59.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.970+0000 7fd62e400640 1 -- 172.21.15.196:0/3757444641 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6280d17d0 con 0x7fd6280b8370 2024-04-28T13:31:59.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.971+0000 7fd624c00640 1 -- 172.21.15.196:0/3757444641 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fd6180593a0 con 0x7fd6280b8370 2024-04-28T13:31:59.971 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:31:59.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.974+0000 7fd62e400640 1 -- 172.21.15.196:0/3757444641 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd60406c8f0 msgr2=0x7fd60406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:59.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.974+0000 7fd62e400640 1 --2- 172.21.15.196:0/3757444641 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd60406c8f0 0x7fd60406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd610034000 tx=0x7fd61002e000 comp rx=0 tx=0).stop 2024-04-28T13:31:59.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.974+0000 7fd62e400640 1 -- 172.21.15.196:0/3757444641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6280b8370 msgr2=0x7fd62814c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:31:59.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.974+0000 7fd62e400640 1 --2- 172.21.15.196:0/3757444641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6280b8370 0x7fd62814c460 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd61800c430 tx=0x7fd61800c8f0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.974+0000 7fd62e400640 1 -- 172.21.15.196:0/3757444641 shutdown_connections 2024-04-28T13:31:59.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.974+0000 7fd62e400640 1 --2- 172.21.15.196:0/3757444641 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd60406c8f0 0x7fd60406edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.974+0000 7fd62e400640 1 --2- 172.21.15.196:0/3757444641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6280d3c10 0x7fd62815efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.974+0000 7fd62e400640 1 --2- 172.21.15.196:0/3757444641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6280d3830 0x7fd62815eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.974+0000 7fd62e400640 1 --2- 172.21.15.196:0/3757444641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6280b8370 0x7fd62814c460 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:31:59.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.974+0000 7fd62e400640 1 -- 172.21.15.196:0/3757444641 >> 172.21.15.196:0/3757444641 conn(0x7fd6280d4fe0 msgr2=0x7fd6280d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:31:59.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.974+0000 7fd62e400640 1 -- 172.21.15.196:0/3757444641 shutdown_connections 2024-04-28T13:31:59.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:31:59.974+0000 7fd62e400640 1 -- 172.21.15.196:0/3757444641 wait complete. 2024-04-28T13:31:59.994 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:32:00.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.202+0000 7f8921c00640 1 Processor -- start 2024-04-28T13:32:00.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.202+0000 7f8921c00640 1 -- start start 2024-04-28T13:32:00.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.203+0000 7f8921c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f891c0b04c0 0x7f891c0af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:00.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.203+0000 7f8921c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f891c1607d0 con 0x7f891c145960 2024-04-28T13:32:00.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.203+0000 7f8921c00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f891c0aea50 con 0x7f891c15fa10 2024-04-28T13:32:00.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.203+0000 7f8921c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f891c1605b0 con 0x7f891c0b04c0 2024-04-28T13:32:00.204 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.204+0000 7f891aa00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f891c145960 0x7f891c0afdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:34388/0 (socket says 172.21.15.196:34388) 2024-04-28T13:32:00.204 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.204+0000 7f891aa00640 1 -- 172.21.15.196:0/3206194935 learned_addr learned my addr 172.21.15.196:0/3206194935 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:00.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.204+0000 7f8913e00640 1 --2- 172.21.15.196:0/3206194935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f891c0b04c0 0x7f891c0af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:00.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.204+0000 7f891a000640 1 -- 172.21.15.196:0/3206194935 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3846437419 0 0) 0x7f891c1607d0 con 0x7f891c145960 2024-04-28T13:32:00.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.204+0000 7f891a000640 1 -- 172.21.15.196:0/3206194935 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88fc003520 con 0x7f891c145960 2024-04-28T13:32:00.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891a000640 1 -- 172.21.15.196:0/3206194935 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2744122390 0 0) 0x7f88fc003520 con 0x7f891c145960 2024-04-28T13:32:00.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891a000640 1 -- 172.21.15.196:0/3206194935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f891c0b04c0 msgr2=0x7f891c0af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:00.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891a000640 1 --2- 172.21.15.196:0/3206194935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f891c0b04c0 0x7f891c0af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891a000640 1 -- 172.21.15.196:0/3206194935 >> v1:172.21.15.203:6789/0 conn(0x7f891c15fa10 legacy=0x7f891c15fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:00.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891a000640 1 -- 172.21.15.196:0/3206194935 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891c0aed30 con 0x7f891c145960 2024-04-28T13:32:00.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891a000640 1 -- 172.21.15.196:0/3206194935 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f890c001670 con 0x7f891c145960 2024-04-28T13:32:00.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891a000640 1 -- 172.21.15.196:0/3206194935 >> v1:172.21.15.196:6789/0 conn(0x7f891c145960 legacy=0x7f891c0afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:00.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891a000640 1 --2- 172.21.15.196:0/3206194935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f891c0b04c0 0x7f88fc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:00.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891a000640 1 --2- 172.21.15.196:0/3206194935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88fc003f60 0x7f88fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:00.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891a000640 1 --2- 172.21.15.196:0/3206194935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88fc004d40 0x7f88fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:00.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891a000640 1 -- 172.21.15.196:0/3206194935 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f891c1605b0 con 0x7f88fc004d40 2024-04-28T13:32:00.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891a000640 1 -- 172.21.15.196:0/3206194935 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc00baa0 con 0x7f88fc003f60 2024-04-28T13:32:00.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891a000640 1 -- 172.21.15.196:0/3206194935 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f88fc00bca0 con 0x7f891c0b04c0 2024-04-28T13:32:00.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891b400640 1 --2- 172.21.15.196:0/3206194935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88fc004d40 0x7f88fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:00.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891aa00640 1 --2- 172.21.15.196:0/3206194935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f891c0b04c0 0x7f88fc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:00.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f8913e00640 1 --2- 172.21.15.196:0/3206194935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88fc003f60 0x7f88fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:00.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891b400640 1 -- 172.21.15.196:0/3206194935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f891c0b04c0 msgr2=0x7f88fc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:00.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891b400640 1 --2- 172.21.15.196:0/3206194935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f891c0b04c0 0x7f88fc0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891b400640 1 -- 172.21.15.196:0/3206194935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88fc003f60 msgr2=0x7f88fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:00.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.205+0000 7f891b400640 1 --2- 172.21.15.196:0/3206194935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88fc003f60 0x7f88fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.206+0000 7f891b400640 1 -- 172.21.15.196:0/3206194935 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88fc003520 con 0x7f88fc004d40 2024-04-28T13:32:00.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.206+0000 7f8913e00640 1 --2- 172.21.15.196:0/3206194935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88fc003f60 0x7f88fc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:00.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.207+0000 7f891b400640 1 --2- 172.21.15.196:0/3206194935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88fc004d40 0x7f88fc0051c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8914001e50 tx=0x7f88fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eededfda98fd4bd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:00.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.207+0000 7f891a000640 1 -- 172.21.15.196:0/3206194935 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914001a70 con 0x7f88fc004d40 2024-04-28T13:32:00.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.207+0000 7f891a000640 1 -- 172.21.15.196:0/3206194935 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8914003970 con 0x7f88fc004d40 2024-04-28T13:32:00.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.207+0000 7f891a000640 1 -- 172.21.15.196:0/3206194935 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914003d60 con 0x7f88fc004d40 2024-04-28T13:32:00.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.207+0000 7f8921c00640 1 -- 172.21.15.196:0/3206194935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88fc004d40 msgr2=0x7f88fc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:00.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.207+0000 7f8921c00640 1 --2- 172.21.15.196:0/3206194935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88fc004d40 0x7f88fc0051c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8914001e50 tx=0x7f88fc005830 comp rx=0 tx=0).stop 2024-04-28T13:32:00.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.207+0000 7f891b400640 1 -- 172.21.15.196:0/3206194935 reap_dead start 2024-04-28T13:32:00.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.207+0000 7f8921c00640 1 -- 172.21.15.196:0/3206194935 shutdown_connections 2024-04-28T13:32:00.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.207+0000 7f8921c00640 1 -- 172.21.15.196:0/3206194935 >> 172.21.15.196:0/3206194935 conn(0x7f891c0cd8a0 msgr2=0x7f891c0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:00.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.208+0000 7f8921c00640 1 -- 172.21.15.196:0/3206194935 shutdown_connections 2024-04-28T13:32:00.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.208+0000 7f8921c00640 1 -- 172.21.15.196:0/3206194935 wait complete. 2024-04-28T13:32:00.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.209+0000 7f8921c00640 1 Processor -- start 2024-04-28T13:32:00.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.209+0000 7f8921c00640 1 -- start start 2024-04-28T13:32:00.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.210+0000 7f8921c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f891c13afb0 0x7f891c13fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:00.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.210+0000 7f8921c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f891c13b390 0x7f891c1402f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:00.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.210+0000 7f8921c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f891c15e600 0x7f891c140830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:00.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.210+0000 7f8921c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f891c0af6b0 con 0x7f891c13afb0 2024-04-28T13:32:00.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.210+0000 7f8921c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f891c0af530 con 0x7f891c15e600 2024-04-28T13:32:00.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.210+0000 7f8921c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f891c0af830 con 0x7f891c13b390 2024-04-28T13:32:00.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.211+0000 7f891aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f891c13afb0 0x7f891c13fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:00.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.211+0000 7f891aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f891c13afb0 0x7f891c13fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52518/0 (socket says 172.21.15.196:52518) 2024-04-28T13:32:00.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.211+0000 7f891aa00640 1 -- 172.21.15.196:0/3894450333 learned_addr learned my addr 172.21.15.196:0/3894450333 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:00.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.211+0000 7f891b400640 1 --2- 172.21.15.196:0/3894450333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f891c15e600 0x7f891c140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:00.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.211+0000 7f8913e00640 1 --2- 172.21.15.196:0/3894450333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f891c13b390 0x7f891c1402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:00.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.211+0000 7f891aa00640 1 -- 172.21.15.196:0/3894450333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f891c13b390 msgr2=0x7f891c1402f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:00.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.211+0000 7f891aa00640 1 --2- 172.21.15.196:0/3894450333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f891c13b390 0x7f891c1402f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.211+0000 7f891aa00640 1 -- 172.21.15.196:0/3894450333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f891c15e600 msgr2=0x7f891c140830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:00.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.211+0000 7f891aa00640 1 --2- 172.21.15.196:0/3894450333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f891c15e600 0x7f891c140830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.211+0000 7f891aa00640 1 -- 172.21.15.196:0/3894450333 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891c0b6380 con 0x7f891c13afb0 2024-04-28T13:32:00.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.211+0000 7f891b400640 1 --2- 172.21.15.196:0/3894450333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f891c15e600 0x7f891c140830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:00.212 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.212+0000 7f891aa00640 1 --2- 172.21.15.196:0/3894450333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f891c13afb0 0x7f891c13fdb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f890c00db10 tx=0x7f890c001e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:00.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.212+0000 7f8912a00640 1 -- 172.21.15.196:0/3894450333 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c019070 con 0x7f891c13afb0 2024-04-28T13:32:00.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.212+0000 7f8912a00640 1 -- 172.21.15.196:0/3894450333 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f890c004d10 con 0x7f891c13afb0 2024-04-28T13:32:00.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.212+0000 7f8912a00640 1 -- 172.21.15.196:0/3894450333 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c0072f0 con 0x7f891c13afb0 2024-04-28T13:32:00.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.212+0000 7f8921c00640 1 -- 172.21.15.196:0/3894450333 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891c0b6660 con 0x7f891c13afb0 2024-04-28T13:32:00.214 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.212+0000 7f8921c00640 1 -- 172.21.15.196:0/3894450333 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f891c13bb60 con 0x7f891c13afb0 2024-04-28T13:32:00.214 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.213+0000 7f8921c00640 1 -- 172.21.15.196:0/3894450333 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f891c06a290 con 0x7f891c13afb0 2024-04-28T13:32:00.214 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.214+0000 7f8912a00640 1 -- 172.21.15.196:0/3894450333 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f890c007490 con 0x7f891c13afb0 2024-04-28T13:32:00.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.216+0000 7f8912a00640 1 --2- 172.21.15.196:0/3894450333 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88f806c8f0 0x7f88f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:00.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.216+0000 7f8912a00640 1 -- 172.21.15.196:0/3894450333 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f890c08b790 con 0x7f891c13afb0 2024-04-28T13:32:00.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.216+0000 7f8913e00640 1 --2- 172.21.15.196:0/3894450333 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88f806c8f0 0x7f88f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:00.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.218+0000 7f8913e00640 1 --2- 172.21.15.196:0/3894450333 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88f806c8f0 0x7f88f806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f891c073fe0 tx=0x7f8904008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:00.220 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.220+0000 7f8912a00640 1 -- 172.21.15.196:0/3894450333 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f890c059b50 con 0x7f891c13afb0 2024-04-28T13:32:00.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.422+0000 7f8921c00640 1 -- 172.21.15.196:0/3894450333 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f891c1629a0 con 0x7f891c13afb0 2024-04-28T13:32:00.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.423+0000 7f8912a00640 1 -- 172.21.15.196:0/3894450333 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f890c059450 con 0x7f891c13afb0 2024-04-28T13:32:00.423 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:32:00.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.425+0000 7f8921c00640 1 -- 172.21.15.196:0/3894450333 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88f806c8f0 msgr2=0x7f88f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:00.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.425+0000 7f8921c00640 1 --2- 172.21.15.196:0/3894450333 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88f806c8f0 0x7f88f806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f891c073fe0 tx=0x7f8904008040 comp rx=0 tx=0).stop 2024-04-28T13:32:00.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.425+0000 7f8921c00640 1 -- 172.21.15.196:0/3894450333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f891c13afb0 msgr2=0x7f891c13fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:00.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.425+0000 7f8921c00640 1 --2- 172.21.15.196:0/3894450333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f891c13afb0 0x7f891c13fdb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f890c00db10 tx=0x7f890c001e00 comp rx=0 tx=0).stop 2024-04-28T13:32:00.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.425+0000 7f8921c00640 1 -- 172.21.15.196:0/3894450333 shutdown_connections 2024-04-28T13:32:00.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.425+0000 7f8921c00640 1 --2- 172.21.15.196:0/3894450333 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88f806c8f0 0x7f88f806edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.425+0000 7f8921c00640 1 --2- 172.21.15.196:0/3894450333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f891c15e600 0x7f891c140830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.425+0000 7f8921c00640 1 --2- 172.21.15.196:0/3894450333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f891c13b390 0x7f891c1402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.425+0000 7f8921c00640 1 --2- 172.21.15.196:0/3894450333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f891c13afb0 0x7f891c13fdb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.425+0000 7f8921c00640 1 -- 172.21.15.196:0/3894450333 >> 172.21.15.196:0/3894450333 conn(0x7f891c0cd8a0 msgr2=0x7f891c06f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:00.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.426+0000 7f8921c00640 1 -- 172.21.15.196:0/3894450333 shutdown_connections 2024-04-28T13:32:00.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.426+0000 7f8921c00640 1 -- 172.21.15.196:0/3894450333 wait complete. 2024-04-28T13:32:00.444 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:32:00.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.647+0000 7f543d200640 1 Processor -- start 2024-04-28T13:32:00.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.647+0000 7f543d200640 1 -- start start 2024-04-28T13:32:00.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.648+0000 7f543d200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f543806c9b0 0x7f543815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:00.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.648+0000 7f543d200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5438073b80 0x7f543815fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:00.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.648+0000 7f543d200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5438160360 con 0x7f543815f1c0 2024-04-28T13:32:00.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.648+0000 7f543d200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5438160640 con 0x7f543806c9b0 2024-04-28T13:32:00.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.648+0000 7f543d200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f54380730a0 con 0x7f5438073b80 2024-04-28T13:32:00.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.648+0000 7f5437400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f543815f1c0 0x7f543806c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:34402/0 (socket says 172.21.15.196:34402) 2024-04-28T13:32:00.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.648+0000 7f5437400640 1 -- 172.21.15.196:0/2233463349 learned_addr learned my addr 172.21.15.196:0/2233463349 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:00.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.648+0000 7f5436a00640 1 --2- 172.21.15.196:0/2233463349 >> v2:172.21.15.196:3300/0 conn(0x7f543806c9b0 0x7f543815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:00.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.649+0000 7f5436000640 1 -- 172.21.15.196:0/2233463349 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2102278075 0 0) 0x7f5438160360 con 0x7f543815f1c0 2024-04-28T13:32:00.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.649+0000 7f5436a00640 1 -- 172.21.15.196:0/2233463349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5438073b80 msgr2=0x7f543815fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:00.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.649+0000 7f5436a00640 1 --2- 172.21.15.196:0/2233463349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5438073b80 0x7f543815fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.649+0000 7f5436a00640 1 -- 172.21.15.196:0/2233463349 >> v1:172.21.15.196:6789/0 conn(0x7f543815f1c0 legacy=0x7f543806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:00.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.649+0000 7f5436a00640 1 -- 172.21.15.196:0/2233463349 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5438073380 con 0x7f543806c9b0 2024-04-28T13:32:00.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.650+0000 7f5436a00640 1 --2- 172.21.15.196:0/2233463349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f543806c9b0 0x7f543815f5a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5420009d90 tx=0x7f542002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e7f872353227bc server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:00.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.650+0000 7f5436000640 1 -- 172.21.15.196:0/2233463349 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5420031600 con 0x7f543806c9b0 2024-04-28T13:32:00.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.650+0000 7f5436000640 1 -- 172.21.15.196:0/2233463349 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54200317a0 con 0x7f543806c9b0 2024-04-28T13:32:00.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.650+0000 7f5436000640 1 -- 172.21.15.196:0/2233463349 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5420031b70 con 0x7f543806c9b0 2024-04-28T13:32:00.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.651+0000 7f543d200640 1 -- 172.21.15.196:0/2233463349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f543806c9b0 msgr2=0x7f543815f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:00.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.651+0000 7f543d200640 1 --2- 172.21.15.196:0/2233463349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f543806c9b0 0x7f543815f5a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5420009d90 tx=0x7f542002a5d0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.651+0000 7f543d200640 1 -- 172.21.15.196:0/2233463349 shutdown_connections 2024-04-28T13:32:00.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.651+0000 7f543d200640 1 --2- 172.21.15.196:0/2233463349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5438073b80 0x7f543815fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.651+0000 7f543d200640 1 --2- 172.21.15.196:0/2233463349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f543806c9b0 0x7f543815f5a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.651+0000 7f543d200640 1 -- 172.21.15.196:0/2233463349 >> 172.21.15.196:0/2233463349 conn(0x7f54380d1c50 msgr2=0x7f54380d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:00.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.651+0000 7f543d200640 1 -- 172.21.15.196:0/2233463349 shutdown_connections 2024-04-28T13:32:00.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.651+0000 7f543d200640 1 -- 172.21.15.196:0/2233463349 wait complete. 2024-04-28T13:32:00.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.653+0000 7f543d200640 1 Processor -- start 2024-04-28T13:32:00.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.653+0000 7f543d200640 1 -- start start 2024-04-28T13:32:00.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.654+0000 7f543d200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f543806c9b0 0x7f543815e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:00.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.654+0000 7f543d200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5438073b80 0x7f54380b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:00.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.654+0000 7f543d200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f543815f1c0 0x7f543806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:00.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.654+0000 7f543d200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54380ae8c0 con 0x7f543815f1c0 2024-04-28T13:32:00.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.654+0000 7f543d200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5438160020 con 0x7f543806c9b0 2024-04-28T13:32:00.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.654+0000 7f543d200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f54380aea40 con 0x7f5438073b80 2024-04-28T13:32:00.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.654+0000 7f5437e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f543815f1c0 0x7f543806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:00.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.654+0000 7f5437e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f543815f1c0 0x7f543806fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52538/0 (socket says 172.21.15.196:52538) 2024-04-28T13:32:00.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.654+0000 7f5437e00640 1 -- 172.21.15.196:0/354607483 learned_addr learned my addr 172.21.15.196:0/354607483 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:00.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.654+0000 7f5436a00640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5438073b80 0x7f54380b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:00.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.654+0000 7f5437400640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f543806c9b0 0x7f543815e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:00.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.655+0000 7f5437e00640 1 -- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5438073b80 msgr2=0x7f54380b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:00.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.655+0000 7f5437e00640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5438073b80 0x7f54380b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.655+0000 7f5437e00640 1 -- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f543806c9b0 msgr2=0x7f543815e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:00.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.655+0000 7f5437e00640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f543806c9b0 0x7f543815e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.655+0000 7f5437e00640 1 -- 172.21.15.196:0/354607483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54380b5d70 con 0x7f543815f1c0 2024-04-28T13:32:00.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.655+0000 7f5437400640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f543806c9b0 0x7f543815e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:00.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.655+0000 7f5436a00640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5438073b80 0x7f54380b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:00.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.655+0000 7f5437e00640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f543815f1c0 0x7f543806fe90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5430009fd0 tx=0x7f543000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:00.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.655+0000 7f542f400640 1 -- 172.21.15.196:0/354607483 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430011830 con 0x7f543815f1c0 2024-04-28T13:32:00.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.655+0000 7f542f400640 1 -- 172.21.15.196:0/354607483 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5430016070 con 0x7f543815f1c0 2024-04-28T13:32:00.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.655+0000 7f542f400640 1 -- 172.21.15.196:0/354607483 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430011bf0 con 0x7f543815f1c0 2024-04-28T13:32:00.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.655+0000 7f543d200640 1 -- 172.21.15.196:0/354607483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54380b6050 con 0x7f543815f1c0 2024-04-28T13:32:00.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.655+0000 7f543d200640 1 -- 172.21.15.196:0/354607483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54380b6610 con 0x7f543815f1c0 2024-04-28T13:32:00.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.656+0000 7f543d200640 1 -- 172.21.15.196:0/354607483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5438050730 con 0x7f543815f1c0 2024-04-28T13:32:00.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.658+0000 7f542f400640 1 -- 172.21.15.196:0/354607483 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f543001b460 con 0x7f543815f1c0 2024-04-28T13:32:00.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.659+0000 7f542f400640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f541406c8f0 0x7f541406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:00.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.659+0000 7f542f400640 1 -- 172.21.15.196:0/354607483 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f543008b8c0 con 0x7f543815f1c0 2024-04-28T13:32:00.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.659+0000 7f5437400640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f541406c8f0 0x7f541406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:00.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.661+0000 7f5437400640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f541406c8f0 0x7f541406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5428000f60 tx=0x7f5428009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:00.663 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.663+0000 7f542f400640 1 -- 172.21.15.196:0/354607483 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5430059ce0 con 0x7f543815f1c0 2024-04-28T13:32:00.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.864+0000 7f543d200640 1 -- 172.21.15.196:0/354607483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f54380cc790 con 0x7f543815f1c0 2024-04-28T13:32:00.870 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.870+0000 7f542f400640 1 -- 172.21.15.196:0/354607483 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f54300595e0 con 0x7f543815f1c0 2024-04-28T13:32:00.870 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:32:00.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.872+0000 7f543d200640 1 -- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f541406c8f0 msgr2=0x7f541406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:00.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.872+0000 7f543d200640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f541406c8f0 0x7f541406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5428000f60 tx=0x7f5428009040 comp rx=0 tx=0).stop 2024-04-28T13:32:00.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.872+0000 7f543d200640 1 -- 172.21.15.196:0/354607483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f543815f1c0 msgr2=0x7f543806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:00.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.872+0000 7f543d200640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f543815f1c0 0x7f543806fe90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5430009fd0 tx=0x7f543000b8e0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.872+0000 7f543d200640 1 -- 172.21.15.196:0/354607483 shutdown_connections 2024-04-28T13:32:00.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.872+0000 7f543d200640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f541406c8f0 0x7f541406edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.872+0000 7f543d200640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f543815f1c0 0x7f543806fe90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.872+0000 7f543d200640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5438073b80 0x7f54380b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.872+0000 7f543d200640 1 --2- 172.21.15.196:0/354607483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f543806c9b0 0x7f543815e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:00.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.872+0000 7f543d200640 1 -- 172.21.15.196:0/354607483 >> 172.21.15.196:0/354607483 conn(0x7f54380d1c50 msgr2=0x7f5438161ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:00.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.873+0000 7f543d200640 1 -- 172.21.15.196:0/354607483 shutdown_connections 2024-04-28T13:32:00.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:00.873+0000 7f543d200640 1 -- 172.21.15.196:0/354607483 wait complete. 2024-04-28T13:32:00.935 INFO:tasks.cephfs_test_runner:Starting test: test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) That a file with ceph.quiesce.block is not quiesced. 2024-04-28T13:32:00.935 INFO:tasks.cephfs_test_runner:test_quiesce_blocked (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-28T13:32:00.935 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:32:01.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.147+0000 7f96f7e00640 1 Processor -- start 2024-04-28T13:32:01.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.147+0000 7f96f7e00640 1 -- start start 2024-04-28T13:32:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.149+0000 7f96f7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f96f80d1760 0x7f96f80b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.149+0000 7f96f7e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f96f809ec80 0x7f96f806df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.149+0000 7f96f7e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96f815e380 con 0x7f96f80b5ee0 2024-04-28T13:32:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.149+0000 7f96f7e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f96f80b4b20 con 0x7f96f80d1760 2024-04-28T13:32:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.149+0000 7f96f7e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f96f80b3f30 con 0x7f96f809ec80 2024-04-28T13:32:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.149+0000 7f96ee000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f96f80d1760 0x7f96f80b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.149+0000 7f96f6a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f96f80b5ee0 0x7f96f80d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:34416/0 (socket says 172.21.15.196:34416) 2024-04-28T13:32:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.149+0000 7f96ee000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f96f80d1760 0x7f96f80b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52542/0 (socket says 172.21.15.196:52542) 2024-04-28T13:32:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.149+0000 7f96f6a00640 1 -- 172.21.15.196:0/3913496402 learned_addr learned my addr 172.21.15.196:0/3913496402 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.150+0000 7f96f7400640 1 --2- 172.21.15.196:0/3913496402 >> v2:172.21.15.203:3300/0 conn(0x7f96f809ec80 0x7f96f806df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:01.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.150+0000 7f96ee000640 1 -- 172.21.15.196:0/3913496402 >> v2:172.21.15.203:3300/0 conn(0x7f96f809ec80 msgr2=0x7f96f806df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:01.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.150+0000 7f96ee000640 1 --2- 172.21.15.196:0/3913496402 >> v2:172.21.15.203:3300/0 conn(0x7f96f809ec80 0x7f96f806df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.150+0000 7f96ee000640 1 -- 172.21.15.196:0/3913496402 >> v1:172.21.15.196:6789/0 conn(0x7f96f80b5ee0 legacy=0x7f96f80d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:01.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.150+0000 7f96ee000640 1 -- 172.21.15.196:0/3913496402 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f80b41f0 con 0x7f96f80d1760 2024-04-28T13:32:01.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.150+0000 7f96f6000640 1 -- 172.21.15.196:0/3913496402 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3798334520 0 0) 0x7f96f815e380 con 0x7f96f80b5ee0 2024-04-28T13:32:01.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.151+0000 7f96ee000640 1 --2- 172.21.15.196:0/3913496402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96f80d1760 0x7f96f80b4f40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f96f8074240 tx=0x7f96e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48958c4c0206c305 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:01.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.151+0000 7f96f6000640 1 -- 172.21.15.196:0/3913496402 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0037070 con 0x7f96f80d1760 2024-04-28T13:32:01.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.151+0000 7f96f6000640 1 -- 172.21.15.196:0/3913496402 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e0032440 con 0x7f96f80d1760 2024-04-28T13:32:01.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.151+0000 7f96f6000640 1 -- 172.21.15.196:0/3913496402 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0032810 con 0x7f96f80d1760 2024-04-28T13:32:01.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.152+0000 7f96f7e00640 1 -- 172.21.15.196:0/3913496402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96f80d1760 msgr2=0x7f96f80b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:01.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.152+0000 7f96f7e00640 1 --2- 172.21.15.196:0/3913496402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96f80d1760 0x7f96f80b4f40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f96f8074240 tx=0x7f96e002a1a0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.152+0000 7f96f7e00640 1 -- 172.21.15.196:0/3913496402 shutdown_connections 2024-04-28T13:32:01.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.152+0000 7f96f7e00640 1 --2- 172.21.15.196:0/3913496402 >> v2:172.21.15.203:3300/0 conn(0x7f96f809ec80 0x7f96f806df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.152+0000 7f96f7e00640 1 --2- 172.21.15.196:0/3913496402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96f80d1760 0x7f96f80b4f40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.152+0000 7f96f7e00640 1 -- 172.21.15.196:0/3913496402 >> 172.21.15.196:0/3913496402 conn(0x7f96f806c6c0 msgr2=0x7f96f806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:01.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.152+0000 7f96f7e00640 1 -- 172.21.15.196:0/3913496402 shutdown_connections 2024-04-28T13:32:01.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.152+0000 7f96f7e00640 1 -- 172.21.15.196:0/3913496402 wait complete. 2024-04-28T13:32:01.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.154+0000 7f96f7e00640 1 Processor -- start 2024-04-28T13:32:01.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.154+0000 7f96f7e00640 1 -- start start 2024-04-28T13:32:01.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.155+0000 7f96f7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96f809ec80 0x7f96f81efc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:01.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.155+0000 7f96f7e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96f80b5ee0 0x7f96f815f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:01.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.155+0000 7f96f7e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f80d1760 0x7f96f81f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:01.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.155+0000 7f96f7e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f96f80b3c20 con 0x7f96f809ec80 2024-04-28T13:32:01.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.155+0000 7f96f7e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96f80b47b0 con 0x7f96f80b5ee0 2024-04-28T13:32:01.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.155+0000 7f96f7e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f96f80add70 con 0x7f96f80d1760 2024-04-28T13:32:01.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.155+0000 7f96f6a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96f809ec80 0x7f96f81efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:01.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.155+0000 7f96f6a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96f809ec80 0x7f96f81efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52556/0 (socket says 172.21.15.196:52556) 2024-04-28T13:32:01.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.155+0000 7f96f6a00640 1 -- 172.21.15.196:0/1545956274 learned_addr learned my addr 172.21.15.196:0/1545956274 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:01.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.156+0000 7f96ee000640 1 --2- 172.21.15.196:0/1545956274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96f80b5ee0 0x7f96f815f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:01.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.156+0000 7f96f6a00640 1 -- 172.21.15.196:0/1545956274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f80d1760 msgr2=0x7f96f81f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:01.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.156+0000 7f96f7400640 1 --2- 172.21.15.196:0/1545956274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f80d1760 0x7f96f81f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:01.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.156+0000 7f96f6a00640 1 --2- 172.21.15.196:0/1545956274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f80d1760 0x7f96f81f5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.156+0000 7f96f6a00640 1 -- 172.21.15.196:0/1545956274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96f80b5ee0 msgr2=0x7f96f815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:01.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.156+0000 7f96f6a00640 1 --2- 172.21.15.196:0/1545956274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96f80b5ee0 0x7f96f815f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.156+0000 7f96f6a00640 1 -- 172.21.15.196:0/1545956274 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f815dfe0 con 0x7f96f809ec80 2024-04-28T13:32:01.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.156+0000 7f96ee000640 1 --2- 172.21.15.196:0/1545956274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96f80b5ee0 0x7f96f815f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:01.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.156+0000 7f96f6a00640 1 --2- 172.21.15.196:0/1545956274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96f809ec80 0x7f96f81efc60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f96e800c4d0 tx=0x7f96e800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:01.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.156+0000 7f96ef400640 1 -- 172.21.15.196:0/1545956274 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8018070 con 0x7f96f809ec80 2024-04-28T13:32:01.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.156+0000 7f96ef400640 1 -- 172.21.15.196:0/1545956274 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e8007500 con 0x7f96f809ec80 2024-04-28T13:32:01.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.156+0000 7f96ef400640 1 -- 172.21.15.196:0/1545956274 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8004a20 con 0x7f96f809ec80 2024-04-28T13:32:01.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.156+0000 7f96f7e00640 1 -- 172.21.15.196:0/1545956274 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f815e2c0 con 0x7f96f809ec80 2024-04-28T13:32:01.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.157+0000 7f96f7e00640 1 -- 172.21.15.196:0/1545956274 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f815e880 con 0x7f96f809ec80 2024-04-28T13:32:01.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.158+0000 7f96f7e00640 1 -- 172.21.15.196:0/1545956274 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f815eac0 con 0x7f96f809ec80 2024-04-28T13:32:01.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.159+0000 7f96ef400640 1 -- 172.21.15.196:0/1545956274 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96e8004bc0 con 0x7f96f809ec80 2024-04-28T13:32:01.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.159+0000 7f96ef400640 1 --2- 172.21.15.196:0/1545956274 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f96d406c8f0 0x7f96d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:01.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.160+0000 7f96ef400640 1 -- 172.21.15.196:0/1545956274 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f96e808ae80 con 0x7f96f809ec80 2024-04-28T13:32:01.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.160+0000 7f96ee000640 1 --2- 172.21.15.196:0/1545956274 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f96d406c8f0 0x7f96d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:01.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.162+0000 7f96ee000640 1 --2- 172.21.15.196:0/1545956274 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f96d406c8f0 0x7f96d406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f96e0034000 tx=0x7f96e002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:01.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.164+0000 7f96ef400640 1 -- 172.21.15.196:0/1545956274 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f96f815eac0 con 0x7f96f809ec80 2024-04-28T13:32:01.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.364+0000 7f96f7e00640 1 -- 172.21.15.196:0/1545956274 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96f80c5c50 con 0x7f96f809ec80 2024-04-28T13:32:01.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.367+0000 7f96ef400640 1 -- 172.21.15.196:0/1545956274 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f96f80c5c50 con 0x7f96f809ec80 2024-04-28T13:32:01.368 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:01.368 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":20,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:31:22.624564+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:32:01.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.369+0000 7f96f7e00640 1 -- 172.21.15.196:0/1545956274 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f96d406c8f0 msgr2=0x7f96d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:01.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.369+0000 7f96f7e00640 1 --2- 172.21.15.196:0/1545956274 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f96d406c8f0 0x7f96d406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f96e0034000 tx=0x7f96e002e000 comp rx=0 tx=0).stop 2024-04-28T13:32:01.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.369+0000 7f96f7e00640 1 -- 172.21.15.196:0/1545956274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96f809ec80 msgr2=0x7f96f81efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:01.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.369+0000 7f96f7e00640 1 --2- 172.21.15.196:0/1545956274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96f809ec80 0x7f96f81efc60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f96e800c4d0 tx=0x7f96e800c990 comp rx=0 tx=0).stop 2024-04-28T13:32:01.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.370+0000 7f96f7e00640 1 -- 172.21.15.196:0/1545956274 shutdown_connections 2024-04-28T13:32:01.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.370+0000 7f96f7e00640 1 --2- 172.21.15.196:0/1545956274 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f96d406c8f0 0x7f96d406edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.370+0000 7f96f7e00640 1 --2- 172.21.15.196:0/1545956274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f80d1760 0x7f96f81f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.370+0000 7f96f7e00640 1 --2- 172.21.15.196:0/1545956274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96f80b5ee0 0x7f96f815f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.370+0000 7f96f7e00640 1 --2- 172.21.15.196:0/1545956274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96f809ec80 0x7f96f81efc60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.370+0000 7f96f7e00640 1 -- 172.21.15.196:0/1545956274 >> 172.21.15.196:0/1545956274 conn(0x7f96f806c6c0 msgr2=0x7f96f806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:01.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.370+0000 7f96f7e00640 1 -- 172.21.15.196:0/1545956274 shutdown_connections 2024-04-28T13:32:01.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.370+0000 7f96f7e00640 1 -- 172.21.15.196:0/1545956274 wait complete. 2024-04-28T13:32:01.390 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:31:19.280975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:31:20.037066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:32:01.391 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:32:01.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.597+0000 7eff13e00640 1 Processor -- start 2024-04-28T13:32:01.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.598+0000 7eff13e00640 1 -- start start 2024-04-28T13:32:01.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.599+0000 7eff13e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff1406e8f0 0x7eff140b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:01.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.599+0000 7eff13e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff1415d700 con 0x7eff140d1480 2024-04-28T13:32:01.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.599+0000 7eff13e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff1415d9e0 con 0x7eff140d10a0 2024-04-28T13:32:01.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.599+0000 7eff13e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7eff1415e310 con 0x7eff1406e8f0 2024-04-28T13:32:01.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.600+0000 7eff13400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7eff140d1480 0x7eff14160c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:34418/0 (socket says 172.21.15.196:34418) 2024-04-28T13:32:01.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.600+0000 7eff13400640 1 -- 172.21.15.196:0/1494214708 learned_addr learned my addr 172.21.15.196:0/1494214708 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:01.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.600+0000 7eff0a000640 1 --2- 172.21.15.196:0/1494214708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff1406e8f0 0x7eff140b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:01.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.600+0000 7eff12000640 1 -- 172.21.15.196:0/1494214708 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1180575690 0 0) 0x7eff1415d700 con 0x7eff140d1480 2024-04-28T13:32:01.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.600+0000 7eff12000640 1 -- 172.21.15.196:0/1494214708 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efef8003520 con 0x7eff140d1480 2024-04-28T13:32:01.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.600+0000 7eff12000640 1 -- 172.21.15.196:0/1494214708 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1382478190 0 0) 0x7efef8003520 con 0x7eff140d1480 2024-04-28T13:32:01.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.600+0000 7eff12000640 1 -- 172.21.15.196:0/1494214708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff1406e8f0 msgr2=0x7eff140b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:01.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.600+0000 7eff12000640 1 --2- 172.21.15.196:0/1494214708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff1406e8f0 0x7eff140b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.600+0000 7eff12000640 1 -- 172.21.15.196:0/1494214708 >> v1:172.21.15.203:6789/0 conn(0x7eff140d10a0 legacy=0x7eff140b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:01.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.600+0000 7eff12000640 1 -- 172.21.15.196:0/1494214708 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1415dcc0 con 0x7eff140d1480 2024-04-28T13:32:01.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff12000640 1 -- 172.21.15.196:0/1494214708 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7eff0c0019f0 con 0x7eff140d1480 2024-04-28T13:32:01.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff12000640 1 -- 172.21.15.196:0/1494214708 >> v1:172.21.15.196:6789/0 conn(0x7eff140d1480 legacy=0x7eff14160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:01.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff12000640 1 --2- 172.21.15.196:0/1494214708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff1406e8f0 0x7efef80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:01.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff12000640 1 --2- 172.21.15.196:0/1494214708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efef8003fb0 0x7efef8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:01.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff12000640 1 --2- 172.21.15.196:0/1494214708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efef8004d40 0x7efef80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:01.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff12000640 1 -- 172.21.15.196:0/1494214708 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff1415e310 con 0x7efef8004d40 2024-04-28T13:32:01.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff12000640 1 -- 172.21.15.196:0/1494214708 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efef800bb00 con 0x7efef8003fb0 2024-04-28T13:32:01.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff12000640 1 -- 172.21.15.196:0/1494214708 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efef800bd00 con 0x7eff1406e8f0 2024-04-28T13:32:01.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff13400640 1 --2- 172.21.15.196:0/1494214708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efef8004d40 0x7efef80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:01.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff0a000640 1 --2- 172.21.15.196:0/1494214708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efef8003fb0 0x7efef8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:01.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff13400640 1 -- 172.21.15.196:0/1494214708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff1406e8f0 msgr2=0x7efef80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:01.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff12a00640 1 --2- 172.21.15.196:0/1494214708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff1406e8f0 0x7efef80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:01.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff13400640 1 --2- 172.21.15.196:0/1494214708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff1406e8f0 0x7efef80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff13400640 1 -- 172.21.15.196:0/1494214708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efef8003fb0 msgr2=0x7efef8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:01.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff13400640 1 --2- 172.21.15.196:0/1494214708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efef8003fb0 0x7efef8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff13400640 1 -- 172.21.15.196:0/1494214708 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef8003520 con 0x7efef8004d40 2024-04-28T13:32:01.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff0a000640 1 --2- 172.21.15.196:0/1494214708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efef8003fb0 0x7efef8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:01.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.601+0000 7eff12a00640 1 --2- 172.21.15.196:0/1494214708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff1406e8f0 0x7efef80039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:01.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.602+0000 7eff13400640 1 --2- 172.21.15.196:0/1494214708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efef8004d40 0x7efef80051c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00a790 tx=0x7efef8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef77feada00fed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:01.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.603+0000 7eff12000640 1 -- 172.21.15.196:0/1494214708 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c00a3b0 con 0x7efef8004d40 2024-04-28T13:32:01.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.603+0000 7eff12000640 1 -- 172.21.15.196:0/1494214708 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff0c02f070 con 0x7efef8004d40 2024-04-28T13:32:01.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.603+0000 7eff12000640 1 -- 172.21.15.196:0/1494214708 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c02bc40 con 0x7efef8004d40 2024-04-28T13:32:01.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.603+0000 7eff13e00640 1 -- 172.21.15.196:0/1494214708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efef8004d40 msgr2=0x7efef80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:01.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.603+0000 7eff13e00640 1 --2- 172.21.15.196:0/1494214708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efef8004d40 0x7efef80051c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00a790 tx=0x7efef8005830 comp rx=0 tx=0).stop 2024-04-28T13:32:01.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.603+0000 7eff13400640 1 -- 172.21.15.196:0/1494214708 reap_dead start 2024-04-28T13:32:01.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.603+0000 7eff13e00640 1 -- 172.21.15.196:0/1494214708 shutdown_connections 2024-04-28T13:32:01.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.603+0000 7eff13e00640 1 -- 172.21.15.196:0/1494214708 >> 172.21.15.196:0/1494214708 conn(0x7eff1406c6c0 msgr2=0x7eff1406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:01.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.603+0000 7eff13e00640 1 -- 172.21.15.196:0/1494214708 shutdown_connections 2024-04-28T13:32:01.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.604+0000 7eff13e00640 1 -- 172.21.15.196:0/1494214708 wait complete. 2024-04-28T13:32:01.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.605+0000 7eff13e00640 1 Processor -- start 2024-04-28T13:32:01.606 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.605+0000 7eff13e00640 1 -- start start 2024-04-28T13:32:01.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.606+0000 7eff13e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efef8004d40 0x7eff141f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:01.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.606+0000 7eff13e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff140b01a0 0x7eff1409cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:01.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.606+0000 7eff13e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff141f8950 0x7eff14152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:01.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.606+0000 7eff13e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff140b3c90 con 0x7eff140b01a0 2024-04-28T13:32:01.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.606+0000 7eff13e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7eff140d18f0 con 0x7eff141f8950 2024-04-28T13:32:01.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.606+0000 7eff13e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7eff140b3e10 con 0x7efef8004d40 2024-04-28T13:32:01.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.607+0000 7eff0a000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff140b01a0 0x7eff1409cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:01.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.607+0000 7eff0a000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff140b01a0 0x7eff1409cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:52572/0 (socket says 172.21.15.196:52572) 2024-04-28T13:32:01.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.607+0000 7eff0a000640 1 -- 172.21.15.196:0/238998118 learned_addr learned my addr 172.21.15.196:0/238998118 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:01.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.607+0000 7eff12a00640 1 --2- 172.21.15.196:0/238998118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efef8004d40 0x7eff141f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:01.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.607+0000 7eff13400640 1 --2- 172.21.15.196:0/238998118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff141f8950 0x7eff14152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:01.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.607+0000 7eff0a000640 1 -- 172.21.15.196:0/238998118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efef8004d40 msgr2=0x7eff141f92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:01.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.607+0000 7eff0a000640 1 --2- 172.21.15.196:0/238998118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efef8004d40 0x7eff141f92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.607+0000 7eff0a000640 1 -- 172.21.15.196:0/238998118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff141f8950 msgr2=0x7eff14152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:01.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.607+0000 7eff0a000640 1 --2- 172.21.15.196:0/238998118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff141f8950 0x7eff14152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.607+0000 7eff0a000640 1 -- 172.21.15.196:0/238998118 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff140cdee0 con 0x7eff140b01a0 2024-04-28T13:32:01.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.607+0000 7eff12a00640 1 --2- 172.21.15.196:0/238998118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efef8004d40 0x7eff141f92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:01.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.607+0000 7eff0a000640 1 --2- 172.21.15.196:0/238998118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff140b01a0 0x7eff1409cd00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efefc0108e0 tx=0x7efefc010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:01.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.608+0000 7eff0b400640 1 -- 172.21.15.196:0/238998118 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc00f040 con 0x7eff140b01a0 2024-04-28T13:32:01.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.608+0000 7eff0b400640 1 -- 172.21.15.196:0/238998118 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efefc00ec20 con 0x7eff140b01a0 2024-04-28T13:32:01.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.608+0000 7eff0b400640 1 -- 172.21.15.196:0/238998118 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc01c400 con 0x7eff140b01a0 2024-04-28T13:32:01.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.608+0000 7eff13e00640 1 -- 172.21.15.196:0/238998118 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff140ce1c0 con 0x7eff140b01a0 2024-04-28T13:32:01.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.608+0000 7eff13e00640 1 -- 172.21.15.196:0/238998118 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff140ce780 con 0x7eff140b01a0 2024-04-28T13:32:01.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.609+0000 7eff13e00640 1 -- 172.21.15.196:0/238998118 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff14050730 con 0x7eff140b01a0 2024-04-28T13:32:01.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.610+0000 7eff0b400640 1 -- 172.21.15.196:0/238998118 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efefc01c5a0 con 0x7eff140b01a0 2024-04-28T13:32:01.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.611+0000 7eff0b400640 1 --2- 172.21.15.196:0/238998118 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efef006c8f0 0x7efef006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:01.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.611+0000 7eff0b400640 1 -- 172.21.15.196:0/238998118 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efefc08c410 con 0x7eff140b01a0 2024-04-28T13:32:01.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.611+0000 7eff12a00640 1 --2- 172.21.15.196:0/238998118 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efef006c8f0 0x7efef006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:01.612 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.613+0000 7eff12a00640 1 --2- 172.21.15.196:0/238998118 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efef006c8f0 0x7efef006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7eff040016f0 tx=0x7eff04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:01.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.615+0000 7eff0b400640 1 -- 172.21.15.196:0/238998118 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efefc018030 con 0x7eff140b01a0 2024-04-28T13:32:01.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.820+0000 7eff13e00640 1 -- 172.21.15.196:0/238998118 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff1406f700 con 0x7eff140b01a0 2024-04-28T13:32:01.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.821+0000 7eff0b400640 1 -- 172.21.15.196:0/238998118 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7efefc05a7a0 con 0x7eff140b01a0 2024-04-28T13:32:01.821 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:32:01.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.824+0000 7eff13e00640 1 -- 172.21.15.196:0/238998118 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efef006c8f0 msgr2=0x7efef006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:01.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.824+0000 7eff13e00640 1 --2- 172.21.15.196:0/238998118 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efef006c8f0 0x7efef006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7eff040016f0 tx=0x7eff04009040 comp rx=0 tx=0).stop 2024-04-28T13:32:01.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.824+0000 7eff13e00640 1 -- 172.21.15.196:0/238998118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff140b01a0 msgr2=0x7eff1409cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:01.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.824+0000 7eff13e00640 1 --2- 172.21.15.196:0/238998118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff140b01a0 0x7eff1409cd00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efefc0108e0 tx=0x7efefc010da0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.824+0000 7eff13e00640 1 -- 172.21.15.196:0/238998118 shutdown_connections 2024-04-28T13:32:01.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.824+0000 7eff13e00640 1 --2- 172.21.15.196:0/238998118 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efef006c8f0 0x7efef006edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.824+0000 7eff13e00640 1 --2- 172.21.15.196:0/238998118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff141f8950 0x7eff14152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.824+0000 7eff13e00640 1 --2- 172.21.15.196:0/238998118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff140b01a0 0x7eff1409cd00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.824+0000 7eff13e00640 1 --2- 172.21.15.196:0/238998118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efef8004d40 0x7eff141f92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:01.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.824+0000 7eff13e00640 1 -- 172.21.15.196:0/238998118 >> 172.21.15.196:0/238998118 conn(0x7eff1406c6c0 msgr2=0x7eff140b5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:01.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.824+0000 7eff13e00640 1 -- 172.21.15.196:0/238998118 shutdown_connections 2024-04-28T13:32:01.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:01.824+0000 7eff13e00640 1 -- 172.21.15.196:0/238998118 wait complete. 2024-04-28T13:32:01.843 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:32:02.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.046+0000 7f8953200640 1 Processor -- start 2024-04-28T13:32:02.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.046+0000 7f8953200640 1 -- start start 2024-04-28T13:32:02.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.048+0000 7f8953200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894c0d17b0 0x7f894c06f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.048+0000 7f8953200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f894c0b5df0 0x7f894c072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.048+0000 7f8953200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f894c06df10 con 0x7f894c15f150 2024-04-28T13:32:02.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.048+0000 7f8953200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f894c06fda0 con 0x7f894c0b5df0 2024-04-28T13:32:02.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.048+0000 7f8953200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f894c06e1f0 con 0x7f894c0d17b0 2024-04-28T13:32:02.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.048+0000 7f8951e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f894c15f150 0x7f894c0d10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43306/0 (socket says 172.21.15.196:43306) 2024-04-28T13:32:02.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.048+0000 7f8951e00640 1 -- 172.21.15.196:0/2876932999 learned_addr learned my addr 172.21.15.196:0/2876932999 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:02.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.049+0000 7f8952800640 1 --2- 172.21.15.196:0/2876932999 >> v2:172.21.15.203:3300/0 conn(0x7f894c0b5df0 0x7f894c072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.049+0000 7f8951400640 1 --2- 172.21.15.196:0/2876932999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894c0d17b0 0x7f894c06f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.049+0000 7f8950a00640 1 -- 172.21.15.196:0/2876932999 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2316473508 0 0) 0x7f894c06df10 con 0x7f894c15f150 2024-04-28T13:32:02.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.049+0000 7f8950a00640 1 -- 172.21.15.196:0/2876932999 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8934003520 con 0x7f894c15f150 2024-04-28T13:32:02.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.049+0000 7f8950a00640 1 -- 172.21.15.196:0/2876932999 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3551863606 0 0) 0x7f8934003520 con 0x7f894c15f150 2024-04-28T13:32:02.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.049+0000 7f8950a00640 1 -- 172.21.15.196:0/2876932999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894c0d17b0 msgr2=0x7f894c06f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:02.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.049+0000 7f8950a00640 1 --2- 172.21.15.196:0/2876932999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894c0d17b0 0x7f894c06f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.049+0000 7f8950a00640 1 -- 172.21.15.196:0/2876932999 >> v2:172.21.15.203:3300/0 conn(0x7f894c0b5df0 msgr2=0x7f894c072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:02.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.049+0000 7f8950a00640 1 --2- 172.21.15.196:0/2876932999 >> v2:172.21.15.203:3300/0 conn(0x7f894c0b5df0 0x7f894c072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.049+0000 7f8950a00640 1 -- 172.21.15.196:0/2876932999 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c06e4d0 con 0x7f894c15f150 2024-04-28T13:32:02.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.050+0000 7f8950a00640 1 -- 172.21.15.196:0/2876932999 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f8940001670 con 0x7f894c15f150 2024-04-28T13:32:02.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.050+0000 7f8950a00640 1 -- 172.21.15.196:0/2876932999 >> v1:172.21.15.196:6789/0 conn(0x7f894c15f150 legacy=0x7f894c0d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.050+0000 7f8950a00640 1 --2- 172.21.15.196:0/2876932999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89340039b0 0x7f8934003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.050+0000 7f8950a00640 1 --2- 172.21.15.196:0/2876932999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894c0d17b0 0x7f8934004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.050+0000 7f8950a00640 1 --2- 172.21.15.196:0/2876932999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8934004d40 0x7f89340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.050+0000 7f8950a00640 1 -- 172.21.15.196:0/2876932999 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f894c06fda0 con 0x7f89340039b0 2024-04-28T13:32:02.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.050+0000 7f8950a00640 1 -- 172.21.15.196:0/2876932999 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f894c06e1f0 con 0x7f8934004d40 2024-04-28T13:32:02.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.050+0000 7f8950a00640 1 -- 172.21.15.196:0/2876932999 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f893400b9c0 con 0x7f894c0d17b0 2024-04-28T13:32:02.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.050+0000 7f8951e00640 1 --2- 172.21.15.196:0/2876932999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89340039b0 0x7f8934003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.050+0000 7f8951400640 1 --2- 172.21.15.196:0/2876932999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894c0d17b0 0x7f8934004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.050+0000 7f8952800640 1 --2- 172.21.15.196:0/2876932999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8934004d40 0x7f89340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.051+0000 7f8951e00640 1 -- 172.21.15.196:0/2876932999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894c0d17b0 msgr2=0x7f8934004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:02.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.051+0000 7f8951e00640 1 --2- 172.21.15.196:0/2876932999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894c0d17b0 0x7f8934004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.051+0000 7f8951e00640 1 -- 172.21.15.196:0/2876932999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8934004d40 msgr2=0x7f89340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:02.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.051+0000 7f8951e00640 1 --2- 172.21.15.196:0/2876932999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8934004d40 0x7f89340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.051+0000 7f8951e00640 1 -- 172.21.15.196:0/2876932999 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8934003520 con 0x7f89340039b0 2024-04-28T13:32:02.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.052+0000 7f8951e00640 1 --2- 172.21.15.196:0/2876932999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89340039b0 0x7f8934003dd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f894000c240 tx=0x7f8934005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fdd597a1ac5ffa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:02.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.052+0000 7f8950a00640 1 -- 172.21.15.196:0/2876932999 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894002e070 con 0x7f89340039b0 2024-04-28T13:32:02.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.052+0000 7f8950a00640 1 -- 172.21.15.196:0/2876932999 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8940005ad0 con 0x7f89340039b0 2024-04-28T13:32:02.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.052+0000 7f8950a00640 1 -- 172.21.15.196:0/2876932999 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940007d30 con 0x7f89340039b0 2024-04-28T13:32:02.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.053+0000 7f8953200640 1 -- 172.21.15.196:0/2876932999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89340039b0 msgr2=0x7f8934003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.053+0000 7f8953200640 1 --2- 172.21.15.196:0/2876932999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89340039b0 0x7f8934003dd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f894000c240 tx=0x7f8934005830 comp rx=0 tx=0).stop 2024-04-28T13:32:02.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.053+0000 7f8952800640 1 -- 172.21.15.196:0/2876932999 reap_dead start 2024-04-28T13:32:02.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.053+0000 7f8953200640 1 -- 172.21.15.196:0/2876932999 shutdown_connections 2024-04-28T13:32:02.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.053+0000 7f8953200640 1 -- 172.21.15.196:0/2876932999 >> 172.21.15.196:0/2876932999 conn(0x7f894c06c6c0 msgr2=0x7f894c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:02.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.053+0000 7f8953200640 1 -- 172.21.15.196:0/2876932999 shutdown_connections 2024-04-28T13:32:02.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.053+0000 7f8953200640 1 -- 172.21.15.196:0/2876932999 wait complete. 2024-04-28T13:32:02.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.055+0000 7f8953200640 1 Processor -- start 2024-04-28T13:32:02.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.055+0000 7f8953200640 1 -- start start 2024-04-28T13:32:02.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.056+0000 7f8953200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89340039b0 0x7f894c159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.056+0000 7f8953200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f894c15c3f0 0x7f894c161850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.056+0000 7f8953200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894c15c870 0x7f894c161d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.056+0000 7f8953200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f894c15f530 con 0x7f89340039b0 2024-04-28T13:32:02.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.056+0000 7f8953200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f894c06ea90 con 0x7f894c15c3f0 2024-04-28T13:32:02.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.056+0000 7f8953200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f894c06f860 con 0x7f894c15c870 2024-04-28T13:32:02.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.056+0000 7f8951e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89340039b0 0x7f894c159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.056+0000 7f8951e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89340039b0 0x7f894c159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34110/0 (socket says 172.21.15.196:34110) 2024-04-28T13:32:02.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.056+0000 7f8951e00640 1 -- 172.21.15.196:0/1933881130 learned_addr learned my addr 172.21.15.196:0/1933881130 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:02.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.057+0000 7f8951e00640 1 -- 172.21.15.196:0/1933881130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894c15c870 msgr2=0x7f894c161d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.057+0000 7f8951e00640 1 --2- 172.21.15.196:0/1933881130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894c15c870 0x7f894c161d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.057+0000 7f8951e00640 1 -- 172.21.15.196:0/1933881130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f894c15c3f0 msgr2=0x7f894c161850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.057+0000 7f8951400640 1 --2- 172.21.15.196:0/1933881130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f894c15c3f0 0x7f894c161850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.057+0000 7f8951e00640 1 --2- 172.21.15.196:0/1933881130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f894c15c3f0 0x7f894c161850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.057+0000 7f8951e00640 1 -- 172.21.15.196:0/1933881130 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c15b070 con 0x7f89340039b0 2024-04-28T13:32:02.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.057+0000 7f8951e00640 1 --2- 172.21.15.196:0/1933881130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89340039b0 0x7f894c159bb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f894002ae30 tx=0x7f894c06e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:02.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.057+0000 7f8946000640 1 -- 172.21.15.196:0/1933881130 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940042070 con 0x7f89340039b0 2024-04-28T13:32:02.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.057+0000 7f8946000640 1 -- 172.21.15.196:0/1933881130 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8940007480 con 0x7f89340039b0 2024-04-28T13:32:02.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.057+0000 7f8953200640 1 -- 172.21.15.196:0/1933881130 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894c15b350 con 0x7f89340039b0 2024-04-28T13:32:02.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.057+0000 7f8946000640 1 -- 172.21.15.196:0/1933881130 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940007d30 con 0x7f89340039b0 2024-04-28T13:32:02.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.057+0000 7f8953200640 1 -- 172.21.15.196:0/1933881130 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894c15b910 con 0x7f89340039b0 2024-04-28T13:32:02.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.058+0000 7f8953200640 1 -- 172.21.15.196:0/1933881130 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f894c050730 con 0x7f89340039b0 2024-04-28T13:32:02.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.059+0000 7f8946000640 1 -- 172.21.15.196:0/1933881130 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8940045070 con 0x7f89340039b0 2024-04-28T13:32:02.060 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.060+0000 7f8946000640 1 --2- 172.21.15.196:0/1933881130 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f892c06c920 0x7f892c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.060 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.060+0000 7f8946000640 1 -- 172.21.15.196:0/1933881130 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f89400ac900 con 0x7f89340039b0 2024-04-28T13:32:02.061 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.061+0000 7f8951400640 1 --2- 172.21.15.196:0/1933881130 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f892c06c920 0x7f892c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.062+0000 7f8951400640 1 --2- 172.21.15.196:0/1933881130 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f892c06c920 0x7f892c06ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8938009950 tx=0x7f8938008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:02.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.065+0000 7f8946000640 1 -- 172.21.15.196:0/1933881130 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f894004c020 con 0x7f89340039b0 2024-04-28T13:32:02.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.266+0000 7f8953200640 1 -- 172.21.15.196:0/1933881130 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f894c0ae890 con 0x7f89340039b0 2024-04-28T13:32:02.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.267+0000 7f8946000640 1 -- 172.21.15.196:0/1933881130 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f89400ac020 con 0x7f89340039b0 2024-04-28T13:32:02.267 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:32:02.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.269+0000 7f8953200640 1 -- 172.21.15.196:0/1933881130 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f892c06c920 msgr2=0x7f892c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.269+0000 7f8953200640 1 --2- 172.21.15.196:0/1933881130 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f892c06c920 0x7f892c06ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8938009950 tx=0x7f8938008040 comp rx=0 tx=0).stop 2024-04-28T13:32:02.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.269+0000 7f8953200640 1 -- 172.21.15.196:0/1933881130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89340039b0 msgr2=0x7f894c159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.269+0000 7f8953200640 1 --2- 172.21.15.196:0/1933881130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89340039b0 0x7f894c159bb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f894002ae30 tx=0x7f894c06e0e0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.269+0000 7f8953200640 1 -- 172.21.15.196:0/1933881130 shutdown_connections 2024-04-28T13:32:02.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.269+0000 7f8953200640 1 --2- 172.21.15.196:0/1933881130 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f892c06c920 0x7f892c06ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.269+0000 7f8953200640 1 --2- 172.21.15.196:0/1933881130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894c15c870 0x7f894c161d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.269+0000 7f8953200640 1 --2- 172.21.15.196:0/1933881130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f894c15c3f0 0x7f894c161850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.269+0000 7f8953200640 1 --2- 172.21.15.196:0/1933881130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89340039b0 0x7f894c159bb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.269+0000 7f8953200640 1 -- 172.21.15.196:0/1933881130 >> 172.21.15.196:0/1933881130 conn(0x7f894c06c6c0 msgr2=0x7f894c0adff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:02.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.269+0000 7f8953200640 1 -- 172.21.15.196:0/1933881130 shutdown_connections 2024-04-28T13:32:02.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.270+0000 7f8953200640 1 -- 172.21.15.196:0/1933881130 wait complete. 2024-04-28T13:32:02.288 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:32:02.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.492+0000 7fa9cec00640 1 Processor -- start 2024-04-28T13:32:02.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.492+0000 7fa9cec00640 1 -- start start 2024-04-28T13:32:02.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.493+0000 7fa9cec00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9c80b4eb0 0x7fa9c80b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.493+0000 7fa9cec00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9c815dca0 con 0x7fa9c80ceff0 2024-04-28T13:32:02.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.493+0000 7fa9cec00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9c815df80 con 0x7fa9c8143840 2024-04-28T13:32:02.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.493+0000 7fa9cec00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c8143c20 con 0x7fa9c80b4eb0 2024-04-28T13:32:02.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.494+0000 7fa9ccc00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa9c80ceff0 0x7fa9c80cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43318/0 (socket says 172.21.15.196:43318) 2024-04-28T13:32:02.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.494+0000 7fa9ccc00640 1 -- 172.21.15.196:0/688723257 learned_addr learned my addr 172.21.15.196:0/688723257 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:02.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.494+0000 7fa9bf400640 1 --2- 172.21.15.196:0/688723257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9c80b4eb0 0x7fa9c80b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.494+0000 7fa9c7400640 1 -- 172.21.15.196:0/688723257 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 945018561 0 0) 0x7fa9c815dca0 con 0x7fa9c80ceff0 2024-04-28T13:32:02.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.494+0000 7fa9c7400640 1 -- 172.21.15.196:0/688723257 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9ac003520 con 0x7fa9c80ceff0 2024-04-28T13:32:02.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.495+0000 7fa9c7400640 1 -- 172.21.15.196:0/688723257 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3852401821 0 0) 0x7fa9ac003520 con 0x7fa9c80ceff0 2024-04-28T13:32:02.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.495+0000 7fa9c7400640 1 -- 172.21.15.196:0/688723257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9c80b4eb0 msgr2=0x7fa9c80b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:02.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.495+0000 7fa9c7400640 1 --2- 172.21.15.196:0/688723257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9c80b4eb0 0x7fa9c80b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.495+0000 7fa9c7400640 1 -- 172.21.15.196:0/688723257 >> v1:172.21.15.203:6789/0 conn(0x7fa9c8143840 legacy=0x7fa9c80b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.495+0000 7fa9c7400640 1 -- 172.21.15.196:0/688723257 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c815e260 con 0x7fa9c80ceff0 2024-04-28T13:32:02.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.495+0000 7fa9c7400640 1 -- 172.21.15.196:0/688723257 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fa9c00019f0 con 0x7fa9c80ceff0 2024-04-28T13:32:02.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.495+0000 7fa9c7400640 1 -- 172.21.15.196:0/688723257 >> v1:172.21.15.196:6789/0 conn(0x7fa9c80ceff0 legacy=0x7fa9c80cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.495+0000 7fa9c7400640 1 --2- 172.21.15.196:0/688723257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9c80b4eb0 0x7fa9ac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.495+0000 7fa9c7400640 1 --2- 172.21.15.196:0/688723257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9ac003fb0 0x7fa9ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.495+0000 7fa9c7400640 1 --2- 172.21.15.196:0/688723257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9ac004d40 0x7fa9ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.495+0000 7fa9c7400640 1 -- 172.21.15.196:0/688723257 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c8143c20 con 0x7fa9ac004d40 2024-04-28T13:32:02.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.495+0000 7fa9c7400640 1 -- 172.21.15.196:0/688723257 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac00bb00 con 0x7fa9ac003fb0 2024-04-28T13:32:02.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.495+0000 7fa9c7400640 1 -- 172.21.15.196:0/688723257 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ac00bd00 con 0x7fa9c80b4eb0 2024-04-28T13:32:02.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.495+0000 7fa9ccc00640 1 --2- 172.21.15.196:0/688723257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9ac004d40 0x7fa9ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.495+0000 7fa9c7e00640 1 --2- 172.21.15.196:0/688723257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9c80b4eb0 0x7fa9ac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.496+0000 7fa9bf400640 1 --2- 172.21.15.196:0/688723257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9ac003fb0 0x7fa9ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.496+0000 7fa9ccc00640 1 -- 172.21.15.196:0/688723257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9c80b4eb0 msgr2=0x7fa9ac0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:02.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.496+0000 7fa9ccc00640 1 --2- 172.21.15.196:0/688723257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9c80b4eb0 0x7fa9ac0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.496+0000 7fa9ccc00640 1 -- 172.21.15.196:0/688723257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9ac003fb0 msgr2=0x7fa9ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:02.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.496+0000 7fa9ccc00640 1 --2- 172.21.15.196:0/688723257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9ac003fb0 0x7fa9ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.496+0000 7fa9ccc00640 1 -- 172.21.15.196:0/688723257 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ac003520 con 0x7fa9ac004d40 2024-04-28T13:32:02.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.496+0000 7fa9bf400640 1 --2- 172.21.15.196:0/688723257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9ac003fb0 0x7fa9ac004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:02.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.497+0000 7fa9ccc00640 1 --2- 172.21.15.196:0/688723257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9ac004d40 0x7fa9ac0051c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000a7b0 tx=0x7fa9ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abc30c187597982 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:02.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.497+0000 7fa9c7400640 1 -- 172.21.15.196:0/688723257 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c000a3d0 con 0x7fa9ac004d40 2024-04-28T13:32:02.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.497+0000 7fa9c7400640 1 -- 172.21.15.196:0/688723257 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c002f070 con 0x7fa9ac004d40 2024-04-28T13:32:02.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.497+0000 7fa9c7400640 1 -- 172.21.15.196:0/688723257 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c002bca0 con 0x7fa9ac004d40 2024-04-28T13:32:02.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.498+0000 7fa9cec00640 1 -- 172.21.15.196:0/688723257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9ac004d40 msgr2=0x7fa9ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.498+0000 7fa9cec00640 1 --2- 172.21.15.196:0/688723257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9ac004d40 0x7fa9ac0051c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000a7b0 tx=0x7fa9ac005830 comp rx=0 tx=0).stop 2024-04-28T13:32:02.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.498+0000 7fa9ccc00640 1 -- 172.21.15.196:0/688723257 reap_dead start 2024-04-28T13:32:02.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.498+0000 7fa9cec00640 1 -- 172.21.15.196:0/688723257 shutdown_connections 2024-04-28T13:32:02.501 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.498+0000 7fa9cec00640 1 -- 172.21.15.196:0/688723257 >> 172.21.15.196:0/688723257 conn(0x7fa9c80cd8a0 msgr2=0x7fa9c80cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:02.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.498+0000 7fa9cec00640 1 -- 172.21.15.196:0/688723257 shutdown_connections 2024-04-28T13:32:02.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.498+0000 7fa9cec00640 1 -- 172.21.15.196:0/688723257 wait complete. 2024-04-28T13:32:02.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.500+0000 7fa9cec00640 1 Processor -- start 2024-04-28T13:32:02.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.500+0000 7fa9cec00640 1 -- start start 2024-04-28T13:32:02.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9cec00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9ac004d40 0x7fa9c81417c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9cec00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9c80c9d50 0x7fa9c81f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9cec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c81f6ed0 0x7fa9c80c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9cec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c815d8c0 con 0x7fa9c81f6ed0 2024-04-28T13:32:02.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9cec00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c80b46f0 con 0x7fa9c80c9d50 2024-04-28T13:32:02.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9cec00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c815da40 con 0x7fa9ac004d40 2024-04-28T13:32:02.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9ccc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c81f6ed0 0x7fa9c80c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9ccc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c81f6ed0 0x7fa9c80c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34130/0 (socket says 172.21.15.196:34130) 2024-04-28T13:32:02.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9ccc00640 1 -- 172.21.15.196:0/1997639044 learned_addr learned my addr 172.21.15.196:0/1997639044 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:02.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9c7e00640 1 --2- 172.21.15.196:0/1997639044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9ac004d40 0x7fa9c81417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9bf400640 1 --2- 172.21.15.196:0/1997639044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9c80c9d50 0x7fa9c81f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9ccc00640 1 -- 172.21.15.196:0/1997639044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9ac004d40 msgr2=0x7fa9c81417c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9ccc00640 1 --2- 172.21.15.196:0/1997639044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9ac004d40 0x7fa9c81417c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9ccc00640 1 -- 172.21.15.196:0/1997639044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9c80c9d50 msgr2=0x7fa9c81f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9ccc00640 1 --2- 172.21.15.196:0/1997639044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9c80c9d50 0x7fa9c81f6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9ccc00640 1 -- 172.21.15.196:0/1997639044 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c80c6a80 con 0x7fa9c81f6ed0 2024-04-28T13:32:02.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.501+0000 7fa9bf400640 1 --2- 172.21.15.196:0/1997639044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9c80c9d50 0x7fa9c81f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:02.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.502+0000 7fa9ccc00640 1 --2- 172.21.15.196:0/1997639044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c81f6ed0 0x7fa9c80c5dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0000f80 tx=0x7fa9c0001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:02.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.502+0000 7fa9c4c00640 1 -- 172.21.15.196:0/1997639044 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0044070 con 0x7fa9c81f6ed0 2024-04-28T13:32:02.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.502+0000 7fa9c4c00640 1 -- 172.21.15.196:0/1997639044 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c002cd10 con 0x7fa9c81f6ed0 2024-04-28T13:32:02.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.502+0000 7fa9cec00640 1 -- 172.21.15.196:0/1997639044 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c80c6d60 con 0x7fa9c81f6ed0 2024-04-28T13:32:02.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.502+0000 7fa9c4c00640 1 -- 172.21.15.196:0/1997639044 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c000a3d0 con 0x7fa9c81f6ed0 2024-04-28T13:32:02.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.502+0000 7fa9cec00640 1 -- 172.21.15.196:0/1997639044 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c80ad880 con 0x7fa9c81f6ed0 2024-04-28T13:32:02.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.503+0000 7fa9cec00640 1 -- 172.21.15.196:0/1997639044 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c806a290 con 0x7fa9c81f6ed0 2024-04-28T13:32:02.506 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.504+0000 7fa9c4c00640 1 -- 172.21.15.196:0/1997639044 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9c0047070 con 0x7fa9c81f6ed0 2024-04-28T13:32:02.506 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.505+0000 7fa9c4c00640 1 --2- 172.21.15.196:0/1997639044 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa9a406c8f0 0x7fa9a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.506 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.505+0000 7fa9c4c00640 1 -- 172.21.15.196:0/1997639044 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa9c00abb60 con 0x7fa9c81f6ed0 2024-04-28T13:32:02.506 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.506+0000 7fa9c7e00640 1 --2- 172.21.15.196:0/1997639044 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa9a406c8f0 0x7fa9a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.507 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.507+0000 7fa9c7e00640 1 --2- 172.21.15.196:0/1997639044 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa9a406c8f0 0x7fa9a406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8007fd0 tx=0x7fa9b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:02.510 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.510+0000 7fa9c4c00640 1 -- 172.21.15.196:0/1997639044 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9c007a020 con 0x7fa9c81f6ed0 2024-04-28T13:32:02.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.708+0000 7fa9cec00640 1 -- 172.21.15.196:0/1997639044 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa9c80ce420 con 0x7fa9c81f6ed0 2024-04-28T13:32:02.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.709+0000 7fa9c4c00640 1 -- 172.21.15.196:0/1997639044 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fa9c0079920 con 0x7fa9c81f6ed0 2024-04-28T13:32:02.709 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:32:02.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.712+0000 7fa9cec00640 1 -- 172.21.15.196:0/1997639044 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa9a406c8f0 msgr2=0x7fa9a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.712+0000 7fa9cec00640 1 --2- 172.21.15.196:0/1997639044 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa9a406c8f0 0x7fa9a406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8007fd0 tx=0x7fa9b8009040 comp rx=0 tx=0).stop 2024-04-28T13:32:02.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.712+0000 7fa9cec00640 1 -- 172.21.15.196:0/1997639044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c81f6ed0 msgr2=0x7fa9c80c5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.712+0000 7fa9cec00640 1 --2- 172.21.15.196:0/1997639044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c81f6ed0 0x7fa9c80c5dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0000f80 tx=0x7fa9c0001cf0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.712+0000 7fa9cec00640 1 -- 172.21.15.196:0/1997639044 shutdown_connections 2024-04-28T13:32:02.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.712+0000 7fa9cec00640 1 --2- 172.21.15.196:0/1997639044 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa9a406c8f0 0x7fa9a406edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.712+0000 7fa9cec00640 1 --2- 172.21.15.196:0/1997639044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9c81f6ed0 0x7fa9c80c5dc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.712+0000 7fa9cec00640 1 --2- 172.21.15.196:0/1997639044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9c80c9d50 0x7fa9c81f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.712+0000 7fa9cec00640 1 --2- 172.21.15.196:0/1997639044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9ac004d40 0x7fa9c81417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.712+0000 7fa9cec00640 1 -- 172.21.15.196:0/1997639044 >> 172.21.15.196:0/1997639044 conn(0x7fa9c80cd8a0 msgr2=0x7fa9c80b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:02.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.712+0000 7fa9cec00640 1 -- 172.21.15.196:0/1997639044 shutdown_connections 2024-04-28T13:32:02.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.712+0000 7fa9cec00640 1 -- 172.21.15.196:0/1997639044 wait complete. 2024-04-28T13:32:02.731 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked' 2024-04-28T13:32:02.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.926+0000 7f2d3fc00640 1 Processor -- start 2024-04-28T13:32:02.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.926+0000 7f2d3fc00640 1 -- start start 2024-04-28T13:32:02.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.928+0000 7f2d3fc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2d380d12a0 0x7f2d381632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.928+0000 7f2d3fc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d38163910 con 0x7f2d38162e50 2024-04-28T13:32:02.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.928+0000 7f2d3fc00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d38163bf0 con 0x7f2d38073fc0 2024-04-28T13:32:02.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.928+0000 7f2d3fc00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2d3806ea50 con 0x7f2d380d12a0 2024-04-28T13:32:02.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.928+0000 7f2d3dc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2d380d12a0 0x7f2d381632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.928+0000 7f2d37e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f2d38162e50 0x7f2d380d0b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43334/0 (socket says 172.21.15.196:43334) 2024-04-28T13:32:02.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.928+0000 7f2d37e00640 1 -- 172.21.15.196:0/4090828046 learned_addr learned my addr 172.21.15.196:0/4090828046 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:02.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.928+0000 7f2d3dc00640 1 -- 172.21.15.196:0/4090828046 >> v1:172.21.15.203:6789/0 conn(0x7f2d38073fc0 legacy=0x7f2d38072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:02.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.928+0000 7f2d3dc00640 1 -- 172.21.15.196:0/4090828046 >> v1:172.21.15.196:6789/0 conn(0x7f2d38162e50 legacy=0x7f2d380d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.928+0000 7f2d3dc00640 1 -- 172.21.15.196:0/4090828046 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3806ed30 con 0x7f2d380d12a0 2024-04-28T13:32:02.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.929+0000 7f2d3dc00640 1 --2- 172.21.15.196:0/4090828046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d380d12a0 0x7f2d381632c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009d90 tx=0x7f2d3002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35ea88dc9a12ed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:02.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.930+0000 7f2d37400640 1 -- 172.21.15.196:0/4090828046 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30007740 con 0x7f2d380d12a0 2024-04-28T13:32:02.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.930+0000 7f2d37400640 1 -- 172.21.15.196:0/4090828046 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d300078e0 con 0x7f2d380d12a0 2024-04-28T13:32:02.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.930+0000 7f2d37400640 1 -- 172.21.15.196:0/4090828046 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30007cb0 con 0x7f2d380d12a0 2024-04-28T13:32:02.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.930+0000 7f2d3fc00640 1 -- 172.21.15.196:0/4090828046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d380d12a0 msgr2=0x7f2d381632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.930+0000 7f2d3fc00640 1 --2- 172.21.15.196:0/4090828046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d380d12a0 0x7f2d381632c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009d90 tx=0x7f2d3002a730 comp rx=0 tx=0).stop 2024-04-28T13:32:02.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.930+0000 7f2d3fc00640 1 -- 172.21.15.196:0/4090828046 shutdown_connections 2024-04-28T13:32:02.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.930+0000 7f2d3fc00640 1 --2- 172.21.15.196:0/4090828046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d380d12a0 0x7f2d381632c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.930+0000 7f2d3fc00640 1 -- 172.21.15.196:0/4090828046 >> 172.21.15.196:0/4090828046 conn(0x7f2d380cd830 msgr2=0x7f2d380cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:02.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.931+0000 7f2d3fc00640 1 -- 172.21.15.196:0/4090828046 shutdown_connections 2024-04-28T13:32:02.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.931+0000 7f2d3fc00640 1 -- 172.21.15.196:0/4090828046 wait complete. 2024-04-28T13:32:02.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.932+0000 7f2d3fc00640 1 Processor -- start 2024-04-28T13:32:02.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.933+0000 7f2d3fc00640 1 -- start start 2024-04-28T13:32:02.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.933+0000 7f2d3fc00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d38073fc0 0x7f2d38146880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.933+0000 7f2d3fc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d380d12a0 0x7f2d38072650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.933+0000 7f2d3fc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d38162e50 0x7f2d381eaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.933+0000 7f2d3fc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2d380743a0 con 0x7f2d38162e50 2024-04-28T13:32:02.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.933+0000 7f2d3fc00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3806f250 con 0x7f2d380d12a0 2024-04-28T13:32:02.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.933+0000 7f2d3fc00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2d381e9760 con 0x7f2d38073fc0 2024-04-28T13:32:02.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.934+0000 7f2d3dc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d38162e50 0x7f2d381eaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.934+0000 7f2d3dc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d38162e50 0x7f2d381eaf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34150/0 (socket says 172.21.15.196:34150) 2024-04-28T13:32:02.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.934+0000 7f2d3dc00640 1 -- 172.21.15.196:0/3645279697 learned_addr learned my addr 172.21.15.196:0/3645279697 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:02.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.934+0000 7f2d37e00640 1 --2- 172.21.15.196:0/3645279697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d380d12a0 0x7f2d38072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.934+0000 7f2d3d200640 1 --2- 172.21.15.196:0/3645279697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d38073fc0 0x7f2d38146880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.934+0000 7f2d3dc00640 1 -- 172.21.15.196:0/3645279697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d38073fc0 msgr2=0x7f2d38146880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.934+0000 7f2d3dc00640 1 --2- 172.21.15.196:0/3645279697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d38073fc0 0x7f2d38146880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.934+0000 7f2d3dc00640 1 -- 172.21.15.196:0/3645279697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d380d12a0 msgr2=0x7f2d38072650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:02.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.934+0000 7f2d3dc00640 1 --2- 172.21.15.196:0/3645279697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d380d12a0 0x7f2d38072650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:02.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.934+0000 7f2d3dc00640 1 -- 172.21.15.196:0/3645279697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d381e8e30 con 0x7f2d38162e50 2024-04-28T13:32:02.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.934+0000 7f2d37e00640 1 --2- 172.21.15.196:0/3645279697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d380d12a0 0x7f2d38072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:02.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.935+0000 7f2d3dc00640 1 --2- 172.21.15.196:0/3645279697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d38162e50 0x7f2d381eaf50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009d90 tx=0x7f2d30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:02.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.935+0000 7f2d34c00640 1 -- 172.21.15.196:0/3645279697 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30040070 con 0x7f2d38162e50 2024-04-28T13:32:02.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.935+0000 7f2d34c00640 1 -- 172.21.15.196:0/3645279697 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d30031bc0 con 0x7f2d38162e50 2024-04-28T13:32:02.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.935+0000 7f2d3fc00640 1 -- 172.21.15.196:0/3645279697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d381e9110 con 0x7f2d38162e50 2024-04-28T13:32:02.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.935+0000 7f2d34c00640 1 -- 172.21.15.196:0/3645279697 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3003c420 con 0x7f2d38162e50 2024-04-28T13:32:02.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.935+0000 7f2d3fc00640 1 -- 172.21.15.196:0/3645279697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d38072b90 con 0x7f2d38162e50 2024-04-28T13:32:02.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.936+0000 7f2d3fc00640 1 -- 172.21.15.196:0/3645279697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d3806a290 con 0x7f2d38162e50 2024-04-28T13:32:02.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.937+0000 7f2d34c00640 1 -- 172.21.15.196:0/3645279697 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d30039040 con 0x7f2d38162e50 2024-04-28T13:32:02.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.938+0000 7f2d34c00640 1 --2- 172.21.15.196:0/3645279697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2d1806c8f0 0x7f2d1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:02.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.938+0000 7f2d34c00640 1 -- 172.21.15.196:0/3645279697 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f2d300ad1f0 con 0x7f2d38162e50 2024-04-28T13:32:02.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.939+0000 7f2d3d200640 1 --2- 172.21.15.196:0/3645279697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2d1806c8f0 0x7f2d1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:02.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.940+0000 7f2d3d200640 1 --2- 172.21.15.196:0/3645279697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2d1806c8f0 0x7f2d1806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2d380738f0 tx=0x7f2d2c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:02.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:02.942+0000 7f2d34c00640 1 -- 172.21.15.196:0/3645279697 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d3007b5c0 con 0x7f2d38162e50 2024-04-28T13:32:03.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:03.221+0000 7f2d3fc00640 1 -- 172.21.15.196:0/3645279697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]} v 0) -- 0x7f2d380b04a0 con 0x7f2d38162e50 2024-04-28T13:32:03.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:03.885+0000 7f2d34c00640 1 -- 172.21.15.196:0/3645279697 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]}]=0 v61) ==== 140+0+0 (secure 0 0 0) 0x7f2d3007aec0 con 0x7f2d38162e50 2024-04-28T13:32:03.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:03.888+0000 7f2d3fc00640 1 -- 172.21.15.196:0/3645279697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2d1806c8f0 msgr2=0x7f2d1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:03.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:03.888+0000 7f2d3fc00640 1 --2- 172.21.15.196:0/3645279697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2d1806c8f0 0x7f2d1806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2d380738f0 tx=0x7f2d2c009040 comp rx=0 tx=0).stop 2024-04-28T13:32:03.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:03.888+0000 7f2d3fc00640 1 -- 172.21.15.196:0/3645279697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d38162e50 msgr2=0x7f2d381eaf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:03.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:03.888+0000 7f2d3fc00640 1 --2- 172.21.15.196:0/3645279697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d38162e50 0x7f2d381eaf50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009d90 tx=0x7f2d30004a40 comp rx=0 tx=0).stop 2024-04-28T13:32:03.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:03.888+0000 7f2d3fc00640 1 -- 172.21.15.196:0/3645279697 shutdown_connections 2024-04-28T13:32:03.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:03.889+0000 7f2d3fc00640 1 --2- 172.21.15.196:0/3645279697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2d1806c8f0 0x7f2d1806edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:03.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:03.889+0000 7f2d3fc00640 1 --2- 172.21.15.196:0/3645279697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d38162e50 0x7f2d381eaf50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:03.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:03.889+0000 7f2d3fc00640 1 --2- 172.21.15.196:0/3645279697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d380d12a0 0x7f2d38072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:03.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:03.889+0000 7f2d3fc00640 1 --2- 172.21.15.196:0/3645279697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d38073fc0 0x7f2d38146880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:03.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:03.889+0000 7f2d3fc00640 1 -- 172.21.15.196:0/3645279697 >> 172.21.15.196:0/3645279697 conn(0x7f2d380cd830 msgr2=0x7f2d38161810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:03.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:03.889+0000 7f2d3fc00640 1 -- 172.21.15.196:0/3645279697 shutdown_connections 2024-04-28T13:32:03.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:03.889+0000 7f2d3fc00640 1 -- 172.21.15.196:0/3645279697 wait complete. 2024-04-28T13:32:03.913 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-28T13:32:04.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.118+0000 7fd21ea00640 1 Processor -- start 2024-04-28T13:32:04.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.118+0000 7fd21ea00640 1 -- start start 2024-04-28T13:32:04.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.120+0000 7fd21ea00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2180d3820 0x7fd2180b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:04.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.120+0000 7fd21ea00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd2180b60b0 0x7fd2180d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:04.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.120+0000 7fd21ea00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2180d3f50 con 0x7fd21815f680 2024-04-28T13:32:04.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.120+0000 7fd21ea00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd21815fa60 con 0x7fd2180b60b0 2024-04-28T13:32:04.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.120+0000 7fd21ea00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd2180b5090 con 0x7fd2180d3820 2024-04-28T13:32:04.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.120+0000 7fd217e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fd21815f680 0x7fd218163aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43344/0 (socket says 172.21.15.196:43344) 2024-04-28T13:32:04.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.120+0000 7fd217e00640 1 -- 172.21.15.196:0/3198177539 learned_addr learned my addr 172.21.15.196:0/3198177539 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:04.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.120+0000 7fd217400640 1 --2- 172.21.15.196:0/3198177539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2180d3820 0x7fd2180b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:04.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.120+0000 7fd216a00640 1 -- 172.21.15.196:0/3198177539 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003251917 0 0) 0x7fd2180d3f50 con 0x7fd21815f680 2024-04-28T13:32:04.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.120+0000 7fd216a00640 1 -- 172.21.15.196:0/3198177539 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1f8003520 con 0x7fd21815f680 2024-04-28T13:32:04.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.120+0000 7fd21ca00640 1 --2- 172.21.15.196:0/3198177539 >> v2:172.21.15.203:3300/0 conn(0x7fd2180b60b0 0x7fd2180d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:04.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.121+0000 7fd216a00640 1 -- 172.21.15.196:0/3198177539 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1688654262 0 0) 0x7fd1f8003520 con 0x7fd21815f680 2024-04-28T13:32:04.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.121+0000 7fd216a00640 1 -- 172.21.15.196:0/3198177539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2180d3820 msgr2=0x7fd2180b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:04.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.121+0000 7fd216a00640 1 --2- 172.21.15.196:0/3198177539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2180d3820 0x7fd2180b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:04.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.121+0000 7fd216a00640 1 -- 172.21.15.196:0/3198177539 >> v2:172.21.15.203:3300/0 conn(0x7fd2180b60b0 msgr2=0x7fd2180d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:04.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.121+0000 7fd216a00640 1 --2- 172.21.15.196:0/3198177539 >> v2:172.21.15.203:3300/0 conn(0x7fd2180b60b0 0x7fd2180d4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:04.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.121+0000 7fd216a00640 1 -- 172.21.15.196:0/3198177539 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2180b5370 con 0x7fd21815f680 2024-04-28T13:32:04.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.121+0000 7fd216a00640 1 -- 172.21.15.196:0/3198177539 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fd208001670 con 0x7fd21815f680 2024-04-28T13:32:04.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.121+0000 7fd216a00640 1 -- 172.21.15.196:0/3198177539 >> v1:172.21.15.196:6789/0 conn(0x7fd21815f680 legacy=0x7fd218163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:04.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.121+0000 7fd216a00640 1 --2- 172.21.15.196:0/3198177539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2180d3820 0x7fd1f80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:04.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.121+0000 7fd216a00640 1 --2- 172.21.15.196:0/3198177539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1f8003f30 0x7fd1f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:04.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.121+0000 7fd216a00640 1 --2- 172.21.15.196:0/3198177539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f8004d40 0x7fd1f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:04.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.121+0000 7fd216a00640 1 -- 172.21.15.196:0/3198177539 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd21815fa60 con 0x7fd1f8004d40 2024-04-28T13:32:04.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.121+0000 7fd216a00640 1 -- 172.21.15.196:0/3198177539 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd2180b5090 con 0x7fd1f8003f30 2024-04-28T13:32:04.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.121+0000 7fd216a00640 1 -- 172.21.15.196:0/3198177539 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f800b9c0 con 0x7fd2180d3820 2024-04-28T13:32:04.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.121+0000 7fd21ca00640 1 --2- 172.21.15.196:0/3198177539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f8004d40 0x7fd1f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:04.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.122+0000 7fd217e00640 1 --2- 172.21.15.196:0/3198177539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2180d3820 0x7fd1f80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:04.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.122+0000 7fd217400640 1 --2- 172.21.15.196:0/3198177539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1f8003f30 0x7fd1f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:04.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.122+0000 7fd21ca00640 1 -- 172.21.15.196:0/3198177539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2180d3820 msgr2=0x7fd1f80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:04.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.122+0000 7fd21ca00640 1 --2- 172.21.15.196:0/3198177539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2180d3820 0x7fd1f80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:04.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.122+0000 7fd21ca00640 1 -- 172.21.15.196:0/3198177539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1f8003f30 msgr2=0x7fd1f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:04.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.122+0000 7fd21ca00640 1 --2- 172.21.15.196:0/3198177539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1f8003f30 0x7fd1f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:04.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.122+0000 7fd21ca00640 1 -- 172.21.15.196:0/3198177539 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f8003520 con 0x7fd1f8004d40 2024-04-28T13:32:04.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.123+0000 7fd21ca00640 1 --2- 172.21.15.196:0/3198177539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f8004d40 0x7fd1f80051c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd210009ac0 tx=0x7fd1f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ab165817e42b7a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:04.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.123+0000 7fd216a00640 1 -- 172.21.15.196:0/3198177539 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210036070 con 0x7fd1f8004d40 2024-04-28T13:32:04.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.123+0000 7fd216a00640 1 -- 172.21.15.196:0/3198177539 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd210031400 con 0x7fd1f8004d40 2024-04-28T13:32:04.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.123+0000 7fd216a00640 1 -- 172.21.15.196:0/3198177539 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210009690 con 0x7fd1f8004d40 2024-04-28T13:32:04.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.124+0000 7fd21ea00640 1 -- 172.21.15.196:0/3198177539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f8004d40 msgr2=0x7fd1f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:04.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.124+0000 7fd21ea00640 1 --2- 172.21.15.196:0/3198177539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f8004d40 0x7fd1f80051c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd210009ac0 tx=0x7fd1f8005830 comp rx=0 tx=0).stop 2024-04-28T13:32:04.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.124+0000 7fd21ca00640 1 -- 172.21.15.196:0/3198177539 reap_dead start 2024-04-28T13:32:04.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.124+0000 7fd21ea00640 1 -- 172.21.15.196:0/3198177539 shutdown_connections 2024-04-28T13:32:04.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.124+0000 7fd21ea00640 1 -- 172.21.15.196:0/3198177539 >> 172.21.15.196:0/3198177539 conn(0x7fd21808eca0 msgr2=0x7fd21809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:04.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.124+0000 7fd21ea00640 1 -- 172.21.15.196:0/3198177539 shutdown_connections 2024-04-28T13:32:04.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.124+0000 7fd21ea00640 1 -- 172.21.15.196:0/3198177539 wait complete. 2024-04-28T13:32:04.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.126+0000 7fd21ea00640 1 Processor -- start 2024-04-28T13:32:04.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.126+0000 7fd21ea00640 1 -- start start 2024-04-28T13:32:04.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.127+0000 7fd21ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2180cb110 0x7fd2180cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:04.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.127+0000 7fd21ea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2180cc880 0x7fd2180ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:04.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.127+0000 7fd21ea00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2180cd220 0x7fd2181f8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:04.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.127+0000 7fd21ea00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd2180b5cf0 con 0x7fd2180cc880 2024-04-28T13:32:04.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.127+0000 7fd21ea00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd2180b5b70 con 0x7fd2180cb110 2024-04-28T13:32:04.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.127+0000 7fd21ea00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd2180b5e70 con 0x7fd2180cd220 2024-04-28T13:32:04.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.127+0000 7fd217400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2180cc880 0x7fd2180ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:04.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.127+0000 7fd217400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2180cc880 0x7fd2180ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34154/0 (socket says 172.21.15.196:34154) 2024-04-28T13:32:04.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.127+0000 7fd217400640 1 -- 172.21.15.196:0/2752650560 learned_addr learned my addr 172.21.15.196:0/2752650560 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:04.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.128+0000 7fd21ca00640 1 --2- 172.21.15.196:0/2752650560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2180cd220 0x7fd2181f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:04.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.128+0000 7fd217400640 1 -- 172.21.15.196:0/2752650560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2180cd220 msgr2=0x7fd2181f8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:04.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.128+0000 7fd217400640 1 --2- 172.21.15.196:0/2752650560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2180cd220 0x7fd2181f8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:04.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.128+0000 7fd217400640 1 -- 172.21.15.196:0/2752650560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2180cb110 msgr2=0x7fd2180cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:04.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.128+0000 7fd217400640 1 --2- 172.21.15.196:0/2752650560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2180cb110 0x7fd2180cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:04.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.128+0000 7fd217400640 1 -- 172.21.15.196:0/2752650560 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd218161c80 con 0x7fd2180cc880 2024-04-28T13:32:04.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.128+0000 7fd217400640 1 --2- 172.21.15.196:0/2752650560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2180cc880 0x7fd2180ccce0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd200007f00 tx=0x7fd2000115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:04.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.128+0000 7fd20fe00640 1 -- 172.21.15.196:0/2752650560 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20001a020 con 0x7fd2180cc880 2024-04-28T13:32:04.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.128+0000 7fd20fe00640 1 -- 172.21.15.196:0/2752650560 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd200016470 con 0x7fd2180cc880 2024-04-28T13:32:04.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.128+0000 7fd20fe00640 1 -- 172.21.15.196:0/2752650560 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200016860 con 0x7fd2180cc880 2024-04-28T13:32:04.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.128+0000 7fd21ea00640 1 -- 172.21.15.196:0/2752650560 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd218161f60 con 0x7fd2180cc880 2024-04-28T13:32:04.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.129+0000 7fd21ea00640 1 -- 172.21.15.196:0/2752650560 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd218162520 con 0x7fd2180cc880 2024-04-28T13:32:04.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.130+0000 7fd21ea00640 1 -- 172.21.15.196:0/2752650560 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd21806a290 con 0x7fd2180cc880 2024-04-28T13:32:04.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.131+0000 7fd20fe00640 1 -- 172.21.15.196:0/2752650560 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd200005d10 con 0x7fd2180cc880 2024-04-28T13:32:04.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.131+0000 7fd20fe00640 1 --2- 172.21.15.196:0/2752650560 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd1f406c8f0 0x7fd1f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:04.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.132+0000 7fd20fe00640 1 -- 172.21.15.196:0/2752650560 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd20008cb20 con 0x7fd2180cc880 2024-04-28T13:32:04.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.132+0000 7fd217e00640 1 --2- 172.21.15.196:0/2752650560 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd1f406c8f0 0x7fd1f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:04.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.134+0000 7fd217e00640 1 --2- 172.21.15.196:0/2752650560 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd1f406c8f0 0x7fd1f406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd208005af0 tx=0x7fd208001470 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:04.136 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.136+0000 7fd20fe00640 1 -- 172.21.15.196:0/2752650560 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd200024030 con 0x7fd2180cc880 2024-04-28T13:32:04.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.335+0000 7fd21ea00640 1 -- 172.21.15.196:0/2752650560 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd2180cd600 con 0x7fd2180cc880 2024-04-28T13:32:04.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.342+0000 7fd20fe00640 1 -- 172.21.15.196:0/2752650560 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fd20005ae30 con 0x7fd2180cc880 2024-04-28T13:32:04.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.345+0000 7fd21ea00640 1 -- 172.21.15.196:0/2752650560 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd1f406c8f0 msgr2=0x7fd1f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:04.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.345+0000 7fd21ea00640 1 --2- 172.21.15.196:0/2752650560 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd1f406c8f0 0x7fd1f406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd208005af0 tx=0x7fd208001470 comp rx=0 tx=0).stop 2024-04-28T13:32:04.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.345+0000 7fd21ea00640 1 -- 172.21.15.196:0/2752650560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2180cc880 msgr2=0x7fd2180ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:04.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.345+0000 7fd21ea00640 1 --2- 172.21.15.196:0/2752650560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2180cc880 0x7fd2180ccce0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd200007f00 tx=0x7fd2000115a0 comp rx=0 tx=0).stop 2024-04-28T13:32:04.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.346+0000 7fd21ea00640 1 -- 172.21.15.196:0/2752650560 shutdown_connections 2024-04-28T13:32:04.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.346+0000 7fd21ea00640 1 --2- 172.21.15.196:0/2752650560 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd1f406c8f0 0x7fd1f406edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:04.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.346+0000 7fd21ea00640 1 --2- 172.21.15.196:0/2752650560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2180cd220 0x7fd2181f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:04.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.346+0000 7fd21ea00640 1 --2- 172.21.15.196:0/2752650560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2180cc880 0x7fd2180ccce0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:04.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.346+0000 7fd21ea00640 1 --2- 172.21.15.196:0/2752650560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2180cb110 0x7fd2180cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:04.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.346+0000 7fd21ea00640 1 -- 172.21.15.196:0/2752650560 >> 172.21.15.196:0/2752650560 conn(0x7fd21808eca0 msgr2=0x7fd218163070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:04.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.346+0000 7fd21ea00640 1 -- 172.21.15.196:0/2752650560 shutdown_connections 2024-04-28T13:32:04.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.346+0000 7fd21ea00640 1 -- 172.21.15.196:0/2752650560 wait complete. 2024-04-28T13:32:04.364 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-28T13:32:04.365 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-28T13:32:04.462 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-28T13:32:04.570 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-28T13:32:04.570 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:32:04.571 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:04.628 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:32:04.628 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:04.682 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:32:04.682 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:04.733 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:32:04.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.928+0000 7fa528200640 1 Processor -- start 2024-04-28T13:32:04.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.928+0000 7fa528200640 1 -- start start 2024-04-28T13:32:04.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.930+0000 7fa528200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa5200d1b80 0x7fa5200d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:04.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.930+0000 7fa528200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5200d0220 con 0x7fa5200b60a0 2024-04-28T13:32:04.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.930+0000 7fa528200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5200d0500 con 0x7fa5200cfda0 2024-04-28T13:32:04.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.930+0000 7fa528200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa5200d07e0 con 0x7fa5200d1b80 2024-04-28T13:32:04.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.930+0000 7fa525800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa5200d1b80 0x7fa5200d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:04.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.930+0000 7fa525800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa5200d1b80 0x7fa5200d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34158/0 (socket says 172.21.15.196:34158) 2024-04-28T13:32:04.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.930+0000 7fa525800640 1 -- 172.21.15.196:0/4096527948 learned_addr learned my addr 172.21.15.196:0/4096527948 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:04.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.931+0000 7fa525800640 1 -- 172.21.15.196:0/4096527948 >> v1:172.21.15.203:6789/0 conn(0x7fa5200cfda0 legacy=0x7fa5200d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:04.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.931+0000 7fa525800640 1 -- 172.21.15.196:0/4096527948 >> v1:172.21.15.196:6789/0 conn(0x7fa5200b60a0 legacy=0x7fa5200b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:04.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.931+0000 7fa525800640 1 -- 172.21.15.196:0/4096527948 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5200d0ac0 con 0x7fa5200d1b80 2024-04-28T13:32:04.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.931+0000 7fa51fe00640 1 -- 172.21.15.196:0/4096527948 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2328516258 0 0) 0x7fa5200d0220 con 0x7fa5200b60a0 2024-04-28T13:32:04.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.932+0000 7fa525800640 1 --2- 172.21.15.196:0/4096527948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5200d1b80 0x7fa5200d1f60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa514009870 tx=0x7fa51402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f1533a7d5b67dc server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:04.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.932+0000 7fa51fe00640 1 -- 172.21.15.196:0/4096527948 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514037070 con 0x7fa5200d1b80 2024-04-28T13:32:04.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.932+0000 7fa51fe00640 1 -- 172.21.15.196:0/4096527948 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa51402ae50 con 0x7fa5200d1b80 2024-04-28T13:32:04.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.932+0000 7fa51fe00640 1 -- 172.21.15.196:0/4096527948 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514032860 con 0x7fa5200d1b80 2024-04-28T13:32:04.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.933+0000 7fa528200640 1 -- 172.21.15.196:0/4096527948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5200d1b80 msgr2=0x7fa5200d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:04.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.933+0000 7fa528200640 1 --2- 172.21.15.196:0/4096527948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5200d1b80 0x7fa5200d1f60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa514009870 tx=0x7fa51402a300 comp rx=0 tx=0).stop 2024-04-28T13:32:04.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.933+0000 7fa528200640 1 -- 172.21.15.196:0/4096527948 shutdown_connections 2024-04-28T13:32:04.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.933+0000 7fa528200640 1 --2- 172.21.15.196:0/4096527948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5200d1b80 0x7fa5200d1f60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:04.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.933+0000 7fa528200640 1 -- 172.21.15.196:0/4096527948 >> 172.21.15.196:0/4096527948 conn(0x7fa52009e540 msgr2=0x7fa5200d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:04.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.933+0000 7fa528200640 1 -- 172.21.15.196:0/4096527948 shutdown_connections 2024-04-28T13:32:04.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.933+0000 7fa528200640 1 -- 172.21.15.196:0/4096527948 wait complete. 2024-04-28T13:32:04.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.935+0000 7fa528200640 1 Processor -- start 2024-04-28T13:32:04.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.935+0000 7fa528200640 1 -- start start 2024-04-28T13:32:04.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.936+0000 7fa528200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5200b60a0 0x7fa52015c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:04.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.936+0000 7fa528200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5200cfda0 0x7fa5201f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:04.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.936+0000 7fa528200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5200d1b80 0x7fa52015d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:04.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.936+0000 7fa528200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5200b9060 con 0x7fa5200d1b80 2024-04-28T13:32:04.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.936+0000 7fa528200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa5200d4170 con 0x7fa5200cfda0 2024-04-28T13:32:04.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.936+0000 7fa528200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa5200b91e0 con 0x7fa5200b60a0 2024-04-28T13:32:04.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.936+0000 7fa526200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5200d1b80 0x7fa52015d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:04.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.936+0000 7fa526200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5200d1b80 0x7fa52015d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34170/0 (socket says 172.21.15.196:34170) 2024-04-28T13:32:04.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.936+0000 7fa526200640 1 -- 172.21.15.196:0/4261557895 learned_addr learned my addr 172.21.15.196:0/4261557895 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:04.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.936+0000 7fa524e00640 1 --2- 172.21.15.196:0/4261557895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5200cfda0 0x7fa5201f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:04.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.936+0000 7fa526200640 1 -- 172.21.15.196:0/4261557895 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5200b60a0 msgr2=0x7fa52015c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:04.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.936+0000 7fa526200640 1 --2- 172.21.15.196:0/4261557895 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5200b60a0 0x7fa52015c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:04.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.936+0000 7fa526200640 1 -- 172.21.15.196:0/4261557895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5200cfda0 msgr2=0x7fa5201f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:04.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.936+0000 7fa526200640 1 --2- 172.21.15.196:0/4261557895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5200cfda0 0x7fa5201f4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:04.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.936+0000 7fa526200640 1 -- 172.21.15.196:0/4261557895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52015ccc0 con 0x7fa5200d1b80 2024-04-28T13:32:04.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.937+0000 7fa524e00640 1 --2- 172.21.15.196:0/4261557895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5200cfda0 0x7fa5201f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:04.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.937+0000 7fa526200640 1 --2- 172.21.15.196:0/4261557895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5200d1b80 0x7fa52015d8d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa51800c790 tx=0x7fa51800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:04.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.937+0000 7fa51d600640 1 -- 172.21.15.196:0/4261557895 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518003da0 con 0x7fa5200d1b80 2024-04-28T13:32:04.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.937+0000 7fa51d600640 1 -- 172.21.15.196:0/4261557895 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa518005070 con 0x7fa5200d1b80 2024-04-28T13:32:04.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.937+0000 7fa528200640 1 -- 172.21.15.196:0/4261557895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5200c8dd0 con 0x7fa5200d1b80 2024-04-28T13:32:04.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.937+0000 7fa51d600640 1 -- 172.21.15.196:0/4261557895 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518005460 con 0x7fa5200d1b80 2024-04-28T13:32:04.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.937+0000 7fa528200640 1 -- 172.21.15.196:0/4261557895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5200c9390 con 0x7fa5200d1b80 2024-04-28T13:32:04.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.938+0000 7fa528200640 1 -- 172.21.15.196:0/4261557895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52006a290 con 0x7fa5200d1b80 2024-04-28T13:32:04.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.939+0000 7fa51d600640 1 -- 172.21.15.196:0/4261557895 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa518005600 con 0x7fa5200d1b80 2024-04-28T13:32:04.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.940+0000 7fa51d600640 1 --2- 172.21.15.196:0/4261557895 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa50006c8f0 0x7fa50006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:04.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.941+0000 7fa51d600640 1 -- 172.21.15.196:0/4261557895 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa51808ba40 con 0x7fa5200d1b80 2024-04-28T13:32:04.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.941+0000 7fa525800640 1 --2- 172.21.15.196:0/4261557895 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa50006c8f0 0x7fa50006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:04.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.942+0000 7fa525800640 1 --2- 172.21.15.196:0/4261557895 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa50006c8f0 0x7fa50006edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa514034000 tx=0x7fa51402c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:04.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:04.945+0000 7fa51d600640 1 -- 172.21.15.196:0/4261557895 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa52015ccc0 con 0x7fa5200d1b80 2024-04-28T13:32:05.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.147+0000 7fa528200640 1 -- 172.21.15.196:0/4261557895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5200d27b0 con 0x7fa5200d1b80 2024-04-28T13:32:05.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.150+0000 7fa51d600640 1 -- 172.21.15.196:0/4261557895 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16890 (secure 0 0 0) 0x7fa51808be20 con 0x7fa5200d1b80 2024-04-28T13:32:05.151 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:05.151 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:03.882452+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:32:05.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.153+0000 7fa528200640 1 -- 172.21.15.196:0/4261557895 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa50006c8f0 msgr2=0x7fa50006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:05.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.153+0000 7fa528200640 1 --2- 172.21.15.196:0/4261557895 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa50006c8f0 0x7fa50006edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa514034000 tx=0x7fa51402c000 comp rx=0 tx=0).stop 2024-04-28T13:32:05.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.153+0000 7fa528200640 1 -- 172.21.15.196:0/4261557895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5200d1b80 msgr2=0x7fa52015d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:05.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.153+0000 7fa528200640 1 --2- 172.21.15.196:0/4261557895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5200d1b80 0x7fa52015d8d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa51800c790 tx=0x7fa51800cc50 comp rx=0 tx=0).stop 2024-04-28T13:32:05.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.154+0000 7fa528200640 1 -- 172.21.15.196:0/4261557895 shutdown_connections 2024-04-28T13:32:05.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.154+0000 7fa528200640 1 --2- 172.21.15.196:0/4261557895 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa50006c8f0 0x7fa50006edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.154+0000 7fa528200640 1 --2- 172.21.15.196:0/4261557895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5200d1b80 0x7fa52015d8d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.154+0000 7fa528200640 1 --2- 172.21.15.196:0/4261557895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5200cfda0 0x7fa5201f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.154+0000 7fa528200640 1 --2- 172.21.15.196:0/4261557895 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5200b60a0 0x7fa52015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.154+0000 7fa528200640 1 -- 172.21.15.196:0/4261557895 >> 172.21.15.196:0/4261557895 conn(0x7fa52009e540 msgr2=0x7fa5200b9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:05.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.154+0000 7fa528200640 1 -- 172.21.15.196:0/4261557895 shutdown_connections 2024-04-28T13:32:05.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.154+0000 7fa528200640 1 -- 172.21.15.196:0/4261557895 wait complete. 2024-04-28T13:32:05.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:31:19.280975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:31:20.037066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:32:05.174 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:32:05.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.379+0000 7f82f3e00640 1 Processor -- start 2024-04-28T13:32:05.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.379+0000 7f82f3e00640 1 -- start start 2024-04-28T13:32:05.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.381+0000 7f82f3e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f82f406e8f0 0x7f82f40b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:05.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.381+0000 7f82f3e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f415d700 con 0x7f82f40d1480 2024-04-28T13:32:05.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.381+0000 7f82f3e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f415d9e0 con 0x7f82f40d10a0 2024-04-28T13:32:05.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.381+0000 7f82f3e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f82f415e310 con 0x7f82f406e8f0 2024-04-28T13:32:05.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.382+0000 7f82ebe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f82f406e8f0 0x7f82f40b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:05.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.382+0000 7f82f3400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f82f40d1480 0x7f82f4160c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43366/0 (socket says 172.21.15.196:43366) 2024-04-28T13:32:05.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.382+0000 7f82f3400640 1 -- 172.21.15.196:0/2812833482 learned_addr learned my addr 172.21.15.196:0/2812833482 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:05.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.382+0000 7f82f2000640 1 -- 172.21.15.196:0/2812833482 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3796221249 0 0) 0x7f82f415d700 con 0x7f82f40d1480 2024-04-28T13:32:05.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.382+0000 7f82ebe00640 1 -- 172.21.15.196:0/2812833482 >> v1:172.21.15.203:6789/0 conn(0x7f82f40d10a0 legacy=0x7f82f40b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:05.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.382+0000 7f82ebe00640 1 -- 172.21.15.196:0/2812833482 >> v1:172.21.15.196:6789/0 conn(0x7f82f40d1480 legacy=0x7f82f4160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:05.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.382+0000 7f82ebe00640 1 -- 172.21.15.196:0/2812833482 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f415dcc0 con 0x7f82f406e8f0 2024-04-28T13:32:05.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.383+0000 7f82ebe00640 1 --2- 172.21.15.196:0/2812833482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f406e8f0 0x7f82f40b3c90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0097a0 tx=0x7f82dc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501b22f18abb0713 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:05.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.383+0000 7f82f2000640 1 -- 172.21.15.196:0/2812833482 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc037070 con 0x7f82f406e8f0 2024-04-28T13:32:05.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.383+0000 7f82f2000640 1 -- 172.21.15.196:0/2812833482 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82dc02ae50 con 0x7f82f406e8f0 2024-04-28T13:32:05.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.383+0000 7f82f2000640 1 -- 172.21.15.196:0/2812833482 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc032660 con 0x7f82f406e8f0 2024-04-28T13:32:05.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.384+0000 7f82f3e00640 1 -- 172.21.15.196:0/2812833482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f406e8f0 msgr2=0x7f82f40b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:05.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.384+0000 7f82f3e00640 1 --2- 172.21.15.196:0/2812833482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f406e8f0 0x7f82f40b3c90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0097a0 tx=0x7f82dc02a140 comp rx=0 tx=0).stop 2024-04-28T13:32:05.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.384+0000 7f82f3e00640 1 -- 172.21.15.196:0/2812833482 shutdown_connections 2024-04-28T13:32:05.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.384+0000 7f82f3e00640 1 --2- 172.21.15.196:0/2812833482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f406e8f0 0x7f82f40b3c90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.384+0000 7f82f3e00640 1 -- 172.21.15.196:0/2812833482 >> 172.21.15.196:0/2812833482 conn(0x7f82f406c6c0 msgr2=0x7f82f406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:05.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.384+0000 7f82f3e00640 1 -- 172.21.15.196:0/2812833482 shutdown_connections 2024-04-28T13:32:05.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.384+0000 7f82f3e00640 1 -- 172.21.15.196:0/2812833482 wait complete. 2024-04-28T13:32:05.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.386+0000 7f82f3e00640 1 Processor -- start 2024-04-28T13:32:05.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.386+0000 7f82f3e00640 1 -- start start 2024-04-28T13:32:05.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.387+0000 7f82f3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82f406e8f0 0x7f82f414eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:05.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.387+0000 7f82f3e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82f40d10a0 0x7f82f40cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:05.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.387+0000 7f82f3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f40d1480 0x7f82f40cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:05.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.387+0000 7f82f3e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82f40d1a30 con 0x7f82f40d1480 2024-04-28T13:32:05.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.387+0000 7f82f3e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f82f40d18b0 con 0x7f82f406e8f0 2024-04-28T13:32:05.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.387+0000 7f82f3e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f82f41613a0 con 0x7f82f40d10a0 2024-04-28T13:32:05.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.387+0000 7f82f3400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f40d1480 0x7f82f40cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:05.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.387+0000 7f82f3400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f40d1480 0x7f82f40cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34186/0 (socket says 172.21.15.196:34186) 2024-04-28T13:32:05.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.387+0000 7f82f3400640 1 -- 172.21.15.196:0/1866405292 learned_addr learned my addr 172.21.15.196:0/1866405292 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:05.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.388+0000 7f82f2a00640 1 --2- 172.21.15.196:0/1866405292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82f406e8f0 0x7f82f414eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:05.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.388+0000 7f82ebe00640 1 --2- 172.21.15.196:0/1866405292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82f40d10a0 0x7f82f40cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:05.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.388+0000 7f82f3400640 1 -- 172.21.15.196:0/1866405292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82f40d10a0 msgr2=0x7f82f40cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:05.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.388+0000 7f82f3400640 1 --2- 172.21.15.196:0/1866405292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82f40d10a0 0x7f82f40cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.388+0000 7f82f3400640 1 -- 172.21.15.196:0/1866405292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82f406e8f0 msgr2=0x7f82f414eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:05.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.388+0000 7f82f3400640 1 --2- 172.21.15.196:0/1866405292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82f406e8f0 0x7f82f414eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.388+0000 7f82f3400640 1 -- 172.21.15.196:0/1866405292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f40d0510 con 0x7f82f40d1480 2024-04-28T13:32:05.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.388+0000 7f82ebe00640 1 --2- 172.21.15.196:0/1866405292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82f40d10a0 0x7f82f40cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:05.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.388+0000 7f82f3400640 1 --2- 172.21.15.196:0/1866405292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f40d1480 0x7f82f40cc340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c700 tx=0x7f82ec00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:05.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.388+0000 7f82eaa00640 1 -- 172.21.15.196:0/1866405292 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec018070 con 0x7f82f40d1480 2024-04-28T13:32:05.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.388+0000 7f82eaa00640 1 -- 172.21.15.196:0/1866405292 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec00ce10 con 0x7f82f40d1480 2024-04-28T13:32:05.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.388+0000 7f82f3e00640 1 -- 172.21.15.196:0/1866405292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f40ce660 con 0x7f82f40d1480 2024-04-28T13:32:05.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.388+0000 7f82eaa00640 1 -- 172.21.15.196:0/1866405292 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec00a800 con 0x7f82f40d1480 2024-04-28T13:32:05.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.389+0000 7f82f3e00640 1 -- 172.21.15.196:0/1866405292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f40cbb40 con 0x7f82f40d1480 2024-04-28T13:32:05.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.390+0000 7f82f3e00640 1 -- 172.21.15.196:0/1866405292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f4161a00 con 0x7f82f40d1480 2024-04-28T13:32:05.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.391+0000 7f82eaa00640 1 -- 172.21.15.196:0/1866405292 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82ec01b070 con 0x7f82f40d1480 2024-04-28T13:32:05.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.392+0000 7f82eaa00640 1 --2- 172.21.15.196:0/1866405292 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f82d006c8f0 0x7f82d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:05.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.392+0000 7f82eaa00640 1 -- 172.21.15.196:0/1866405292 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f82ec08b0e0 con 0x7f82f40d1480 2024-04-28T13:32:05.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.393+0000 7f82f2a00640 1 --2- 172.21.15.196:0/1866405292 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f82d006c8f0 0x7f82d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:05.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.394+0000 7f82f2a00640 1 --2- 172.21.15.196:0/1866405292 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f82d006c8f0 0x7f82d006edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f82e4004ea0 tx=0x7f82e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:05.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.397+0000 7f82eaa00640 1 -- 172.21.15.196:0/1866405292 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f82f4161a00 con 0x7f82f40d1480 2024-04-28T13:32:05.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.597+0000 7f82f3e00640 1 -- 172.21.15.196:0/1866405292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f82f40cb650 con 0x7f82f40d1480 2024-04-28T13:32:05.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.597+0000 7f82eaa00640 1 -- 172.21.15.196:0/1866405292 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f82ec059490 con 0x7f82f40d1480 2024-04-28T13:32:05.598 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:32:05.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.600+0000 7f82f3e00640 1 -- 172.21.15.196:0/1866405292 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f82d006c8f0 msgr2=0x7f82d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:05.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.600+0000 7f82f3e00640 1 --2- 172.21.15.196:0/1866405292 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f82d006c8f0 0x7f82d006edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f82e4004ea0 tx=0x7f82e4009040 comp rx=0 tx=0).stop 2024-04-28T13:32:05.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.600+0000 7f82f3e00640 1 -- 172.21.15.196:0/1866405292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f40d1480 msgr2=0x7f82f40cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:05.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.600+0000 7f82f3e00640 1 --2- 172.21.15.196:0/1866405292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f40d1480 0x7f82f40cc340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c700 tx=0x7f82ec00cbc0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.600+0000 7f82f3e00640 1 -- 172.21.15.196:0/1866405292 shutdown_connections 2024-04-28T13:32:05.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.600+0000 7f82f3e00640 1 --2- 172.21.15.196:0/1866405292 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f82d006c8f0 0x7f82d006edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.600+0000 7f82f3e00640 1 --2- 172.21.15.196:0/1866405292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82f40d1480 0x7f82f40cc340 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.600+0000 7f82f3e00640 1 --2- 172.21.15.196:0/1866405292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82f40d10a0 0x7f82f40cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.600+0000 7f82f3e00640 1 --2- 172.21.15.196:0/1866405292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82f406e8f0 0x7f82f414eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.600+0000 7f82f3e00640 1 -- 172.21.15.196:0/1866405292 >> 172.21.15.196:0/1866405292 conn(0x7f82f406c6c0 msgr2=0x7f82f406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:05.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.601+0000 7f82f3e00640 1 -- 172.21.15.196:0/1866405292 shutdown_connections 2024-04-28T13:32:05.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.601+0000 7f82f3e00640 1 -- 172.21.15.196:0/1866405292 wait complete. 2024-04-28T13:32:05.620 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:32:05.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.822+0000 7f22a1c00640 1 Processor -- start 2024-04-28T13:32:05.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.823+0000 7f22a1c00640 1 -- start start 2024-04-28T13:32:05.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.824+0000 7f22a1c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229c146830 0x7f229c06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:05.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.824+0000 7f22a1c00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f229c06c7e0 0x7f229c0d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:05.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.824+0000 7f22a1c00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f229c071970 con 0x7f229c0d0bc0 2024-04-28T13:32:05.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.824+0000 7f22a1c00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f229c06cc40 con 0x7f229c06c7e0 2024-04-28T13:32:05.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.824+0000 7f22a1c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f229c0d1050 con 0x7f229c146830 2024-04-28T13:32:05.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.825+0000 7f229aa00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229c146830 0x7f229c06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:05.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.825+0000 7f229aa00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229c146830 0x7f229c06c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:59522/0 (socket says 172.21.15.196:59522) 2024-04-28T13:32:05.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.825+0000 7f229aa00640 1 -- 172.21.15.196:0/666959282 learned_addr learned my addr 172.21.15.196:0/666959282 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:05.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.825+0000 7f229a000640 1 --2- 172.21.15.196:0/666959282 >> v2:172.21.15.203:3300/0 conn(0x7f229c06c7e0 0x7f229c0d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:05.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.826+0000 7f229aa00640 1 -- 172.21.15.196:0/666959282 >> v2:172.21.15.203:3300/0 conn(0x7f229c06c7e0 msgr2=0x7f229c0d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:05.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.826+0000 7f229aa00640 1 --2- 172.21.15.196:0/666959282 >> v2:172.21.15.203:3300/0 conn(0x7f229c06c7e0 0x7f229c0d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.826+0000 7f229aa00640 1 -- 172.21.15.196:0/666959282 >> v1:172.21.15.203:6789/0 conn(0x7f229c0d0bc0 legacy=0x7f229c0731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:05.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.826+0000 7f229aa00640 1 -- 172.21.15.196:0/666959282 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c0d12f0 con 0x7f229c146830 2024-04-28T13:32:05.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.826+0000 7f2299600640 1 -- 172.21.15.196:0/666959282 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2166304623 0 0) 0x7f229c071970 con 0x7f229c0d0bc0 2024-04-28T13:32:05.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.828+0000 7f229aa00640 1 --2- 172.21.15.196:0/666959282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229c146830 0x7f229c06c2a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f228c009870 tx=0x7f228c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=275798323234401b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:05.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.828+0000 7f2299600640 1 -- 172.21.15.196:0/666959282 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c037070 con 0x7f229c146830 2024-04-28T13:32:05.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.828+0000 7f2299600640 1 -- 172.21.15.196:0/666959282 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f228c032440 con 0x7f229c146830 2024-04-28T13:32:05.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.828+0000 7f2299600640 1 -- 172.21.15.196:0/666959282 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c032810 con 0x7f229c146830 2024-04-28T13:32:05.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.829+0000 7f22a1c00640 1 -- 172.21.15.196:0/666959282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229c146830 msgr2=0x7f229c06c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:05.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.829+0000 7f22a1c00640 1 --2- 172.21.15.196:0/666959282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229c146830 0x7f229c06c2a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f228c009870 tx=0x7f228c02a250 comp rx=0 tx=0).stop 2024-04-28T13:32:05.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.829+0000 7f22a1c00640 1 -- 172.21.15.196:0/666959282 shutdown_connections 2024-04-28T13:32:05.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.829+0000 7f22a1c00640 1 --2- 172.21.15.196:0/666959282 >> v2:172.21.15.203:3300/0 conn(0x7f229c06c7e0 0x7f229c0d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.829+0000 7f22a1c00640 1 --2- 172.21.15.196:0/666959282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229c146830 0x7f229c06c2a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.829+0000 7f22a1c00640 1 -- 172.21.15.196:0/666959282 >> 172.21.15.196:0/666959282 conn(0x7f229c0b3b60 msgr2=0x7f229c0b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:05.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.829+0000 7f22a1c00640 1 -- 172.21.15.196:0/666959282 shutdown_connections 2024-04-28T13:32:05.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.829+0000 7f22a1c00640 1 -- 172.21.15.196:0/666959282 wait complete. 2024-04-28T13:32:05.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.831+0000 7f22a1c00640 1 Processor -- start 2024-04-28T13:32:05.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.831+0000 7f22a1c00640 1 -- start start 2024-04-28T13:32:05.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.832+0000 7f22a1c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f229c06c7e0 0x7f229c13cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:05.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.832+0000 7f22a1c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f229c0d0bc0 0x7f229c142c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:05.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.832+0000 7f22a1c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229c146830 0x7f229c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:05.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.832+0000 7f22a1c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f229c0d1b90 con 0x7f229c06c7e0 2024-04-28T13:32:05.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.832+0000 7f22a1c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f229c0d18b0 con 0x7f229c0d0bc0 2024-04-28T13:32:05.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.832+0000 7f22a1c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f229c06f500 con 0x7f229c146830 2024-04-28T13:32:05.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.832+0000 7f229aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f229c06c7e0 0x7f229c13cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:05.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.832+0000 7f229aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f229c06c7e0 0x7f229c13cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34194/0 (socket says 172.21.15.196:34194) 2024-04-28T13:32:05.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.832+0000 7f229aa00640 1 -- 172.21.15.196:0/882557172 learned_addr learned my addr 172.21.15.196:0/882557172 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:05.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.832+0000 7f229a000640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f229c0d0bc0 0x7f229c142c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:05.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.832+0000 7f229aa00640 1 -- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229c146830 msgr2=0x7f229c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:05.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.832+0000 7f229b400640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229c146830 0x7f229c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:05.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.832+0000 7f229aa00640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229c146830 0x7f229c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.833+0000 7f229aa00640 1 -- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f229c0d0bc0 msgr2=0x7f229c142c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:05.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.833+0000 7f229aa00640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f229c0d0bc0 0x7f229c142c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:05.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.833+0000 7f229aa00640 1 -- 172.21.15.196:0/882557172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c160350 con 0x7f229c06c7e0 2024-04-28T13:32:05.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.833+0000 7f229a000640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f229c0d0bc0 0x7f229c142c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:05.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.833+0000 7f229b400640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229c146830 0x7f229c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:05.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.833+0000 7f229aa00640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f229c06c7e0 0x7f229c13cad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f228c02a290 tx=0x7f228c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:05.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.833+0000 7f2292a00640 1 -- 172.21.15.196:0/882557172 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c041030 con 0x7f229c06c7e0 2024-04-28T13:32:05.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.833+0000 7f2292a00640 1 -- 172.21.15.196:0/882557172 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f228c030440 con 0x7f229c06c7e0 2024-04-28T13:32:05.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.833+0000 7f22a1c00640 1 -- 172.21.15.196:0/882557172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229c0b0f60 con 0x7f229c06c7e0 2024-04-28T13:32:05.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.833+0000 7f2292a00640 1 -- 172.21.15.196:0/882557172 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c037040 con 0x7f229c06c7e0 2024-04-28T13:32:05.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.833+0000 7f22a1c00640 1 -- 172.21.15.196:0/882557172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229c13e460 con 0x7f229c06c7e0 2024-04-28T13:32:05.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.834+0000 7f22a1c00640 1 -- 172.21.15.196:0/882557172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229c0693d0 con 0x7f229c06c7e0 2024-04-28T13:32:05.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.835+0000 7f2292a00640 1 -- 172.21.15.196:0/882557172 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f228c044070 con 0x7f229c06c7e0 2024-04-28T13:32:05.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.836+0000 7f2292a00640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f227806c8f0 0x7f227806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:05.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.836+0000 7f2292a00640 1 -- 172.21.15.196:0/882557172 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f228c0abf00 con 0x7f229c06c7e0 2024-04-28T13:32:05.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.837+0000 7f229a000640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f227806c8f0 0x7f227806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:05.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.838+0000 7f229a000640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f227806c8f0 0x7f227806edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f229c13d010 tx=0x7f2284008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:05.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:05.841+0000 7f2292a00640 1 -- 172.21.15.196:0/882557172 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f228c04b070 con 0x7f229c06c7e0 2024-04-28T13:32:06.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.041+0000 7f22a1c00640 1 -- 172.21.15.196:0/882557172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f229c0b57f0 con 0x7f229c06c7e0 2024-04-28T13:32:06.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.042+0000 7f2292a00640 1 -- 172.21.15.196:0/882557172 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f229c160350 con 0x7f229c06c7e0 2024-04-28T13:32:06.042 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:32:06.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.045+0000 7f22a1c00640 1 -- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f227806c8f0 msgr2=0x7f227806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:06.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.045+0000 7f22a1c00640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f227806c8f0 0x7f227806edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f229c13d010 tx=0x7f2284008040 comp rx=0 tx=0).stop 2024-04-28T13:32:06.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.045+0000 7f22a1c00640 1 -- 172.21.15.196:0/882557172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f229c06c7e0 msgr2=0x7f229c13cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:06.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.045+0000 7f22a1c00640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f229c06c7e0 0x7f229c13cad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f228c02a290 tx=0x7f228c0046c0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.045+0000 7f22a1c00640 1 -- 172.21.15.196:0/882557172 shutdown_connections 2024-04-28T13:32:06.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.045+0000 7f22a1c00640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f227806c8f0 0x7f227806edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.045+0000 7f22a1c00640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229c146830 0x7f229c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.045+0000 7f22a1c00640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f229c0d0bc0 0x7f229c142c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.045+0000 7f22a1c00640 1 --2- 172.21.15.196:0/882557172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f229c06c7e0 0x7f229c13cad0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.045+0000 7f22a1c00640 1 -- 172.21.15.196:0/882557172 >> 172.21.15.196:0/882557172 conn(0x7f229c0b3b60 msgr2=0x7f229c0731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:06.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.045+0000 7f22a1c00640 1 -- 172.21.15.196:0/882557172 shutdown_connections 2024-04-28T13:32:06.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.045+0000 7f22a1c00640 1 -- 172.21.15.196:0/882557172 wait complete. 2024-04-28T13:32:06.065 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:32:06.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.275+0000 7f7133200640 1 Processor -- start 2024-04-28T13:32:06.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.275+0000 7f7133200640 1 -- start start 2024-04-28T13:32:06.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.276+0000 7f7133200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f713406f470 0x7f71340b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:06.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.276+0000 7f7133200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7134071ef0 0x7f7134072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:06.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.276+0000 7f7133200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f713415f6b0 0x7f713415fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:06.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.276+0000 7f7133200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f71340b46d0 con 0x7f713406f470 2024-04-28T13:32:06.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.277+0000 7f7133200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f71340741e0 con 0x7f713415f6b0 2024-04-28T13:32:06.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.277+0000 7f7133200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7134160080 con 0x7f7134071ef0 2024-04-28T13:32:06.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.277+0000 7f7131e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f713406f470 0x7f71340b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:06.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.277+0000 7f7131e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f713406f470 0x7f71340b4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34198/0 (socket says 172.21.15.196:34198) 2024-04-28T13:32:06.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.277+0000 7f7131e00640 1 -- 172.21.15.196:0/3930089599 learned_addr learned my addr 172.21.15.196:0/3930089599 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:06.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.277+0000 7f7131e00640 1 -- 172.21.15.196:0/3930089599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7134071ef0 msgr2=0x7f7134072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:06.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.277+0000 7f712be00640 1 --2- 172.21.15.196:0/3930089599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7134071ef0 0x7f7134072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:06.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.277+0000 7f7132800640 1 --2- 172.21.15.196:0/3930089599 >> v2:172.21.15.203:3300/0 conn(0x7f713415f6b0 0x7f713415fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:06.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.277+0000 7f7131e00640 1 --2- 172.21.15.196:0/3930089599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7134071ef0 0x7f7134072370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.277+0000 7f7131e00640 1 -- 172.21.15.196:0/3930089599 >> v2:172.21.15.203:3300/0 conn(0x7f713415f6b0 msgr2=0x7f713415fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:06.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.277+0000 7f7131e00640 1 --2- 172.21.15.196:0/3930089599 >> v2:172.21.15.203:3300/0 conn(0x7f713415f6b0 0x7f713415fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.277+0000 7f7131e00640 1 -- 172.21.15.196:0/3930089599 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71340b50a0 con 0x7f713406f470 2024-04-28T13:32:06.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.277+0000 7f7132800640 1 --2- 172.21.15.196:0/3930089599 >> v2:172.21.15.203:3300/0 conn(0x7f713415f6b0 0x7f713415fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:06.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.278+0000 7f7131e00640 1 --2- 172.21.15.196:0/3930089599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f713406f470 0x7f71340b4b60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f71340b6060 tx=0x7f712402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40552556aab95717 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:06.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.279+0000 7f7131400640 1 -- 172.21.15.196:0/3930089599 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124037070 con 0x7f713406f470 2024-04-28T13:32:06.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.279+0000 7f7131400640 1 -- 172.21.15.196:0/3930089599 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7124032440 con 0x7f713406f470 2024-04-28T13:32:06.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.279+0000 7f7131400640 1 -- 172.21.15.196:0/3930089599 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124032810 con 0x7f713406f470 2024-04-28T13:32:06.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.279+0000 7f7133200640 1 -- 172.21.15.196:0/3930089599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f713406f470 msgr2=0x7f71340b4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:06.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.279+0000 7f7133200640 1 --2- 172.21.15.196:0/3930089599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f713406f470 0x7f71340b4b60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f71340b6060 tx=0x7f712402a1e0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.280+0000 7f7133200640 1 -- 172.21.15.196:0/3930089599 shutdown_connections 2024-04-28T13:32:06.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.280+0000 7f7133200640 1 --2- 172.21.15.196:0/3930089599 >> v2:172.21.15.203:3300/0 conn(0x7f713415f6b0 0x7f713415fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.280+0000 7f7133200640 1 --2- 172.21.15.196:0/3930089599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7134071ef0 0x7f7134072370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.280+0000 7f7133200640 1 --2- 172.21.15.196:0/3930089599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f713406f470 0x7f71340b4b60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.280+0000 7f7133200640 1 -- 172.21.15.196:0/3930089599 >> 172.21.15.196:0/3930089599 conn(0x7f71340b2990 msgr2=0x7f71340b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:06.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.280+0000 7f7133200640 1 -- 172.21.15.196:0/3930089599 shutdown_connections 2024-04-28T13:32:06.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.280+0000 7f7133200640 1 -- 172.21.15.196:0/3930089599 wait complete. 2024-04-28T13:32:06.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.281+0000 7f7133200640 1 Processor -- start 2024-04-28T13:32:06.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.282+0000 7f7133200640 1 -- start start 2024-04-28T13:32:06.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.283+0000 7f7133200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f713406f470 0x7f71341f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:06.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.283+0000 7f7133200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7134071ef0 0x7f713415ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:06.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.283+0000 7f7133200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f713415f6b0 0x7f713409d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:06.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.283+0000 7f7133200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f71340b4080 con 0x7f713415f6b0 2024-04-28T13:32:06.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.283+0000 7f7133200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f71341605e0 con 0x7f7134071ef0 2024-04-28T13:32:06.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.283+0000 7f7133200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f71340b4360 con 0x7f713406f470 2024-04-28T13:32:06.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.283+0000 7f7132800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f713415f6b0 0x7f713409d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:06.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.283+0000 7f7132800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f713415f6b0 0x7f713409d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34210/0 (socket says 172.21.15.196:34210) 2024-04-28T13:32:06.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.283+0000 7f7132800640 1 -- 172.21.15.196:0/1529350385 learned_addr learned my addr 172.21.15.196:0/1529350385 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:06.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.283+0000 7f7131e00640 1 --2- 172.21.15.196:0/1529350385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f713406f470 0x7f71341f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:06.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.283+0000 7f7132800640 1 -- 172.21.15.196:0/1529350385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f713406f470 msgr2=0x7f71341f7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:06.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.283+0000 7f7132800640 1 --2- 172.21.15.196:0/1529350385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f713406f470 0x7f71341f7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.283+0000 7f7132800640 1 -- 172.21.15.196:0/1529350385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7134071ef0 msgr2=0x7f713415ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:06.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.283+0000 7f7132800640 1 --2- 172.21.15.196:0/1529350385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7134071ef0 0x7f713415ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.283+0000 7f7132800640 1 -- 172.21.15.196:0/1529350385 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713409dca0 con 0x7f713415f6b0 2024-04-28T13:32:06.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.284+0000 7f7132800640 1 --2- 172.21.15.196:0/1529350385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f713415f6b0 0x7f713409d760 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f712c00cca0 tx=0x7f712c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:06.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.284+0000 7f712a000640 1 -- 172.21.15.196:0/1529350385 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c00f680 con 0x7f713415f6b0 2024-04-28T13:32:06.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.284+0000 7f712a000640 1 -- 172.21.15.196:0/1529350385 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712c00f840 con 0x7f713415f6b0 2024-04-28T13:32:06.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.284+0000 7f7133200640 1 -- 172.21.15.196:0/1529350385 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71341f8560 con 0x7f713415f6b0 2024-04-28T13:32:06.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.284+0000 7f712a000640 1 -- 172.21.15.196:0/1529350385 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c00fc30 con 0x7f713415f6b0 2024-04-28T13:32:06.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.284+0000 7f7133200640 1 -- 172.21.15.196:0/1529350385 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71341f8b20 con 0x7f713415f6b0 2024-04-28T13:32:06.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.286+0000 7f7133200640 1 -- 172.21.15.196:0/1529350385 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7134050730 con 0x7f713415f6b0 2024-04-28T13:32:06.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.286+0000 7f712a000640 1 -- 172.21.15.196:0/1529350385 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f712c0213f0 con 0x7f713415f6b0 2024-04-28T13:32:06.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.287+0000 7f712a000640 1 --2- 172.21.15.196:0/1529350385 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f711006c8f0 0x7f711006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:06.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.287+0000 7f712a000640 1 -- 172.21.15.196:0/1529350385 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f712c08c850 con 0x7f713415f6b0 2024-04-28T13:32:06.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.288+0000 7f7131e00640 1 --2- 172.21.15.196:0/1529350385 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f711006c8f0 0x7f711006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:06.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.289+0000 7f7131e00640 1 --2- 172.21.15.196:0/1529350385 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f711006c8f0 0x7f711006edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7124034000 tx=0x7f712402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:06.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.292+0000 7f712a000640 1 -- 172.21.15.196:0/1529350385 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f712c014030 con 0x7f713415f6b0 2024-04-28T13:32:06.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.493+0000 7f7133200640 1 -- 172.21.15.196:0/1529350385 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f71340b3510 con 0x7f713415f6b0 2024-04-28T13:32:06.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.494+0000 7f712a000640 1 -- 172.21.15.196:0/1529350385 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f712c05ac90 con 0x7f713415f6b0 2024-04-28T13:32:06.838 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:32:06.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.497+0000 7f7133200640 1 -- 172.21.15.196:0/1529350385 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f711006c8f0 msgr2=0x7f711006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:06.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.497+0000 7f7133200640 1 --2- 172.21.15.196:0/1529350385 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f711006c8f0 0x7f711006edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7124034000 tx=0x7f712402e000 comp rx=0 tx=0).stop 2024-04-28T13:32:06.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.497+0000 7f7133200640 1 -- 172.21.15.196:0/1529350385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f713415f6b0 msgr2=0x7f713409d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:06.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.497+0000 7f7133200640 1 --2- 172.21.15.196:0/1529350385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f713415f6b0 0x7f713409d760 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f712c00cca0 tx=0x7f712c00a650 comp rx=0 tx=0).stop 2024-04-28T13:32:06.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.497+0000 7f7133200640 1 -- 172.21.15.196:0/1529350385 shutdown_connections 2024-04-28T13:32:06.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.497+0000 7f7133200640 1 --2- 172.21.15.196:0/1529350385 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f711006c8f0 0x7f711006edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.497+0000 7f7133200640 1 --2- 172.21.15.196:0/1529350385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f713415f6b0 0x7f713409d760 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.497+0000 7f7133200640 1 --2- 172.21.15.196:0/1529350385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7134071ef0 0x7f713415ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.497+0000 7f7133200640 1 --2- 172.21.15.196:0/1529350385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f713406f470 0x7f71341f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:06.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.497+0000 7f7133200640 1 -- 172.21.15.196:0/1529350385 >> 172.21.15.196:0/1529350385 conn(0x7f71340b2990 msgr2=0x7f71340b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:06.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.497+0000 7f7133200640 1 -- 172.21.15.196:0/1529350385 shutdown_connections 2024-04-28T13:32:06.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:06.497+0000 7f7133200640 1 -- 172.21.15.196:0/1529350385 wait complete. 2024-04-28T13:32:06.841 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T13:32:07.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.047+0000 7f406e400640 1 Processor -- start 2024-04-28T13:32:07.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.048+0000 7f406e400640 1 -- start start 2024-04-28T13:32:07.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.049+0000 7f406e400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f40680d1760 0x7f40680b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:07.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.049+0000 7f406e400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406815e380 con 0x7f40680b5ee0 2024-04-28T13:32:07.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.049+0000 7f406e400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40680b4b20 con 0x7f406809ec80 2024-04-28T13:32:07.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.049+0000 7f406e400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f40680b3f30 con 0x7f40680d1760 2024-04-28T13:32:07.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.050+0000 7f405fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f40680d1760 0x7f40680b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:07.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.050+0000 7f405fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f40680d1760 0x7f40680b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34216/0 (socket says 172.21.15.196:34216) 2024-04-28T13:32:07.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.050+0000 7f405fe00640 1 -- 172.21.15.196:0/2782218490 learned_addr learned my addr 172.21.15.196:0/2782218490 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:07.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.050+0000 7f405fe00640 1 -- 172.21.15.196:0/2782218490 >> v1:172.21.15.203:6789/0 conn(0x7f406809ec80 legacy=0x7f406806df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:07.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.050+0000 7f405fe00640 1 -- 172.21.15.196:0/2782218490 >> v1:172.21.15.196:6789/0 conn(0x7f40680b5ee0 legacy=0x7f40680d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:07.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.050+0000 7f405fe00640 1 -- 172.21.15.196:0/2782218490 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40680b41f0 con 0x7f40680d1760 2024-04-28T13:32:07.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.051+0000 7f405fe00640 1 --2- 172.21.15.196:0/2782218490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40680d1760 0x7f40680b4f40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4050009870 tx=0x7f405002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef0464305694bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:07.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.052+0000 7f4067e00640 1 -- 172.21.15.196:0/2782218490 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050037070 con 0x7f40680d1760 2024-04-28T13:32:07.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.052+0000 7f4067e00640 1 -- 172.21.15.196:0/2782218490 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4050032440 con 0x7f40680d1760 2024-04-28T13:32:07.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.052+0000 7f4067e00640 1 -- 172.21.15.196:0/2782218490 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050032810 con 0x7f40680d1760 2024-04-28T13:32:07.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.052+0000 7f406e400640 1 -- 172.21.15.196:0/2782218490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40680d1760 msgr2=0x7f40680b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:07.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.052+0000 7f406e400640 1 --2- 172.21.15.196:0/2782218490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40680d1760 0x7f40680b4f40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4050009870 tx=0x7f405002a2a0 comp rx=0 tx=0).stop 2024-04-28T13:32:07.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.052+0000 7f406e400640 1 -- 172.21.15.196:0/2782218490 shutdown_connections 2024-04-28T13:32:07.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.052+0000 7f406e400640 1 --2- 172.21.15.196:0/2782218490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40680d1760 0x7f40680b4f40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:07.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.052+0000 7f406e400640 1 -- 172.21.15.196:0/2782218490 >> 172.21.15.196:0/2782218490 conn(0x7f406806c6c0 msgr2=0x7f406806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:07.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.052+0000 7f406e400640 1 -- 172.21.15.196:0/2782218490 shutdown_connections 2024-04-28T13:32:07.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.052+0000 7f406e400640 1 -- 172.21.15.196:0/2782218490 wait complete. 2024-04-28T13:32:07.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.054+0000 7f406e400640 1 Processor -- start 2024-04-28T13:32:07.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.054+0000 7f406e400640 1 -- start start 2024-04-28T13:32:07.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.055+0000 7f406e400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f406809ec80 0x7f406814b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:07.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.055+0000 7f406e400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40680b5ee0 0x7f40680ce580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:07.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.055+0000 7f406e400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40680d1760 0x7f40680cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:07.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.055+0000 7f406e400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40680b3c20 con 0x7f406809ec80 2024-04-28T13:32:07.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.055+0000 7f406e400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f40680b47b0 con 0x7f40680b5ee0 2024-04-28T13:32:07.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.055+0000 7f406e400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f40680add70 con 0x7f40680d1760 2024-04-28T13:32:07.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.055+0000 7f406d000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f406809ec80 0x7f406814b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:07.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.055+0000 7f406d000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f406809ec80 0x7f406814b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34222/0 (socket says 172.21.15.196:34222) 2024-04-28T13:32:07.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.055+0000 7f406d000640 1 -- 172.21.15.196:0/567347097 learned_addr learned my addr 172.21.15.196:0/567347097 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:07.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.056+0000 7f406d000640 1 -- 172.21.15.196:0/567347097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40680d1760 msgr2=0x7f40680cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:07.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.056+0000 7f406d000640 1 --2- 172.21.15.196:0/567347097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40680d1760 0x7f40680cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:07.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.056+0000 7f405fe00640 1 --2- 172.21.15.196:0/567347097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40680b5ee0 0x7f40680ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:07.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.056+0000 7f406d000640 1 -- 172.21.15.196:0/567347097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40680b5ee0 msgr2=0x7f40680ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:07.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.056+0000 7f406d000640 1 --2- 172.21.15.196:0/567347097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40680b5ee0 0x7f40680ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:07.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.056+0000 7f406d000640 1 -- 172.21.15.196:0/567347097 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406809c580 con 0x7f406809ec80 2024-04-28T13:32:07.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.056+0000 7f406d000640 1 --2- 172.21.15.196:0/567347097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f406809ec80 0x7f406814b6f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f405800c430 tx=0x7f405800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:07.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.057+0000 7f4065600640 1 -- 172.21.15.196:0/567347097 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40580052d0 con 0x7f406809ec80 2024-04-28T13:32:07.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.057+0000 7f4065600640 1 -- 172.21.15.196:0/567347097 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4058005490 con 0x7f406809ec80 2024-04-28T13:32:07.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.057+0000 7f4065600640 1 -- 172.21.15.196:0/567347097 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058007260 con 0x7f406809ec80 2024-04-28T13:32:07.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.057+0000 7f406e400640 1 -- 172.21.15.196:0/567347097 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406809c820 con 0x7f406809ec80 2024-04-28T13:32:07.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.057+0000 7f406e400640 1 -- 172.21.15.196:0/567347097 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406809cde0 con 0x7f406809ec80 2024-04-28T13:32:07.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.058+0000 7f406e400640 1 -- 172.21.15.196:0/567347097 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406806a290 con 0x7f406809ec80 2024-04-28T13:32:07.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.059+0000 7f4065600640 1 -- 172.21.15.196:0/567347097 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4058007400 con 0x7f406809ec80 2024-04-28T13:32:07.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.060+0000 7f4065600640 1 --2- 172.21.15.196:0/567347097 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f404406c920 0x7f404406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:07.060 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.060+0000 7f4065600640 1 -- 172.21.15.196:0/567347097 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f405808b140 con 0x7f406809ec80 2024-04-28T13:32:07.060 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.061+0000 7f405fe00640 1 --2- 172.21.15.196:0/567347097 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f404406c920 0x7f404406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:07.061 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.062+0000 7f405fe00640 1 --2- 172.21.15.196:0/567347097 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f404406c920 0x7f404406ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4050034000 tx=0x7f405002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:07.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.064+0000 7f4065600640 1 -- 172.21.15.196:0/567347097 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f40580594e0 con 0x7f406809ec80 2024-04-28T13:32:07.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.276+0000 7f406e400640 1 -- 172.21.15.196:0/567347097 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f406806ee60 con 0x7f406809ec80 2024-04-28T13:32:07.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.277+0000 7f4065600640 1 -- 172.21.15.196:0/567347097 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f4058058de0 con 0x7f406809ec80 2024-04-28T13:32:07.277 INFO:teuthology.orchestra.run.smithi196.stderr:listed 0 entries 2024-04-28T13:32:07.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.280+0000 7f406e400640 1 -- 172.21.15.196:0/567347097 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f404406c920 msgr2=0x7f404406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:07.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.280+0000 7f406e400640 1 --2- 172.21.15.196:0/567347097 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f404406c920 0x7f404406ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4050034000 tx=0x7f405002e000 comp rx=0 tx=0).stop 2024-04-28T13:32:07.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.280+0000 7f406e400640 1 -- 172.21.15.196:0/567347097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f406809ec80 msgr2=0x7f406814b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:07.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.280+0000 7f406e400640 1 --2- 172.21.15.196:0/567347097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f406809ec80 0x7f406814b6f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f405800c430 tx=0x7f405800c8f0 comp rx=0 tx=0).stop 2024-04-28T13:32:07.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.280+0000 7f406e400640 1 -- 172.21.15.196:0/567347097 shutdown_connections 2024-04-28T13:32:07.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.280+0000 7f406e400640 1 --2- 172.21.15.196:0/567347097 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f404406c920 0x7f404406ede0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:07.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.280+0000 7f406e400640 1 --2- 172.21.15.196:0/567347097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40680d1760 0x7f40680cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:07.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.280+0000 7f406e400640 1 --2- 172.21.15.196:0/567347097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40680b5ee0 0x7f40680ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:07.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.280+0000 7f406e400640 1 --2- 172.21.15.196:0/567347097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f406809ec80 0x7f406814b6f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:07.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.280+0000 7f406e400640 1 -- 172.21.15.196:0/567347097 >> 172.21.15.196:0/567347097 conn(0x7f406806c6c0 msgr2=0x7f4068160000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:07.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.280+0000 7f406e400640 1 -- 172.21.15.196:0/567347097 shutdown_connections 2024-04-28T13:32:07.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.280+0000 7f406e400640 1 -- 172.21.15.196:0/567347097 wait complete. 2024-04-28T13:32:07.298 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-28T13:32:07.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:07.299 DEBUG:teuthology.orchestra.run.smithi196:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-28T13:32:07.390 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:32:07.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:07.391 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-28T13:32:07.443 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:32:07.443 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:07.500 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:32:07.501 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:07.550 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:32:07.550 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:07.600 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:32:07.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.814+0000 7f2f3f200640 1 Processor -- start 2024-04-28T13:32:07.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.814+0000 7f2f3f200640 1 -- start start 2024-04-28T13:32:07.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.816+0000 7f2f3f200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2f400d1760 0x7f2f400b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:07.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.816+0000 7f2f3f200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f4006dc70 con 0x7f2f4015ef00 2024-04-28T13:32:07.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.816+0000 7f2f3f200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f4006df50 con 0x7f2f4015f2e0 2024-04-28T13:32:07.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.816+0000 7f2f3f200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2f4006e230 con 0x7f2f400d1760 2024-04-28T13:32:07.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.816+0000 7f2f3e800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2f400d1760 0x7f2f400b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:07.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.816+0000 7f2f3e800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2f400d1760 0x7f2f400b5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34226/0 (socket says 172.21.15.196:34226) 2024-04-28T13:32:07.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.816+0000 7f2f3e800640 1 -- 172.21.15.196:0/2278992953 learned_addr learned my addr 172.21.15.196:0/2278992953 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:07.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.816+0000 7f2f3de00640 1 --1- 172.21.15.196:0/2278992953 >> v1:172.21.15.196:6789/0 conn(0x7f2f4015ef00 0x7f2f4006f7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43382/0 (socket says 172.21.15.196:43382) 2024-04-28T13:32:07.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.817+0000 7f2f3e800640 1 -- 172.21.15.196:0/2278992953 >> v1:172.21.15.203:6789/0 conn(0x7f2f4015f2e0 legacy=0x7f2f400d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:07.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.817+0000 7f2f3e800640 1 -- 172.21.15.196:0/2278992953 >> v1:172.21.15.196:6789/0 conn(0x7f2f4015ef00 legacy=0x7f2f4006f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:07.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.817+0000 7f2f3e800640 1 -- 172.21.15.196:0/2278992953 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4006e510 con 0x7f2f400d1760 2024-04-28T13:32:07.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.817+0000 7f2f3ca00640 1 -- 172.21.15.196:0/2278992953 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2025699354 0 0) 0x7f2f4006dc70 con 0x7f2f4015ef00 2024-04-28T13:32:07.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.818+0000 7f2f3e800640 1 --2- 172.21.15.196:0/2278992953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f400d1760 0x7f2f400b5c10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009e50 tx=0x7f2f3802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462ef5d1515e5000 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:07.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.818+0000 7f2f3ca00640 1 -- 172.21.15.196:0/2278992953 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f38036070 con 0x7f2f400d1760 2024-04-28T13:32:07.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.818+0000 7f2f3ca00640 1 -- 172.21.15.196:0/2278992953 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f38031600 con 0x7f2f400d1760 2024-04-28T13:32:07.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.818+0000 7f2f3ca00640 1 -- 172.21.15.196:0/2278992953 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f380319d0 con 0x7f2f400d1760 2024-04-28T13:32:07.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.819+0000 7f2f3f200640 1 -- 172.21.15.196:0/2278992953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f400d1760 msgr2=0x7f2f400b5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:07.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.819+0000 7f2f3f200640 1 --2- 172.21.15.196:0/2278992953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f400d1760 0x7f2f400b5c10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009e50 tx=0x7f2f3802a6d0 comp rx=0 tx=0).stop 2024-04-28T13:32:07.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.819+0000 7f2f3f200640 1 -- 172.21.15.196:0/2278992953 shutdown_connections 2024-04-28T13:32:07.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.819+0000 7f2f3f200640 1 --2- 172.21.15.196:0/2278992953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f400d1760 0x7f2f400b5c10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:07.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.819+0000 7f2f3f200640 1 -- 172.21.15.196:0/2278992953 >> 172.21.15.196:0/2278992953 conn(0x7f2f4006c6c0 msgr2=0x7f2f4006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:07.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.819+0000 7f2f3f200640 1 -- 172.21.15.196:0/2278992953 shutdown_connections 2024-04-28T13:32:07.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.819+0000 7f2f3f200640 1 -- 172.21.15.196:0/2278992953 wait complete. 2024-04-28T13:32:07.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.820+0000 7f2f3f200640 1 Processor -- start 2024-04-28T13:32:07.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.821+0000 7f2f3f200640 1 -- start start 2024-04-28T13:32:07.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.821+0000 7f2f3f200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f400d1760 0x7f2f400af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:07.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.821+0000 7f2f3f200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f4015ef00 0x7f2f40148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:07.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.821+0000 7f2f3f200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f4015f2e0 0x7f2f40147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:07.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.821+0000 7f2f3f200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f40169df0 con 0x7f2f4015f2e0 2024-04-28T13:32:07.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.821+0000 7f2f3f200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2f40073320 con 0x7f2f400d1760 2024-04-28T13:32:07.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.822+0000 7f2f3f200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2f40169f70 con 0x7f2f4015ef00 2024-04-28T13:32:07.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.822+0000 7f2f3e800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f4015f2e0 0x7f2f40147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:07.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.822+0000 7f2f3e800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f4015f2e0 0x7f2f40147660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34230/0 (socket says 172.21.15.196:34230) 2024-04-28T13:32:07.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.822+0000 7f2f3e800640 1 -- 172.21.15.196:0/4194681339 learned_addr learned my addr 172.21.15.196:0/4194681339 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:07.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.822+0000 7f2f3de00640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f400d1760 0x7f2f400af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:07.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.822+0000 7f2f3e800640 1 -- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f4015ef00 msgr2=0x7f2f40148f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:07.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.822+0000 7f2f3d400640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f4015ef00 0x7f2f40148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:07.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.822+0000 7f2f3e800640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f4015ef00 0x7f2f40148f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:07.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.822+0000 7f2f3e800640 1 -- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f400d1760 msgr2=0x7f2f400af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:07.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.822+0000 7f2f3e800640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f400d1760 0x7f2f400af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:07.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.822+0000 7f2f3e800640 1 -- 172.21.15.196:0/4194681339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f40149910 con 0x7f2f4015f2e0 2024-04-28T13:32:07.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.822+0000 7f2f3de00640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f400d1760 0x7f2f400af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:07.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.822+0000 7f2f3d400640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f4015ef00 0x7f2f40148f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:07.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.823+0000 7f2f3e800640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f4015f2e0 0x7f2f40147660 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009e50 tx=0x7f2f38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:07.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.823+0000 7f2f36000640 1 -- 172.21.15.196:0/4194681339 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f38041070 con 0x7f2f4015f2e0 2024-04-28T13:32:07.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.823+0000 7f2f36000640 1 -- 172.21.15.196:0/4194681339 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f38003990 con 0x7f2f4015f2e0 2024-04-28T13:32:07.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.823+0000 7f2f36000640 1 -- 172.21.15.196:0/4194681339 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f38036040 con 0x7f2f4015f2e0 2024-04-28T13:32:07.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.823+0000 7f2f3f200640 1 -- 172.21.15.196:0/4194681339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f40149bf0 con 0x7f2f4015f2e0 2024-04-28T13:32:07.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.823+0000 7f2f3f200640 1 -- 172.21.15.196:0/4194681339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f4014a1b0 con 0x7f2f4015f2e0 2024-04-28T13:32:07.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.824+0000 7f2f3f200640 1 -- 172.21.15.196:0/4194681339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f4006a290 con 0x7f2f4015f2e0 2024-04-28T13:32:07.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.825+0000 7f2f36000640 1 -- 172.21.15.196:0/4194681339 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f38007800 con 0x7f2f4015f2e0 2024-04-28T13:32:07.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.826+0000 7f2f36000640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2f2006c8f0 0x7f2f2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:07.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.826+0000 7f2f36000640 1 -- 172.21.15.196:0/4194681339 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f2f380ac8a0 con 0x7f2f4015f2e0 2024-04-28T13:32:07.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.827+0000 7f2f3de00640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2f2006c8f0 0x7f2f2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:07.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.828+0000 7f2f3de00640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2f2006c8f0 0x7f2f2006edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2f30000f60 tx=0x7f2f30009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:07.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:07.831+0000 7f2f36000640 1 -- 172.21.15.196:0/4194681339 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2f3804b020 con 0x7f2f4015f2e0 2024-04-28T13:32:08.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.031+0000 7f2f3f200640 1 -- 172.21.15.196:0/4194681339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f401629e0 con 0x7f2f4015f2e0 2024-04-28T13:32:08.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.034+0000 7f2f36000640 1 -- 172.21.15.196:0/4194681339 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16890 (secure 0 0 0) 0x7f2f3807ac70 con 0x7f2f4015f2e0 2024-04-28T13:32:08.034 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:08.035 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:03.882452+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:32:08.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.037+0000 7f2f3f200640 1 -- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2f2006c8f0 msgr2=0x7f2f2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:08.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.037+0000 7f2f3f200640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2f2006c8f0 0x7f2f2006edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2f30000f60 tx=0x7f2f30009040 comp rx=0 tx=0).stop 2024-04-28T13:32:08.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.037+0000 7f2f3f200640 1 -- 172.21.15.196:0/4194681339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f4015f2e0 msgr2=0x7f2f40147660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:08.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.037+0000 7f2f3f200640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f4015f2e0 0x7f2f40147660 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009e50 tx=0x7f2f38004a40 comp rx=0 tx=0).stop 2024-04-28T13:32:08.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.037+0000 7f2f3f200640 1 -- 172.21.15.196:0/4194681339 shutdown_connections 2024-04-28T13:32:08.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.037+0000 7f2f3f200640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f2f2006c8f0 0x7f2f2006edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.037+0000 7f2f3f200640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f4015f2e0 0x7f2f40147660 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.037+0000 7f2f3f200640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f4015ef00 0x7f2f40148f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.037+0000 7f2f3f200640 1 --2- 172.21.15.196:0/4194681339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f400d1760 0x7f2f400af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.037+0000 7f2f3f200640 1 -- 172.21.15.196:0/4194681339 >> 172.21.15.196:0/4194681339 conn(0x7f2f4006c6c0 msgr2=0x7f2f40071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:08.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.037+0000 7f2f3f200640 1 -- 172.21.15.196:0/4194681339 shutdown_connections 2024-04-28T13:32:08.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.037+0000 7f2f3f200640 1 -- 172.21.15.196:0/4194681339 wait complete. 2024-04-28T13:32:08.056 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:31:19.280975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:31:20.037066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:32:08.057 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:32:08.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.255+0000 7f365b200640 1 Processor -- start 2024-04-28T13:32:08.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.255+0000 7f365b200640 1 -- start start 2024-04-28T13:32:08.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.256+0000 7f365b200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f36540af930 0x7f36540d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:08.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.256+0000 7f365b200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36540d15e0 0x7f36540b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:08.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.256+0000 7f365b200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3654072fe0 con 0x7f365415be50 2024-04-28T13:32:08.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.256+0000 7f365b200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f36540d1a30 con 0x7f36540af930 2024-04-28T13:32:08.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.256+0000 7f365b200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3654073ec0 con 0x7f36540d15e0 2024-04-28T13:32:08.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.257+0000 7f3659e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f365415be50 0x7f36540728d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43388/0 (socket says 172.21.15.196:43388) 2024-04-28T13:32:08.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.257+0000 7f3659e00640 1 -- 172.21.15.196:0/3135713111 learned_addr learned my addr 172.21.15.196:0/3135713111 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:08.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.257+0000 7f3659400640 1 --2- 172.21.15.196:0/3135713111 >> v2:172.21.15.203:3300/0 conn(0x7f36540af930 0x7f36540d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:08.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.257+0000 7f365a800640 1 --2- 172.21.15.196:0/3135713111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36540d15e0 0x7f36540b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:08.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.257+0000 7f3658a00640 1 -- 172.21.15.196:0/3135713111 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4199694194 0 0) 0x7f3654072fe0 con 0x7f365415be50 2024-04-28T13:32:08.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.257+0000 7f3658a00640 1 -- 172.21.15.196:0/3135713111 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f363c003520 con 0x7f365415be50 2024-04-28T13:32:08.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f3658a00640 1 -- 172.21.15.196:0/3135713111 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1021476454 0 0) 0x7f363c003520 con 0x7f365415be50 2024-04-28T13:32:08.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f3658a00640 1 -- 172.21.15.196:0/3135713111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36540d15e0 msgr2=0x7f36540b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:08.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f3658a00640 1 --2- 172.21.15.196:0/3135713111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36540d15e0 0x7f36540b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f3658a00640 1 -- 172.21.15.196:0/3135713111 >> v2:172.21.15.203:3300/0 conn(0x7f36540af930 msgr2=0x7f36540d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:08.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f3658a00640 1 --2- 172.21.15.196:0/3135713111 >> v2:172.21.15.203:3300/0 conn(0x7f36540af930 0x7f36540d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f3658a00640 1 -- 172.21.15.196:0/3135713111 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36540732b0 con 0x7f365415be50 2024-04-28T13:32:08.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f3658a00640 1 -- 172.21.15.196:0/3135713111 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f3648001670 con 0x7f365415be50 2024-04-28T13:32:08.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f3658a00640 1 -- 172.21.15.196:0/3135713111 >> v1:172.21.15.196:6789/0 conn(0x7f365415be50 legacy=0x7f36540728d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:08.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f3658a00640 1 --2- 172.21.15.196:0/3135713111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f363c0039b0 0x7f363c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:08.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f3658a00640 1 --2- 172.21.15.196:0/3135713111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36540d15e0 0x7f363c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:08.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f3658a00640 1 --2- 172.21.15.196:0/3135713111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f363c004d40 0x7f363c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:08.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f3658a00640 1 -- 172.21.15.196:0/3135713111 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36540d1a30 con 0x7f363c004d40 2024-04-28T13:32:08.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f3658a00640 1 -- 172.21.15.196:0/3135713111 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3654073ec0 con 0x7f363c0039b0 2024-04-28T13:32:08.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f3658a00640 1 -- 172.21.15.196:0/3135713111 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f363c00b9c0 con 0x7f36540d15e0 2024-04-28T13:32:08.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f365a800640 1 --2- 172.21.15.196:0/3135713111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f363c004d40 0x7f363c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:08.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.258+0000 7f3659400640 1 --2- 172.21.15.196:0/3135713111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36540d15e0 0x7f363c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:08.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.259+0000 7f365a800640 1 -- 172.21.15.196:0/3135713111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36540d15e0 msgr2=0x7f363c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:08.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.259+0000 7f365a800640 1 --2- 172.21.15.196:0/3135713111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36540d15e0 0x7f363c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.259+0000 7f3659e00640 1 --2- 172.21.15.196:0/3135713111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f363c0039b0 0x7f363c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:08.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.259+0000 7f365a800640 1 -- 172.21.15.196:0/3135713111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f363c0039b0 msgr2=0x7f363c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:08.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.259+0000 7f365a800640 1 --2- 172.21.15.196:0/3135713111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f363c0039b0 0x7f363c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.259+0000 7f365a800640 1 -- 172.21.15.196:0/3135713111 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c003520 con 0x7f363c004d40 2024-04-28T13:32:08.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.259+0000 7f3659e00640 1 --2- 172.21.15.196:0/3135713111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f363c0039b0 0x7f363c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:08.262 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.260+0000 7f365a800640 1 --2- 172.21.15.196:0/3135713111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f363c004d40 0x7f363c0051c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3650009fc0 tx=0x7f363c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28c6f1af976b9bc server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:08.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.260+0000 7f3658a00640 1 -- 172.21.15.196:0/3135713111 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650009c10 con 0x7f363c004d40 2024-04-28T13:32:08.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.260+0000 7f3658a00640 1 -- 172.21.15.196:0/3135713111 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3650036070 con 0x7f363c004d40 2024-04-28T13:32:08.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.260+0000 7f3658a00640 1 -- 172.21.15.196:0/3135713111 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650031880 con 0x7f363c004d40 2024-04-28T13:32:08.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.260+0000 7f365b200640 1 -- 172.21.15.196:0/3135713111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f363c004d40 msgr2=0x7f363c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:08.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.260+0000 7f365b200640 1 --2- 172.21.15.196:0/3135713111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f363c004d40 0x7f363c0051c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3650009fc0 tx=0x7f363c005830 comp rx=0 tx=0).stop 2024-04-28T13:32:08.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.260+0000 7f365a800640 1 -- 172.21.15.196:0/3135713111 reap_dead start 2024-04-28T13:32:08.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.260+0000 7f365b200640 1 -- 172.21.15.196:0/3135713111 shutdown_connections 2024-04-28T13:32:08.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.261+0000 7f365b200640 1 -- 172.21.15.196:0/3135713111 >> 172.21.15.196:0/3135713111 conn(0x7f365406c6c0 msgr2=0x7f365406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:08.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.261+0000 7f365b200640 1 -- 172.21.15.196:0/3135713111 shutdown_connections 2024-04-28T13:32:08.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.261+0000 7f365b200640 1 -- 172.21.15.196:0/3135713111 wait complete. 2024-04-28T13:32:08.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.262+0000 7f365b200640 1 Processor -- start 2024-04-28T13:32:08.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.262+0000 7f365b200640 1 -- start start 2024-04-28T13:32:08.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.263+0000 7f365b200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f363c004d40 0x7f365414cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:08.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.263+0000 7f365b200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365409c470 0x7f365415e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:08.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.263+0000 7f365b200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36541edd10 0x7f365406eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:08.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.263+0000 7f365b200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3654073b50 con 0x7f365409c470 2024-04-28T13:32:08.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.264+0000 7f365b200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3654073870 con 0x7f363c004d40 2024-04-28T13:32:08.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.264+0000 7f365b200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f365415c230 con 0x7f36541edd10 2024-04-28T13:32:08.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.264+0000 7f3659400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365409c470 0x7f365415e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:08.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.264+0000 7f3659400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365409c470 0x7f365415e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34250/0 (socket says 172.21.15.196:34250) 2024-04-28T13:32:08.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.264+0000 7f3659400640 1 -- 172.21.15.196:0/2895315961 learned_addr learned my addr 172.21.15.196:0/2895315961 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:08.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.264+0000 7f3659e00640 1 --2- 172.21.15.196:0/2895315961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f363c004d40 0x7f365414cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:08.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.264+0000 7f3659400640 1 -- 172.21.15.196:0/2895315961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36541edd10 msgr2=0x7f365406eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:08.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.264+0000 7f3659400640 1 --2- 172.21.15.196:0/2895315961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36541edd10 0x7f365406eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.264+0000 7f3659400640 1 -- 172.21.15.196:0/2895315961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f363c004d40 msgr2=0x7f365414cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:08.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.264+0000 7f3659400640 1 --2- 172.21.15.196:0/2895315961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f363c004d40 0x7f365414cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.264+0000 7f3659400640 1 -- 172.21.15.196:0/2895315961 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365414c510 con 0x7f365409c470 2024-04-28T13:32:08.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.264+0000 7f3659400640 1 --2- 172.21.15.196:0/2895315961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365409c470 0x7f365415e6c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3640010910 tx=0x7f3640010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:08.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.265+0000 7f364e000640 1 -- 172.21.15.196:0/2895315961 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364000f040 con 0x7f365409c470 2024-04-28T13:32:08.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.265+0000 7f364e000640 1 -- 172.21.15.196:0/2895315961 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f364000eba0 con 0x7f365409c470 2024-04-28T13:32:08.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.265+0000 7f364e000640 1 -- 172.21.15.196:0/2895315961 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364001c400 con 0x7f365409c470 2024-04-28T13:32:08.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.265+0000 7f365b200640 1 -- 172.21.15.196:0/2895315961 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365415ec00 con 0x7f365409c470 2024-04-28T13:32:08.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.265+0000 7f365b200640 1 -- 172.21.15.196:0/2895315961 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f365415eea0 con 0x7f365409c470 2024-04-28T13:32:08.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.266+0000 7f365b200640 1 -- 172.21.15.196:0/2895315961 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f365414cc80 con 0x7f365409c470 2024-04-28T13:32:08.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.267+0000 7f364e000640 1 -- 172.21.15.196:0/2895315961 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f364001c5a0 con 0x7f365409c470 2024-04-28T13:32:08.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.268+0000 7f364e000640 1 --2- 172.21.15.196:0/2895315961 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f363406c920 0x7f363406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:08.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.269+0000 7f364e000640 1 -- 172.21.15.196:0/2895315961 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f364008c1f0 con 0x7f365409c470 2024-04-28T13:32:08.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.269+0000 7f3659e00640 1 --2- 172.21.15.196:0/2895315961 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f363406c920 0x7f363406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:08.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.270+0000 7f3659e00640 1 --2- 172.21.15.196:0/2895315961 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f363406c920 0x7f363406ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3648000c00 tx=0x7f3648009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:08.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.273+0000 7f364e000640 1 -- 172.21.15.196:0/2895315961 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3640018030 con 0x7f365409c470 2024-04-28T13:32:08.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.474+0000 7f365b200640 1 -- 172.21.15.196:0/2895315961 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3654050730 con 0x7f365409c470 2024-04-28T13:32:08.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.475+0000 7f364e000640 1 -- 172.21.15.196:0/2895315961 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f364005a590 con 0x7f365409c470 2024-04-28T13:32:08.475 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:32:08.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.478+0000 7f365b200640 1 -- 172.21.15.196:0/2895315961 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f363406c920 msgr2=0x7f363406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:08.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.478+0000 7f365b200640 1 --2- 172.21.15.196:0/2895315961 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f363406c920 0x7f363406ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3648000c00 tx=0x7f3648009040 comp rx=0 tx=0).stop 2024-04-28T13:32:08.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.478+0000 7f365b200640 1 -- 172.21.15.196:0/2895315961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365409c470 msgr2=0x7f365415e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:08.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.478+0000 7f365b200640 1 --2- 172.21.15.196:0/2895315961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365409c470 0x7f365415e6c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3640010910 tx=0x7f3640010dd0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.478+0000 7f365b200640 1 -- 172.21.15.196:0/2895315961 shutdown_connections 2024-04-28T13:32:08.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.478+0000 7f365b200640 1 --2- 172.21.15.196:0/2895315961 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f363406c920 0x7f363406ede0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.478+0000 7f365b200640 1 --2- 172.21.15.196:0/2895315961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36541edd10 0x7f365406eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.478+0000 7f365b200640 1 --2- 172.21.15.196:0/2895315961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365409c470 0x7f365415e6c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.478+0000 7f365b200640 1 --2- 172.21.15.196:0/2895315961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f363c004d40 0x7f365414cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.478+0000 7f365b200640 1 -- 172.21.15.196:0/2895315961 >> 172.21.15.196:0/2895315961 conn(0x7f365406c6c0 msgr2=0x7f36540b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:08.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.479+0000 7f365b200640 1 -- 172.21.15.196:0/2895315961 shutdown_connections 2024-04-28T13:32:08.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.479+0000 7f365b200640 1 -- 172.21.15.196:0/2895315961 wait complete. 2024-04-28T13:32:08.499 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:32:08.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.698+0000 7f88e1a00640 1 Processor -- start 2024-04-28T13:32:08.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.699+0000 7f88e1a00640 1 -- start start 2024-04-28T13:32:08.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.700+0000 7f88e1a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88dc0706a0 0x7f88dc06fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:08.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.700+0000 7f88e1a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f88dc06c9b0 0x7f88dc0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:08.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.700+0000 7f88e1a00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88dc070b30 con 0x7f88dc15f2b0 2024-04-28T13:32:08.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.700+0000 7f88e1a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f88dc0af610 con 0x7f88dc06c9b0 2024-04-28T13:32:08.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.700+0000 7f88e1a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f88dc0700b0 con 0x7f88dc0706a0 2024-04-28T13:32:08.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.700+0000 7f88db400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f88dc06c9b0 0x7f88dc0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:08.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.700+0000 7f88db400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f88dc06c9b0 0x7f88dc0afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34252/0 (socket says 172.21.15.196:34252) 2024-04-28T13:32:08.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.700+0000 7f88db400640 1 -- 172.21.15.196:0/1939542189 learned_addr learned my addr 172.21.15.196:0/1939542189 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:08.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.701+0000 7f88d3e00640 1 --2- 172.21.15.196:0/1939542189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88dc0706a0 0x7f88dc06fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:08.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.701+0000 7f88db400640 1 -- 172.21.15.196:0/1939542189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88dc0706a0 msgr2=0x7f88dc06fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:08.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.701+0000 7f88db400640 1 --2- 172.21.15.196:0/1939542189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88dc0706a0 0x7f88dc06fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.701+0000 7f88db400640 1 -- 172.21.15.196:0/1939542189 >> v1:172.21.15.203:6789/0 conn(0x7f88dc15f2b0 legacy=0x7f88dc06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:08.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.701+0000 7f88db400640 1 -- 172.21.15.196:0/1939542189 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc070e10 con 0x7f88dc06c9b0 2024-04-28T13:32:08.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.702+0000 7f88db400640 1 --2- 172.21.15.196:0/1939542189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88dc06c9b0 0x7f88dc0afa80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009e50 tx=0x7f88d402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236fe5ae80e15797 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:08.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.702+0000 7f88da000640 1 -- 172.21.15.196:0/1939542189 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4036070 con 0x7f88dc06c9b0 2024-04-28T13:32:08.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.702+0000 7f88da000640 1 -- 172.21.15.196:0/1939542189 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d4031600 con 0x7f88dc06c9b0 2024-04-28T13:32:08.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.702+0000 7f88da000640 1 -- 172.21.15.196:0/1939542189 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d40319d0 con 0x7f88dc06c9b0 2024-04-28T13:32:08.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.703+0000 7f88e1a00640 1 -- 172.21.15.196:0/1939542189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88dc06c9b0 msgr2=0x7f88dc0afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:08.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.703+0000 7f88e1a00640 1 --2- 172.21.15.196:0/1939542189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88dc06c9b0 0x7f88dc0afa80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009e50 tx=0x7f88d402a710 comp rx=0 tx=0).stop 2024-04-28T13:32:08.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.703+0000 7f88e1a00640 1 -- 172.21.15.196:0/1939542189 shutdown_connections 2024-04-28T13:32:08.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.703+0000 7f88e1a00640 1 --2- 172.21.15.196:0/1939542189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88dc06c9b0 0x7f88dc0afa80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.703+0000 7f88e1a00640 1 --2- 172.21.15.196:0/1939542189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88dc0706a0 0x7f88dc06fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.703+0000 7f88e1a00640 1 -- 172.21.15.196:0/1939542189 >> 172.21.15.196:0/1939542189 conn(0x7f88dc06f050 msgr2=0x7f88dc06f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:08.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.703+0000 7f88e1a00640 1 -- 172.21.15.196:0/1939542189 shutdown_connections 2024-04-28T13:32:08.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.703+0000 7f88e1a00640 1 -- 172.21.15.196:0/1939542189 wait complete. 2024-04-28T13:32:08.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.705+0000 7f88e1a00640 1 Processor -- start 2024-04-28T13:32:08.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.705+0000 7f88e1a00640 1 -- start start 2024-04-28T13:32:08.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88e1a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88dc06c9b0 0x7f88dc145820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:08.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88e1a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88dc15f2b0 0x7f88dc161210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:08.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88e1a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88dc154140 0x7f88dc073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:08.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88e1a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc0b4f40 con 0x7f88dc15f2b0 2024-04-28T13:32:08.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88e1a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc0713d0 con 0x7f88dc06c9b0 2024-04-28T13:32:08.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88e1a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f88dc0b1890 con 0x7f88dc154140 2024-04-28T13:32:08.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88d3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88dc15f2b0 0x7f88dc161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:08.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88d3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88dc15f2b0 0x7f88dc161210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34264/0 (socket says 172.21.15.196:34264) 2024-04-28T13:32:08.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88d3e00640 1 -- 172.21.15.196:0/2258574106 learned_addr learned my addr 172.21.15.196:0/2258574106 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:08.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88daa00640 1 --2- 172.21.15.196:0/2258574106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88dc06c9b0 0x7f88dc145820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:08.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88db400640 1 --2- 172.21.15.196:0/2258574106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88dc154140 0x7f88dc073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:08.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88d3e00640 1 -- 172.21.15.196:0/2258574106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88dc154140 msgr2=0x7f88dc073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:08.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88d3e00640 1 --2- 172.21.15.196:0/2258574106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88dc154140 0x7f88dc073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88d3e00640 1 -- 172.21.15.196:0/2258574106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88dc06c9b0 msgr2=0x7f88dc145820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:08.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88d3e00640 1 --2- 172.21.15.196:0/2258574106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88dc06c9b0 0x7f88dc145820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88d3e00640 1 -- 172.21.15.196:0/2258574106 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc1546b0 con 0x7f88dc15f2b0 2024-04-28T13:32:08.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.706+0000 7f88daa00640 1 --2- 172.21.15.196:0/2258574106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88dc06c9b0 0x7f88dc145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:08.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.707+0000 7f88d3e00640 1 --2- 172.21.15.196:0/2258574106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88dc15f2b0 0x7f88dc161210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f88c400d800 tx=0x7f88c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:08.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.707+0000 7f88d2a00640 1 -- 172.21.15.196:0/2258574106 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c400b660 con 0x7f88dc15f2b0 2024-04-28T13:32:08.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.707+0000 7f88d2a00640 1 -- 172.21.15.196:0/2258574106 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c4019070 con 0x7f88dc15f2b0 2024-04-28T13:32:08.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.707+0000 7f88d2a00640 1 -- 172.21.15.196:0/2258574106 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c400ba20 con 0x7f88dc15f2b0 2024-04-28T13:32:08.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.707+0000 7f88e1a00640 1 -- 172.21.15.196:0/2258574106 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88dc154990 con 0x7f88dc15f2b0 2024-04-28T13:32:08.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.707+0000 7f88e1a00640 1 -- 172.21.15.196:0/2258574106 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88dc154f50 con 0x7f88dc15f2b0 2024-04-28T13:32:08.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.709+0000 7f88e1a00640 1 -- 172.21.15.196:0/2258574106 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88dc06a290 con 0x7f88dc15f2b0 2024-04-28T13:32:08.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.709+0000 7f88d2a00640 1 -- 172.21.15.196:0/2258574106 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88c400bbc0 con 0x7f88dc15f2b0 2024-04-28T13:32:08.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.710+0000 7f88d2a00640 1 --2- 172.21.15.196:0/2258574106 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88b806c8f0 0x7f88b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:08.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.710+0000 7f88d2a00640 1 -- 172.21.15.196:0/2258574106 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f88c408c030 con 0x7f88dc15f2b0 2024-04-28T13:32:08.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.711+0000 7f88daa00640 1 --2- 172.21.15.196:0/2258574106 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88b806c8f0 0x7f88b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:08.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.712+0000 7f88daa00640 1 --2- 172.21.15.196:0/2258574106 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88b806c8f0 0x7f88b806edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0015e0 tx=0x7f88cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:08.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.714+0000 7f88d2a00640 1 -- 172.21.15.196:0/2258574106 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f88c405a3e0 con 0x7f88dc15f2b0 2024-04-28T13:32:08.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.913+0000 7f88e1a00640 1 -- 172.21.15.196:0/2258574106 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f88dc073230 con 0x7f88dc15f2b0 2024-04-28T13:32:08.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.914+0000 7f88d2a00640 1 -- 172.21.15.196:0/2258574106 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f88c4059ce0 con 0x7f88dc15f2b0 2024-04-28T13:32:08.914 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:32:08.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.916+0000 7f88e1a00640 1 -- 172.21.15.196:0/2258574106 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88b806c8f0 msgr2=0x7f88b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:08.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.916+0000 7f88e1a00640 1 --2- 172.21.15.196:0/2258574106 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88b806c8f0 0x7f88b806edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0015e0 tx=0x7f88cc009040 comp rx=0 tx=0).stop 2024-04-28T13:32:08.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.917+0000 7f88e1a00640 1 -- 172.21.15.196:0/2258574106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88dc15f2b0 msgr2=0x7f88dc161210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:08.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.917+0000 7f88e1a00640 1 --2- 172.21.15.196:0/2258574106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88dc15f2b0 0x7f88dc161210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f88c400d800 tx=0x7f88c400dcc0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.917+0000 7f88e1a00640 1 -- 172.21.15.196:0/2258574106 shutdown_connections 2024-04-28T13:32:08.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.917+0000 7f88e1a00640 1 --2- 172.21.15.196:0/2258574106 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f88b806c8f0 0x7f88b806edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.917+0000 7f88e1a00640 1 --2- 172.21.15.196:0/2258574106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88dc154140 0x7f88dc073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.917+0000 7f88e1a00640 1 --2- 172.21.15.196:0/2258574106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88dc15f2b0 0x7f88dc161210 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.917+0000 7f88e1a00640 1 --2- 172.21.15.196:0/2258574106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88dc06c9b0 0x7f88dc145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:08.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.917+0000 7f88e1a00640 1 -- 172.21.15.196:0/2258574106 >> 172.21.15.196:0/2258574106 conn(0x7f88dc06f050 msgr2=0x7f88dc0b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:08.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.917+0000 7f88e1a00640 1 -- 172.21.15.196:0/2258574106 shutdown_connections 2024-04-28T13:32:08.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:08.917+0000 7f88e1a00640 1 -- 172.21.15.196:0/2258574106 wait complete. 2024-04-28T13:32:08.936 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:32:09.139 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.139+0000 7f94a7600640 1 Processor -- start 2024-04-28T13:32:09.139 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.139+0000 7f94a7600640 1 -- start start 2024-04-28T13:32:09.140 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.141+0000 7f94a7600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94a006f340 0x7f94a00b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:09.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.141+0000 7f94a7600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f94a00b3e50 0x7f94a00720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:09.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.141+0000 7f94a7600640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94a00b46a0 con 0x7f94a00d0300 2024-04-28T13:32:09.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.141+0000 7f94a7600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f94a00b5db0 con 0x7f94a00b3e50 2024-04-28T13:32:09.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.141+0000 7f94a7600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f94a00b6090 con 0x7f94a006f340 2024-04-28T13:32:09.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.142+0000 7f94a4c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94a006f340 0x7f94a00b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:09.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.142+0000 7f94a4c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94a006f340 0x7f94a00b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:59634/0 (socket says 172.21.15.196:59634) 2024-04-28T13:32:09.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.142+0000 7f94a4c00640 1 -- 172.21.15.196:0/1980048097 learned_addr learned my addr 172.21.15.196:0/1980048097 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:09.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.142+0000 7f94a5600640 1 --2- 172.21.15.196:0/1980048097 >> v2:172.21.15.203:3300/0 conn(0x7f94a00b3e50 0x7f94a00720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:09.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.142+0000 7f94a4c00640 1 -- 172.21.15.196:0/1980048097 >> v2:172.21.15.203:3300/0 conn(0x7f94a00b3e50 msgr2=0x7f94a00720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:09.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.142+0000 7f94a4c00640 1 --2- 172.21.15.196:0/1980048097 >> v2:172.21.15.203:3300/0 conn(0x7f94a00b3e50 0x7f94a00720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.142+0000 7f94a4c00640 1 -- 172.21.15.196:0/1980048097 >> v1:172.21.15.203:6789/0 conn(0x7f94a00d0300 legacy=0x7f94a006f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:09.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.142+0000 7f94a4c00640 1 -- 172.21.15.196:0/1980048097 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a00b6370 con 0x7f94a006f340 2024-04-28T13:32:09.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.144+0000 7f94a4c00640 1 --2- 172.21.15.196:0/1980048097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94a006f340 0x7f94a00b6b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f94940098a0 tx=0x7f949402a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9d9470b682c37a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:09.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.144+0000 7f949f400640 1 -- 172.21.15.196:0/1980048097 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494037070 con 0x7f94a006f340 2024-04-28T13:32:09.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.144+0000 7f949f400640 1 -- 172.21.15.196:0/1980048097 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9494032440 con 0x7f94a006f340 2024-04-28T13:32:09.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.144+0000 7f949f400640 1 -- 172.21.15.196:0/1980048097 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494032810 con 0x7f94a006f340 2024-04-28T13:32:09.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.145+0000 7f94a7600640 1 -- 172.21.15.196:0/1980048097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94a006f340 msgr2=0x7f94a00b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:09.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.145+0000 7f94a7600640 1 --2- 172.21.15.196:0/1980048097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94a006f340 0x7f94a00b6b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f94940098a0 tx=0x7f949402a160 comp rx=0 tx=0).stop 2024-04-28T13:32:09.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.145+0000 7f94a7600640 1 -- 172.21.15.196:0/1980048097 shutdown_connections 2024-04-28T13:32:09.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.145+0000 7f94a7600640 1 --2- 172.21.15.196:0/1980048097 >> v2:172.21.15.203:3300/0 conn(0x7f94a00b3e50 0x7f94a00720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.145+0000 7f94a7600640 1 --2- 172.21.15.196:0/1980048097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94a006f340 0x7f94a00b6b70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.145+0000 7f94a7600640 1 -- 172.21.15.196:0/1980048097 >> 172.21.15.196:0/1980048097 conn(0x7f94a00cd8a0 msgr2=0x7f94a00cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:09.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.145+0000 7f94a7600640 1 -- 172.21.15.196:0/1980048097 shutdown_connections 2024-04-28T13:32:09.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.145+0000 7f94a7600640 1 -- 172.21.15.196:0/1980048097 wait complete. 2024-04-28T13:32:09.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.147+0000 7f94a7600640 1 Processor -- start 2024-04-28T13:32:09.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.148+0000 7f94a7600640 1 -- start start 2024-04-28T13:32:09.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.149+0000 7f94a7600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94a006f340 0x7f94a0157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:09.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.149+0000 7f94a7600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94a00b3e50 0x7f94a00c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:09.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.149+0000 7f94a7600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94a00d0300 0x7f94a00c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:09.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.149+0000 7f94a7600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f94a00b4460 con 0x7f94a006f340 2024-04-28T13:32:09.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.149+0000 7f94a7600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f94a00b42e0 con 0x7f94a00b3e50 2024-04-28T13:32:09.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.149+0000 7f94a7600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f94a0142a30 con 0x7f94a00d0300 2024-04-28T13:32:09.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.149+0000 7f94a4c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94a006f340 0x7f94a0157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:09.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.149+0000 7f94a4c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94a006f340 0x7f94a0157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34268/0 (socket says 172.21.15.196:34268) 2024-04-28T13:32:09.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.149+0000 7f94a4c00640 1 -- 172.21.15.196:0/4211790040 learned_addr learned my addr 172.21.15.196:0/4211790040 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:09.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.150+0000 7f94a4c00640 1 -- 172.21.15.196:0/4211790040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94a00d0300 msgr2=0x7f94a00c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:09.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.150+0000 7f94a4c00640 1 --2- 172.21.15.196:0/4211790040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94a00d0300 0x7f94a00c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.150+0000 7f94a4c00640 1 -- 172.21.15.196:0/4211790040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94a00b3e50 msgr2=0x7f94a00c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:09.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.150+0000 7f94a4c00640 1 --2- 172.21.15.196:0/4211790040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94a00b3e50 0x7f94a00c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.150+0000 7f94a4c00640 1 -- 172.21.15.196:0/4211790040 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a00c7390 con 0x7f94a006f340 2024-04-28T13:32:09.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.150+0000 7f94a4c00640 1 --2- 172.21.15.196:0/4211790040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94a006f340 0x7f94a0157bb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9494000c00 tx=0x7f94940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:09.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.150+0000 7f949cc00640 1 -- 172.21.15.196:0/4211790040 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494037040 con 0x7f94a006f340 2024-04-28T13:32:09.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.150+0000 7f949cc00640 1 -- 172.21.15.196:0/4211790040 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9494042070 con 0x7f94a006f340 2024-04-28T13:32:09.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.150+0000 7f949cc00640 1 -- 172.21.15.196:0/4211790040 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94940308a0 con 0x7f94a006f340 2024-04-28T13:32:09.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.150+0000 7f94a7600640 1 -- 172.21.15.196:0/4211790040 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a00c7670 con 0x7f94a006f340 2024-04-28T13:32:09.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.150+0000 7f94a7600640 1 -- 172.21.15.196:0/4211790040 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a00c7950 con 0x7f94a006f340 2024-04-28T13:32:09.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.152+0000 7f94a7600640 1 -- 172.21.15.196:0/4211790040 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a006a290 con 0x7f94a006f340 2024-04-28T13:32:09.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.152+0000 7f949cc00640 1 -- 172.21.15.196:0/4211790040 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9494030a40 con 0x7f94a006f340 2024-04-28T13:32:09.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.153+0000 7f949cc00640 1 --2- 172.21.15.196:0/4211790040 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f948006c920 0x7f948006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:09.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.153+0000 7f949cc00640 1 -- 172.21.15.196:0/4211790040 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f94940ac3c0 con 0x7f94a006f340 2024-04-28T13:32:09.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.154+0000 7f949fe00640 1 --2- 172.21.15.196:0/4211790040 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f948006c920 0x7f948006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:09.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.155+0000 7f949fe00640 1 --2- 172.21.15.196:0/4211790040 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f948006c920 0x7f948006ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f948c000f60 tx=0x7f948c0079d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:09.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.158+0000 7f949cc00640 1 -- 172.21.15.196:0/4211790040 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f949407a6d0 con 0x7f94a006f340 2024-04-28T13:32:09.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.348+0000 7f94a7600640 1 -- 172.21.15.196:0/4211790040 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f94a0074660 con 0x7f94a006f340 2024-04-28T13:32:09.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.349+0000 7f949cc00640 1 -- 172.21.15.196:0/4211790040 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f9494079fd0 con 0x7f94a006f340 2024-04-28T13:32:09.349 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:32:09.352 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.352+0000 7f94a7600640 1 -- 172.21.15.196:0/4211790040 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f948006c920 msgr2=0x7f948006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:09.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.352+0000 7f94a7600640 1 --2- 172.21.15.196:0/4211790040 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f948006c920 0x7f948006ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f948c000f60 tx=0x7f948c0079d0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.352+0000 7f94a7600640 1 -- 172.21.15.196:0/4211790040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94a006f340 msgr2=0x7f94a0157bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:09.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.352+0000 7f94a7600640 1 --2- 172.21.15.196:0/4211790040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94a006f340 0x7f94a0157bb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9494000c00 tx=0x7f94940046c0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.352+0000 7f94a7600640 1 -- 172.21.15.196:0/4211790040 shutdown_connections 2024-04-28T13:32:09.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.352+0000 7f94a7600640 1 --2- 172.21.15.196:0/4211790040 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f948006c920 0x7f948006ede0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.352+0000 7f94a7600640 1 --2- 172.21.15.196:0/4211790040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94a00d0300 0x7f94a00c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.352+0000 7f94a7600640 1 --2- 172.21.15.196:0/4211790040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94a00b3e50 0x7f94a00c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.352+0000 7f94a7600640 1 --2- 172.21.15.196:0/4211790040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94a006f340 0x7f94a0157bb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.352+0000 7f94a7600640 1 -- 172.21.15.196:0/4211790040 >> 172.21.15.196:0/4211790040 conn(0x7f94a00cd8a0 msgr2=0x7f94a015c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:09.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.352+0000 7f94a7600640 1 -- 172.21.15.196:0/4211790040 shutdown_connections 2024-04-28T13:32:09.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.352+0000 7f94a7600640 1 -- 172.21.15.196:0/4211790040 wait complete. 2024-04-28T13:32:09.371 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T13:32:09.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.570+0000 7f4effe00640 1 Processor -- start 2024-04-28T13:32:09.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.571+0000 7f4effe00640 1 -- start start 2024-04-28T13:32:09.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.572+0000 7f4effe00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4f000d1760 0x7f4f000b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:09.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.572+0000 7f4effe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f0009ec80 0x7f4f0006df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:09.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.572+0000 7f4effe00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f0015e380 con 0x7f4f000b5ee0 2024-04-28T13:32:09.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.572+0000 7f4effe00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4f000b4b20 con 0x7f4f000d1760 2024-04-28T13:32:09.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.572+0000 7f4effe00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4f000b3f30 con 0x7f4f0009ec80 2024-04-28T13:32:09.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.573+0000 7f4eff400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f0009ec80 0x7f4f0006df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:09.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.573+0000 7f4eff400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f0009ec80 0x7f4f0006df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:59650/0 (socket says 172.21.15.196:59650) 2024-04-28T13:32:09.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.573+0000 7f4eff400640 1 -- 172.21.15.196:0/1210550098 learned_addr learned my addr 172.21.15.196:0/1210550098 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:09.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.573+0000 7f4efe000640 1 --2- 172.21.15.196:0/1210550098 >> v2:172.21.15.203:3300/0 conn(0x7f4f000d1760 0x7f4f000b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:09.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.574+0000 7f4eff400640 1 -- 172.21.15.196:0/1210550098 >> v2:172.21.15.203:3300/0 conn(0x7f4f000d1760 msgr2=0x7f4f000b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:09.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.574+0000 7f4eff400640 1 --2- 172.21.15.196:0/1210550098 >> v2:172.21.15.203:3300/0 conn(0x7f4f000d1760 0x7f4f000b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.574+0000 7f4eff400640 1 -- 172.21.15.196:0/1210550098 >> v1:172.21.15.203:6789/0 conn(0x7f4f000b5ee0 legacy=0x7f4f000d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:09.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.574+0000 7f4eff400640 1 -- 172.21.15.196:0/1210550098 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f000b41f0 con 0x7f4f0009ec80 2024-04-28T13:32:09.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.575+0000 7f4eff400640 1 --2- 172.21.15.196:0/1210550098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f0009ec80 0x7f4f0006df50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009e50 tx=0x7f4ef802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81324223906198d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:09.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.576+0000 7f4efd600640 1 -- 172.21.15.196:0/1210550098 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8036070 con 0x7f4f0009ec80 2024-04-28T13:32:09.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.576+0000 7f4efd600640 1 -- 172.21.15.196:0/1210550098 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef8031600 con 0x7f4f0009ec80 2024-04-28T13:32:09.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.576+0000 7f4efd600640 1 -- 172.21.15.196:0/1210550098 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef80319d0 con 0x7f4f0009ec80 2024-04-28T13:32:09.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.576+0000 7f4effe00640 1 -- 172.21.15.196:0/1210550098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f0009ec80 msgr2=0x7f4f0006df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:09.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.576+0000 7f4effe00640 1 --2- 172.21.15.196:0/1210550098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f0009ec80 0x7f4f0006df50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009e50 tx=0x7f4ef802a710 comp rx=0 tx=0).stop 2024-04-28T13:32:09.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.576+0000 7f4effe00640 1 -- 172.21.15.196:0/1210550098 shutdown_connections 2024-04-28T13:32:09.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.576+0000 7f4effe00640 1 --2- 172.21.15.196:0/1210550098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f0009ec80 0x7f4f0006df50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.576+0000 7f4effe00640 1 --2- 172.21.15.196:0/1210550098 >> v2:172.21.15.203:3300/0 conn(0x7f4f000d1760 0x7f4f000b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.577+0000 7f4effe00640 1 -- 172.21.15.196:0/1210550098 >> 172.21.15.196:0/1210550098 conn(0x7f4f0006c6c0 msgr2=0x7f4f0006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:09.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.577+0000 7f4effe00640 1 -- 172.21.15.196:0/1210550098 shutdown_connections 2024-04-28T13:32:09.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.577+0000 7f4effe00640 1 -- 172.21.15.196:0/1210550098 wait complete. 2024-04-28T13:32:09.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.579+0000 7f4effe00640 1 Processor -- start 2024-04-28T13:32:09.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.579+0000 7f4effe00640 1 -- start start 2024-04-28T13:32:09.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.580+0000 7f4effe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f0009ec80 0x7f4f000b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:09.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.580+0000 7f4effe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f000b5ee0 0x7f4f0009c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:09.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.580+0000 7f4effe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f000d1760 0x7f4f001557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:09.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.580+0000 7f4effe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f000b3c20 con 0x7f4f0009ec80 2024-04-28T13:32:09.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.580+0000 7f4effe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f000b47b0 con 0x7f4f000b5ee0 2024-04-28T13:32:09.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.580+0000 7f4effe00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4f000add70 con 0x7f4f000d1760 2024-04-28T13:32:09.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.580+0000 7f4efea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f0009ec80 0x7f4f000b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:09.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.580+0000 7f4efea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f0009ec80 0x7f4f000b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34276/0 (socket says 172.21.15.196:34276) 2024-04-28T13:32:09.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.580+0000 7f4efea00640 1 -- 172.21.15.196:0/566693658 learned_addr learned my addr 172.21.15.196:0/566693658 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:09.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.581+0000 7f4eff400640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f000d1760 0x7f4f001557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:09.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.581+0000 7f4efe000640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f000b5ee0 0x7f4f0009c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:09.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.581+0000 7f4efea00640 1 -- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f000d1760 msgr2=0x7f4f001557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:09.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.581+0000 7f4efea00640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f000d1760 0x7f4f001557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.581+0000 7f4efea00640 1 -- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f000b5ee0 msgr2=0x7f4f0009c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:09.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.581+0000 7f4efea00640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f000b5ee0 0x7f4f0009c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.581+0000 7f4efea00640 1 -- 172.21.15.196:0/566693658 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0009c9a0 con 0x7f4f0009ec80 2024-04-28T13:32:09.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.581+0000 7f4efe000640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f000b5ee0 0x7f4f0009c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:09.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.581+0000 7f4eff400640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f000d1760 0x7f4f001557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:32:09.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.581+0000 7f4efea00640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f0009ec80 0x7f4f000b2110 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000c450 tx=0x7f4ef000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:09.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.582+0000 7f4ef6a00640 1 -- 172.21.15.196:0/566693658 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef00052d0 con 0x7f4f0009ec80 2024-04-28T13:32:09.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.582+0000 7f4ef6a00640 1 -- 172.21.15.196:0/566693658 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef0005470 con 0x7f4f0009ec80 2024-04-28T13:32:09.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.582+0000 7f4ef6a00640 1 -- 172.21.15.196:0/566693658 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0012870 con 0x7f4f0009ec80 2024-04-28T13:32:09.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.582+0000 7f4effe00640 1 -- 172.21.15.196:0/566693658 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0009cc80 con 0x7f4f0009ec80 2024-04-28T13:32:09.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.582+0000 7f4effe00640 1 -- 172.21.15.196:0/566693658 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f0009d240 con 0x7f4f0009ec80 2024-04-28T13:32:09.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.583+0000 7f4effe00640 1 -- 172.21.15.196:0/566693658 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0006a290 con 0x7f4f0009ec80 2024-04-28T13:32:09.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.584+0000 7f4ef6a00640 1 -- 172.21.15.196:0/566693658 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ef0012a10 con 0x7f4f0009ec80 2024-04-28T13:32:09.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.585+0000 7f4ef6a00640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4ee006c8f0 0x7f4ee006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:09.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.585+0000 7f4ef6a00640 1 -- 172.21.15.196:0/566693658 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4ef008b7d0 con 0x7f4f0009ec80 2024-04-28T13:32:09.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.586+0000 7f4efe000640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4ee006c8f0 0x7f4ee006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:09.587 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.587+0000 7f4efe000640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4ee006c8f0 0x7f4ee006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4f0015cfb0 tx=0x7f4eec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:09.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.589+0000 7f4ef6a00640 1 -- 172.21.15.196:0/566693658 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ef0059b80 con 0x7f4f0009ec80 2024-04-28T13:32:09.687 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:32:09.697 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:32:09.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.808+0000 7f4effe00640 1 -- 172.21.15.196:0/566693658 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4f0006ee60 con 0x7f4f0009ec80 2024-04-28T13:32:09.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.808+0000 7f4ef6a00640 1 -- 172.21.15.196:0/566693658 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f4ef0059480 con 0x7f4f0009ec80 2024-04-28T13:32:09.808 INFO:teuthology.orchestra.run.smithi196.stderr:listed 0 entries 2024-04-28T13:32:09.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.811+0000 7f4effe00640 1 -- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4ee006c8f0 msgr2=0x7f4ee006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:09.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.811+0000 7f4effe00640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4ee006c8f0 0x7f4ee006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4f0015cfb0 tx=0x7f4eec008040 comp rx=0 tx=0).stop 2024-04-28T13:32:09.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.811+0000 7f4effe00640 1 -- 172.21.15.196:0/566693658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f0009ec80 msgr2=0x7f4f000b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:09.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.811+0000 7f4effe00640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f0009ec80 0x7f4f000b2110 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000c450 tx=0x7f4ef000c910 comp rx=0 tx=0).stop 2024-04-28T13:32:09.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.811+0000 7f4effe00640 1 -- 172.21.15.196:0/566693658 shutdown_connections 2024-04-28T13:32:09.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.811+0000 7f4effe00640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4ee006c8f0 0x7f4ee006edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.811+0000 7f4effe00640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f000d1760 0x7f4f001557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.811+0000 7f4effe00640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f000b5ee0 0x7f4f0009c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.811+0000 7f4effe00640 1 --2- 172.21.15.196:0/566693658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f0009ec80 0x7f4f000b2110 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:09.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.811+0000 7f4effe00640 1 -- 172.21.15.196:0/566693658 >> 172.21.15.196:0/566693658 conn(0x7f4f0006c6c0 msgr2=0x7f4f00071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:09.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.811+0000 7f4effe00640 1 -- 172.21.15.196:0/566693658 shutdown_connections 2024-04-28T13:32:09.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:09.811+0000 7f4effe00640 1 -- 172.21.15.196:0/566693658 wait complete. 2024-04-28T13:32:09.830 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-28T13:32:09.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:09.830 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-28T13:32:09.925 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:32:09.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:09.925 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-28T13:32:09.991 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:32:09.991 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:10.046 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:32:10.046 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:10.099 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:32:10.099 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:10.151 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:32:10.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.359+0000 7f923be00640 1 Processor -- start 2024-04-28T13:32:10.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.359+0000 7f923be00640 1 -- start start 2024-04-28T13:32:10.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.361+0000 7f923be00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f923c073370 0x7f923c072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.361+0000 7f923be00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f923c0b3c20 0x7f923c0b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.361+0000 7f923be00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f923c0b47a0 con 0x7f923c0d0540 2024-04-28T13:32:10.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.361+0000 7f923be00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f923c0738e0 con 0x7f923c0b3c20 2024-04-28T13:32:10.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.361+0000 7f923be00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f923c0aea80 con 0x7f923c073370 2024-04-28T13:32:10.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.361+0000 7f923aa00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f923c0d0540 0x7f923c0af560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43392/0 (socket says 172.21.15.196:43392) 2024-04-28T13:32:10.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.361+0000 7f923aa00640 1 -- 172.21.15.196:0/4037069509 learned_addr learned my addr 172.21.15.196:0/4037069509 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:10.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.361+0000 7f923a000640 1 --2- 172.21.15.196:0/4037069509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f923c073370 0x7f923c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.361+0000 7f9239600640 1 -- 172.21.15.196:0/4037069509 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 725401612 0 0) 0x7f923c0b47a0 con 0x7f923c0d0540 2024-04-28T13:32:10.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.361+0000 7f923b400640 1 --2- 172.21.15.196:0/4037069509 >> v2:172.21.15.203:3300/0 conn(0x7f923c0b3c20 0x7f923c0b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f9239600640 1 -- 172.21.15.196:0/4037069509 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9220003520 con 0x7f923c0d0540 2024-04-28T13:32:10.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f9239600640 1 -- 172.21.15.196:0/4037069509 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4214071156 0 0) 0x7f9220003520 con 0x7f923c0d0540 2024-04-28T13:32:10.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f9239600640 1 -- 172.21.15.196:0/4037069509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f923c073370 msgr2=0x7f923c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:10.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f9239600640 1 --2- 172.21.15.196:0/4037069509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f923c073370 0x7f923c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f9239600640 1 -- 172.21.15.196:0/4037069509 >> v2:172.21.15.203:3300/0 conn(0x7f923c0b3c20 msgr2=0x7f923c0b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:10.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f9239600640 1 --2- 172.21.15.196:0/4037069509 >> v2:172.21.15.203:3300/0 conn(0x7f923c0b3c20 0x7f923c0b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f9239600640 1 -- 172.21.15.196:0/4037069509 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923c0aed60 con 0x7f923c0d0540 2024-04-28T13:32:10.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f9239600640 1 -- 172.21.15.196:0/4037069509 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f922c001670 con 0x7f923c0d0540 2024-04-28T13:32:10.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f9239600640 1 -- 172.21.15.196:0/4037069509 >> v1:172.21.15.196:6789/0 conn(0x7f923c0d0540 legacy=0x7f923c0af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:10.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f9239600640 1 --2- 172.21.15.196:0/4037069509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92200039b0 0x7f9220003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f9239600640 1 --2- 172.21.15.196:0/4037069509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9220004800 0x7f9220004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f9239600640 1 --2- 172.21.15.196:0/4037069509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92200051c0 0x7f922000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f9239600640 1 -- 172.21.15.196:0/4037069509 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f923c0738e0 con 0x7f9220004800 2024-04-28T13:32:10.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f9239600640 1 -- 172.21.15.196:0/4037069509 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f923c0aea80 con 0x7f92200051c0 2024-04-28T13:32:10.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f9239600640 1 -- 172.21.15.196:0/4037069509 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f922000c0e0 con 0x7f92200039b0 2024-04-28T13:32:10.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f923a000640 1 --2- 172.21.15.196:0/4037069509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f923c073370 0x7f923c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:32:10.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.362+0000 7f923a000640 1 --2- 172.21.15.196:0/4037069509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9220004800 0x7f9220004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.363+0000 7f923b400640 1 --2- 172.21.15.196:0/4037069509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92200051c0 0x7f922000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.363+0000 7f923aa00640 1 --2- 172.21.15.196:0/4037069509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92200039b0 0x7f9220003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.363+0000 7f923a000640 1 -- 172.21.15.196:0/4037069509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92200039b0 msgr2=0x7f9220003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:10.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.363+0000 7f923a000640 1 --2- 172.21.15.196:0/4037069509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92200039b0 0x7f9220003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.363+0000 7f923a000640 1 -- 172.21.15.196:0/4037069509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92200051c0 msgr2=0x7f922000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:10.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.363+0000 7f923a000640 1 --2- 172.21.15.196:0/4037069509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92200051c0 0x7f922000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.363+0000 7f923a000640 1 -- 172.21.15.196:0/4037069509 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9220003520 con 0x7f9220004800 2024-04-28T13:32:10.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.364+0000 7f923a000640 1 --2- 172.21.15.196:0/4037069509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9220004800 0x7f9220004c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9228006fd0 tx=0x7f9220005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a721891278d8cf79 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:10.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.364+0000 7f9239600640 1 -- 172.21.15.196:0/4037069509 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228037070 con 0x7f9220004800 2024-04-28T13:32:10.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.364+0000 7f9239600640 1 -- 172.21.15.196:0/4037069509 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f922802aae0 con 0x7f9220004800 2024-04-28T13:32:10.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.364+0000 7f9239600640 1 -- 172.21.15.196:0/4037069509 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922800f040 con 0x7f9220004800 2024-04-28T13:32:10.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.365+0000 7f923be00640 1 -- 172.21.15.196:0/4037069509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9220004800 msgr2=0x7f9220004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:10.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.365+0000 7f923be00640 1 --2- 172.21.15.196:0/4037069509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9220004800 0x7f9220004c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9228006fd0 tx=0x7f9220005830 comp rx=0 tx=0).stop 2024-04-28T13:32:10.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.365+0000 7f923b400640 1 -- 172.21.15.196:0/4037069509 reap_dead start 2024-04-28T13:32:10.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.365+0000 7f923be00640 1 -- 172.21.15.196:0/4037069509 shutdown_connections 2024-04-28T13:32:10.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.365+0000 7f923be00640 1 -- 172.21.15.196:0/4037069509 >> 172.21.15.196:0/4037069509 conn(0x7f923c06c6c0 msgr2=0x7f923c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:10.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.365+0000 7f923be00640 1 -- 172.21.15.196:0/4037069509 shutdown_connections 2024-04-28T13:32:10.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.365+0000 7f923be00640 1 -- 172.21.15.196:0/4037069509 wait complete. 2024-04-28T13:32:10.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.367+0000 7f923be00640 1 Processor -- start 2024-04-28T13:32:10.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.367+0000 7f923be00640 1 -- start start 2024-04-28T13:32:10.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.368+0000 7f923be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9220004800 0x7f923c14c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.368+0000 7f923be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c0b14b0 0x7f923c0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.368+0000 7f923be00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f923c156020 0x7f923c1564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.368+0000 7f923be00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f923c0d0920 con 0x7f923c0b14b0 2024-04-28T13:32:10.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.368+0000 7f923be00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f923c0ae6c0 con 0x7f9220004800 2024-04-28T13:32:10.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.368+0000 7f923be00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f923c14f350 con 0x7f923c156020 2024-04-28T13:32:10.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.368+0000 7f923a000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c0b14b0 0x7f923c0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.368+0000 7f923a000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c0b14b0 0x7f923c0b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34298/0 (socket says 172.21.15.196:34298) 2024-04-28T13:32:10.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.368+0000 7f923a000640 1 -- 172.21.15.196:0/1680303613 learned_addr learned my addr 172.21.15.196:0/1680303613 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:10.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.368+0000 7f923aa00640 1 --2- 172.21.15.196:0/1680303613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9220004800 0x7f923c14c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.368+0000 7f923b400640 1 --2- 172.21.15.196:0/1680303613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f923c156020 0x7f923c1564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.368+0000 7f923a000640 1 -- 172.21.15.196:0/1680303613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f923c156020 msgr2=0x7f923c1564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:10.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.368+0000 7f923a000640 1 --2- 172.21.15.196:0/1680303613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f923c156020 0x7f923c1564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.369+0000 7f923a000640 1 -- 172.21.15.196:0/1680303613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9220004800 msgr2=0x7f923c14c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:10.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.369+0000 7f923a000640 1 --2- 172.21.15.196:0/1680303613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9220004800 0x7f923c14c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.369+0000 7f923a000640 1 -- 172.21.15.196:0/1680303613 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923c14e2f0 con 0x7f923c0b14b0 2024-04-28T13:32:10.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.369+0000 7f923a000640 1 --2- 172.21.15.196:0/1680303613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c0b14b0 0x7f923c0b1930 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f922802a5d0 tx=0x7f9228004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:10.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.369+0000 7f9232a00640 1 -- 172.21.15.196:0/1680303613 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228040020 con 0x7f923c0b14b0 2024-04-28T13:32:10.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.369+0000 7f9232a00640 1 -- 172.21.15.196:0/1680303613 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9228038040 con 0x7f923c0b14b0 2024-04-28T13:32:10.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.369+0000 7f9232a00640 1 -- 172.21.15.196:0/1680303613 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228037070 con 0x7f923c0b14b0 2024-04-28T13:32:10.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.369+0000 7f923be00640 1 -- 172.21.15.196:0/1680303613 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f923c14e590 con 0x7f923c0b14b0 2024-04-28T13:32:10.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.369+0000 7f923be00640 1 -- 172.21.15.196:0/1680303613 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f923c14eb50 con 0x7f923c0b14b0 2024-04-28T13:32:10.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.370+0000 7f923be00640 1 -- 172.21.15.196:0/1680303613 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f923c06a290 con 0x7f923c0b14b0 2024-04-28T13:32:10.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.371+0000 7f9232a00640 1 -- 172.21.15.196:0/1680303613 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9228043020 con 0x7f923c0b14b0 2024-04-28T13:32:10.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.372+0000 7f9232a00640 1 --2- 172.21.15.196:0/1680303613 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f921c06c920 0x7f921c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.372+0000 7f9232a00640 1 -- 172.21.15.196:0/1680303613 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f92280abf30 con 0x7f923c0b14b0 2024-04-28T13:32:10.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.373+0000 7f923aa00640 1 --2- 172.21.15.196:0/1680303613 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f921c06c920 0x7f921c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.374+0000 7f923aa00640 1 --2- 172.21.15.196:0/1680303613 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f921c06c920 0x7f921c06ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f922c001200 tx=0x7f922c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:10.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.377+0000 7f9232a00640 1 -- 172.21.15.196:0/1680303613 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f922804a020 con 0x7f923c0b14b0 2024-04-28T13:32:10.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.577+0000 7f923be00640 1 -- 172.21.15.196:0/1680303613 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f923c15f830 con 0x7f923c0b14b0 2024-04-28T13:32:10.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.579+0000 7f9232a00640 1 -- 172.21.15.196:0/1680303613 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16890 (secure 0 0 0) 0x7f922807a240 con 0x7f923c0b14b0 2024-04-28T13:32:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:10.579 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:03.882452+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:32:10.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.581+0000 7f923be00640 1 -- 172.21.15.196:0/1680303613 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f921c06c920 msgr2=0x7f921c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:10.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.581+0000 7f923be00640 1 --2- 172.21.15.196:0/1680303613 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f921c06c920 0x7f921c06ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f922c001200 tx=0x7f922c009040 comp rx=0 tx=0).stop 2024-04-28T13:32:10.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.581+0000 7f923be00640 1 -- 172.21.15.196:0/1680303613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c0b14b0 msgr2=0x7f923c0b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:10.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.581+0000 7f923be00640 1 --2- 172.21.15.196:0/1680303613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c0b14b0 0x7f923c0b1930 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f922802a5d0 tx=0x7f9228004730 comp rx=0 tx=0).stop 2024-04-28T13:32:10.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.581+0000 7f923be00640 1 -- 172.21.15.196:0/1680303613 shutdown_connections 2024-04-28T13:32:10.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.581+0000 7f923be00640 1 --2- 172.21.15.196:0/1680303613 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f921c06c920 0x7f921c06ede0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.582+0000 7f923be00640 1 --2- 172.21.15.196:0/1680303613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f923c156020 0x7f923c1564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.582+0000 7f923be00640 1 --2- 172.21.15.196:0/1680303613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c0b14b0 0x7f923c0b1930 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.582+0000 7f923be00640 1 --2- 172.21.15.196:0/1680303613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9220004800 0x7f923c14c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.582+0000 7f923be00640 1 -- 172.21.15.196:0/1680303613 >> 172.21.15.196:0/1680303613 conn(0x7f923c06c6c0 msgr2=0x7f923c153290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:10.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.582+0000 7f923be00640 1 -- 172.21.15.196:0/1680303613 shutdown_connections 2024-04-28T13:32:10.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.582+0000 7f923be00640 1 -- 172.21.15.196:0/1680303613 wait complete. 2024-04-28T13:32:10.602 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:31:19.280975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:31:20.037066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:32:10.603 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:32:10.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.804+0000 7f5a24600640 1 Processor -- start 2024-04-28T13:32:10.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.804+0000 7f5a24600640 1 -- start start 2024-04-28T13:32:10.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.806+0000 7f5a24600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a1c143840 0x7f5a1c0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.806+0000 7f5a24600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5a1c0b4520 0x7f5a1c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.806+0000 7f5a24600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a1c15dca0 con 0x7f5a1c0b53f0 2024-04-28T13:32:10.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.806+0000 7f5a24600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f5a1c143c20 con 0x7f5a1c0b4520 2024-04-28T13:32:10.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.806+0000 7f5a24600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1c15df80 con 0x7f5a1c143840 2024-04-28T13:32:10.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.806+0000 7f5a22600640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f5a1c0b53f0 0x7f5a1c160e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43408/0 (socket says 172.21.15.196:43408) 2024-04-28T13:32:10.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.806+0000 7f5a22600640 1 -- 172.21.15.196:0/2705062958 learned_addr learned my addr 172.21.15.196:0/2705062958 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:10.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.806+0000 7f5a21c00640 1 --2- 172.21.15.196:0/2705062958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a1c143840 0x7f5a1c0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.806+0000 7f5a21200640 1 --2- 172.21.15.196:0/2705062958 >> v2:172.21.15.203:3300/0 conn(0x7f5a1c0b4520 0x7f5a1c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 -- 172.21.15.196:0/2705062958 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1042868964 0 0) 0x7f5a1c15dca0 con 0x7f5a1c0b53f0 2024-04-28T13:32:10.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 -- 172.21.15.196:0/2705062958 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a04003520 con 0x7f5a1c0b53f0 2024-04-28T13:32:10.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 -- 172.21.15.196:0/2705062958 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2352992037 0 0) 0x7f5a04003520 con 0x7f5a1c0b53f0 2024-04-28T13:32:10.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 -- 172.21.15.196:0/2705062958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a1c143840 msgr2=0x7f5a1c0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:10.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 --2- 172.21.15.196:0/2705062958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a1c143840 0x7f5a1c0b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 -- 172.21.15.196:0/2705062958 >> v2:172.21.15.203:3300/0 conn(0x7f5a1c0b4520 msgr2=0x7f5a1c0b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:10.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 --2- 172.21.15.196:0/2705062958 >> v2:172.21.15.203:3300/0 conn(0x7f5a1c0b4520 0x7f5a1c0b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 -- 172.21.15.196:0/2705062958 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c15e260 con 0x7f5a1c0b53f0 2024-04-28T13:32:10.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 -- 172.21.15.196:0/2705062958 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f5a140019f0 con 0x7f5a1c0b53f0 2024-04-28T13:32:10.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 -- 172.21.15.196:0/2705062958 >> v1:172.21.15.196:6789/0 conn(0x7f5a1c0b53f0 legacy=0x7f5a1c160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:10.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 --2- 172.21.15.196:0/2705062958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a040039b0 0x7f5a04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 --2- 172.21.15.196:0/2705062958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a1c143840 0x7f5a04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 --2- 172.21.15.196:0/2705062958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a04004d40 0x7f5a040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 -- 172.21.15.196:0/2705062958 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c143c20 con 0x7f5a040039b0 2024-04-28T13:32:10.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 -- 172.21.15.196:0/2705062958 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c15df80 con 0x7f5a04004d40 2024-04-28T13:32:10.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.807+0000 7f5a1be00640 1 -- 172.21.15.196:0/2705062958 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5a0400b9c0 con 0x7f5a1c143840 2024-04-28T13:32:10.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.808+0000 7f5a21c00640 1 --2- 172.21.15.196:0/2705062958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a040039b0 0x7f5a04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.808+0000 7f5a22600640 1 --2- 172.21.15.196:0/2705062958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a04004d40 0x7f5a040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.808+0000 7f5a21200640 1 --2- 172.21.15.196:0/2705062958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a1c143840 0x7f5a04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.808+0000 7f5a21c00640 1 -- 172.21.15.196:0/2705062958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a1c143840 msgr2=0x7f5a04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:10.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.808+0000 7f5a21c00640 1 --2- 172.21.15.196:0/2705062958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a1c143840 0x7f5a04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.808+0000 7f5a21c00640 1 -- 172.21.15.196:0/2705062958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a04004d40 msgr2=0x7f5a040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:10.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.808+0000 7f5a21c00640 1 --2- 172.21.15.196:0/2705062958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a04004d40 0x7f5a040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.808+0000 7f5a21c00640 1 -- 172.21.15.196:0/2705062958 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a04003520 con 0x7f5a040039b0 2024-04-28T13:32:10.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.809+0000 7f5a21c00640 1 --2- 172.21.15.196:0/2705062958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a040039b0 0x7f5a04003dd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009650 tx=0x7f5a04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f0f8f71b11b9fa server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:10.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.810+0000 7f5a1be00640 1 -- 172.21.15.196:0/2705062958 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10038020 con 0x7f5a040039b0 2024-04-28T13:32:10.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.810+0000 7f5a1be00640 1 -- 172.21.15.196:0/2705062958 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a10034480 con 0x7f5a040039b0 2024-04-28T13:32:10.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.810+0000 7f5a1be00640 1 -- 172.21.15.196:0/2705062958 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10009690 con 0x7f5a040039b0 2024-04-28T13:32:10.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.810+0000 7f5a24600640 1 -- 172.21.15.196:0/2705062958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a040039b0 msgr2=0x7f5a04003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:10.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.810+0000 7f5a24600640 1 --2- 172.21.15.196:0/2705062958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a040039b0 0x7f5a04003dd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009650 tx=0x7f5a04005830 comp rx=0 tx=0).stop 2024-04-28T13:32:10.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.810+0000 7f5a22600640 1 -- 172.21.15.196:0/2705062958 reap_dead start 2024-04-28T13:32:10.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.811+0000 7f5a24600640 1 -- 172.21.15.196:0/2705062958 shutdown_connections 2024-04-28T13:32:10.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.811+0000 7f5a24600640 1 -- 172.21.15.196:0/2705062958 >> 172.21.15.196:0/2705062958 conn(0x7f5a1c0cd8a0 msgr2=0x7f5a1c0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:10.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.811+0000 7f5a24600640 1 -- 172.21.15.196:0/2705062958 shutdown_connections 2024-04-28T13:32:10.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.811+0000 7f5a24600640 1 -- 172.21.15.196:0/2705062958 wait complete. 2024-04-28T13:32:10.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.813+0000 7f5a24600640 1 Processor -- start 2024-04-28T13:32:10.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.813+0000 7f5a24600640 1 -- start start 2024-04-28T13:32:10.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a24600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a040039b0 0x7f5a1c1f01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a24600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a1c0c85e0 0x7f5a1c0c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a24600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a1c1443e0 0x7f5a1c144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a24600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c15da40 con 0x7f5a040039b0 2024-04-28T13:32:10.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a24600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c15d8c0 con 0x7f5a1c1443e0 2024-04-28T13:32:10.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a24600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1c0d0000 con 0x7f5a1c0c85e0 2024-04-28T13:32:10.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a21c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a040039b0 0x7f5a1c1f01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a21c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a040039b0 0x7f5a1c1f01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34314/0 (socket says 172.21.15.196:34314) 2024-04-28T13:32:10.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a21c00640 1 -- 172.21.15.196:0/1295705317 learned_addr learned my addr 172.21.15.196:0/1295705317 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:10.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a22600640 1 --2- 172.21.15.196:0/1295705317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a1c1443e0 0x7f5a1c144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a21c00640 1 -- 172.21.15.196:0/1295705317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a1c0c85e0 msgr2=0x7f5a1c0c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:10.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a21200640 1 --2- 172.21.15.196:0/1295705317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a1c0c85e0 0x7f5a1c0c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a21c00640 1 --2- 172.21.15.196:0/1295705317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a1c0c85e0 0x7f5a1c0c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a21c00640 1 -- 172.21.15.196:0/1295705317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a1c1443e0 msgr2=0x7f5a1c144860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:10.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a21c00640 1 --2- 172.21.15.196:0/1295705317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a1c1443e0 0x7f5a1c144860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:10.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.815+0000 7f5a21c00640 1 -- 172.21.15.196:0/1295705317 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c145110 con 0x7f5a040039b0 2024-04-28T13:32:10.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.816+0000 7f5a22600640 1 --2- 172.21.15.196:0/1295705317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a1c1443e0 0x7f5a1c144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:10.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.816+0000 7f5a21c00640 1 --2- 172.21.15.196:0/1295705317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a040039b0 0x7f5a1c1f01d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5a10037040 tx=0x7f5a10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:10.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.816+0000 7f5a19600640 1 -- 172.21.15.196:0/1295705317 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10038040 con 0x7f5a040039b0 2024-04-28T13:32:10.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.816+0000 7f5a19600640 1 -- 172.21.15.196:0/1295705317 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a100044e0 con 0x7f5a040039b0 2024-04-28T13:32:10.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.816+0000 7f5a19600640 1 -- 172.21.15.196:0/1295705317 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10003850 con 0x7f5a040039b0 2024-04-28T13:32:10.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.816+0000 7f5a24600640 1 -- 172.21.15.196:0/1295705317 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1c1453f0 con 0x7f5a040039b0 2024-04-28T13:32:10.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.817+0000 7f5a24600640 1 -- 172.21.15.196:0/1295705317 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a1c1459b0 con 0x7f5a040039b0 2024-04-28T13:32:10.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.818+0000 7f5a24600640 1 -- 172.21.15.196:0/1295705317 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1c06a290 con 0x7f5a040039b0 2024-04-28T13:32:10.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.819+0000 7f5a19600640 1 -- 172.21.15.196:0/1295705317 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a1003b020 con 0x7f5a040039b0 2024-04-28T13:32:10.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.820+0000 7f5a19600640 1 --2- 172.21.15.196:0/1295705317 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f59fc06c920 0x7f59fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:10.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.820+0000 7f5a19600640 1 -- 172.21.15.196:0/1295705317 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f5a100b0d40 con 0x7f5a040039b0 2024-04-28T13:32:10.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.821+0000 7f5a21200640 1 --2- 172.21.15.196:0/1295705317 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f59fc06c920 0x7f59fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:10.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.822+0000 7f5a21200640 1 --2- 172.21.15.196:0/1295705317 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f59fc06c920 0x7f59fc06ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c146e40 tx=0x7f5a08008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:10.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:10.824+0000 7f5a19600640 1 -- 172.21.15.196:0/1295705317 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5a10030040 con 0x7f5a040039b0 2024-04-28T13:32:11.018 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.018+0000 7f5a24600640 1 -- 172.21.15.196:0/1295705317 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5a1c0ce420 con 0x7f5a040039b0 2024-04-28T13:32:11.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.019+0000 7f5a19600640 1 -- 172.21.15.196:0/1295705317 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f5a1007f0e0 con 0x7f5a040039b0 2024-04-28T13:32:11.019 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:32:11.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.022+0000 7f5a24600640 1 -- 172.21.15.196:0/1295705317 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f59fc06c920 msgr2=0x7f59fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:11.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.022+0000 7f5a24600640 1 --2- 172.21.15.196:0/1295705317 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f59fc06c920 0x7f59fc06ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c146e40 tx=0x7f5a08008040 comp rx=0 tx=0).stop 2024-04-28T13:32:11.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.022+0000 7f5a24600640 1 -- 172.21.15.196:0/1295705317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a040039b0 msgr2=0x7f5a1c1f01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:11.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.022+0000 7f5a24600640 1 --2- 172.21.15.196:0/1295705317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a040039b0 0x7f5a1c1f01d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5a10037040 tx=0x7f5a10004730 comp rx=0 tx=0).stop 2024-04-28T13:32:11.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.022+0000 7f5a24600640 1 -- 172.21.15.196:0/1295705317 shutdown_connections 2024-04-28T13:32:11.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.022+0000 7f5a24600640 1 --2- 172.21.15.196:0/1295705317 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f59fc06c920 0x7f59fc06ede0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.022+0000 7f5a24600640 1 --2- 172.21.15.196:0/1295705317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a1c1443e0 0x7f5a1c144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.022+0000 7f5a24600640 1 --2- 172.21.15.196:0/1295705317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a1c0c85e0 0x7f5a1c0c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.022+0000 7f5a24600640 1 --2- 172.21.15.196:0/1295705317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a040039b0 0x7f5a1c1f01d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.022+0000 7f5a24600640 1 -- 172.21.15.196:0/1295705317 >> 172.21.15.196:0/1295705317 conn(0x7f5a1c0cd8a0 msgr2=0x7f5a1c15f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:11.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.022+0000 7f5a24600640 1 -- 172.21.15.196:0/1295705317 shutdown_connections 2024-04-28T13:32:11.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.022+0000 7f5a24600640 1 -- 172.21.15.196:0/1295705317 wait complete. 2024-04-28T13:32:11.040 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:32:11.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.236+0000 7f0562800640 1 Processor -- start 2024-04-28T13:32:11.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.237+0000 7f0562800640 1 -- start start 2024-04-28T13:32:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.238+0000 7f0562800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f055c0d0600 0x7f055c0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.238+0000 7f0562800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f055c0cfb80 con 0x7f055c0d3c10 2024-04-28T13:32:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.238+0000 7f0562800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f055c0d2ee0 con 0x7f055c0d3830 2024-04-28T13:32:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.238+0000 7f0562800640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f055c0d0f90 con 0x7f055c0d0600 2024-04-28T13:32:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.238+0000 7f0560a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f055c0d3c10 0x7f055c0b7e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43412/0 (socket says 172.21.15.196:43412) 2024-04-28T13:32:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.238+0000 7f0560a00640 1 -- 172.21.15.196:0/2857240537 learned_addr learned my addr 172.21.15.196:0/2857240537 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:11.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f0561e00640 1 --2- 172.21.15.196:0/2857240537 >> v2:172.21.15.203:3300/0 conn(0x7f055c0d0600 0x7f055c0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:11.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f055be00640 1 -- 172.21.15.196:0/2857240537 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3542701803 0 0) 0x7f0548001110 con 0x7f055c0d3c10 2024-04-28T13:32:11.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f055be00640 1 -- 172.21.15.196:0/2857240537 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0540003520 con 0x7f055c0d3c10 2024-04-28T13:32:11.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f055be00640 1 -- 172.21.15.196:0/2857240537 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1795900742 0 0) 0x7f0540003520 con 0x7f055c0d3c10 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f055be00640 1 -- 172.21.15.196:0/2857240537 >> v2:172.21.15.203:3300/0 conn(0x7f055c0d0600 msgr2=0x7f055c0d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f055be00640 1 --2- 172.21.15.196:0/2857240537 >> v2:172.21.15.203:3300/0 conn(0x7f055c0d0600 0x7f055c0d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f055be00640 1 -- 172.21.15.196:0/2857240537 >> v1:172.21.15.203:6789/0 conn(0x7f055c0d3830 legacy=0x7f055c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f055be00640 1 -- 172.21.15.196:0/2857240537 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c0d31c0 con 0x7f055c0d3c10 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f055be00640 1 -- 172.21.15.196:0/2857240537 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f0548001840 con 0x7f055c0d3c10 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f055be00640 1 -- 172.21.15.196:0/2857240537 >> v1:172.21.15.196:6789/0 conn(0x7f055c0d3c10 legacy=0x7f055c0b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f055be00640 1 --2- 172.21.15.196:0/2857240537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05400039b0 0x7f0540003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f055be00640 1 --2- 172.21.15.196:0/2857240537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004800 0x7f0540004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f055be00640 1 --2- 172.21.15.196:0/2857240537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05400051c0 0x7f054000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f055be00640 1 -- 172.21.15.196:0/2857240537 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f054000c030 con 0x7f0540004800 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f055be00640 1 -- 172.21.15.196:0/2857240537 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f054000c1e0 con 0x7f05400051c0 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.239+0000 7f055be00640 1 -- 172.21.15.196:0/2857240537 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f054000c3e0 con 0x7f05400039b0 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.240+0000 7f0560a00640 1 --2- 172.21.15.196:0/2857240537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004800 0x7f0540004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.240+0000 7f0560a00640 1 -- 172.21.15.196:0/2857240537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05400039b0 msgr2=0x7f0540003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.240+0000 7f0560a00640 1 --2- 172.21.15.196:0/2857240537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05400039b0 0x7f0540003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.240+0000 7f0560a00640 1 -- 172.21.15.196:0/2857240537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05400051c0 msgr2=0x7f054000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:11.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.240+0000 7f0560a00640 1 --2- 172.21.15.196:0/2857240537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05400051c0 0x7f054000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.240+0000 7f0560a00640 1 -- 172.21.15.196:0/2857240537 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0540003520 con 0x7f0540004800 2024-04-28T13:32:11.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.240+0000 7f0561e00640 1 -- 172.21.15.196:0/2857240537 reap_dead start 2024-04-28T13:32:11.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.241+0000 7f0560a00640 1 --2- 172.21.15.196:0/2857240537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004800 0x7f0540004c80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f054800c4c0 tx=0x7f0540005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbcb4f6bf26c178 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:11.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.241+0000 7f055be00640 1 -- 172.21.15.196:0/2857240537 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054800c0e0 con 0x7f0540004800 2024-04-28T13:32:11.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.241+0000 7f055be00640 1 -- 172.21.15.196:0/2857240537 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0548034070 con 0x7f0540004800 2024-04-28T13:32:11.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.241+0000 7f055be00640 1 -- 172.21.15.196:0/2857240537 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548005d70 con 0x7f0540004800 2024-04-28T13:32:11.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.242+0000 7f0562800640 1 -- 172.21.15.196:0/2857240537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004800 msgr2=0x7f0540004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:11.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.242+0000 7f0562800640 1 --2- 172.21.15.196:0/2857240537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004800 0x7f0540004c80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f054800c4c0 tx=0x7f0540005830 comp rx=0 tx=0).stop 2024-04-28T13:32:11.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.242+0000 7f0562800640 1 -- 172.21.15.196:0/2857240537 shutdown_connections 2024-04-28T13:32:11.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.242+0000 7f0562800640 1 --2- 172.21.15.196:0/2857240537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0540004800 0x7f0540004c80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.242+0000 7f0562800640 1 -- 172.21.15.196:0/2857240537 >> 172.21.15.196:0/2857240537 conn(0x7f055c0d4fe0 msgr2=0x7f055c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:11.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.242+0000 7f0562800640 1 -- 172.21.15.196:0/2857240537 shutdown_connections 2024-04-28T13:32:11.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.242+0000 7f0562800640 1 -- 172.21.15.196:0/2857240537 wait complete. 2024-04-28T13:32:11.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.244+0000 7f0562800640 1 Processor -- start 2024-04-28T13:32:11.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.244+0000 7f0562800640 1 -- start start 2024-04-28T13:32:11.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.245+0000 7f0562800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0540004800 0x7f055c15e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.245+0000 7f0562800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f055c0cb4e0 0x7f055c15f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.245+0000 7f0562800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f055c14c360 0x7f055c0b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.245+0000 7f0562800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f055c0d5d90 con 0x7f055c0cb4e0 2024-04-28T13:32:11.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.245+0000 7f0562800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f055c0d00e0 con 0x7f0540004800 2024-04-28T13:32:11.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.245+0000 7f0562800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f055c0d2bb0 con 0x7f055c14c360 2024-04-28T13:32:11.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.245+0000 7f0560a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f055c0cb4e0 0x7f055c15f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:11.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.245+0000 7f0560a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f055c0cb4e0 0x7f055c15f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34324/0 (socket says 172.21.15.196:34324) 2024-04-28T13:32:11.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.245+0000 7f0560a00640 1 -- 172.21.15.196:0/80281784 learned_addr learned my addr 172.21.15.196:0/80281784 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:11.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.246+0000 7f0561e00640 1 --2- 172.21.15.196:0/80281784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f055c14c360 0x7f055c0b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:11.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.246+0000 7f0560a00640 1 -- 172.21.15.196:0/80281784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f055c14c360 msgr2=0x7f055c0b9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:11.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.246+0000 7f0561400640 1 --2- 172.21.15.196:0/80281784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0540004800 0x7f055c15e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:11.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.246+0000 7f0560a00640 1 --2- 172.21.15.196:0/80281784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f055c14c360 0x7f055c0b9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.246+0000 7f0560a00640 1 -- 172.21.15.196:0/80281784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0540004800 msgr2=0x7f055c15e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:11.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.246+0000 7f0560a00640 1 --2- 172.21.15.196:0/80281784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0540004800 0x7f055c15e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.246+0000 7f0560a00640 1 -- 172.21.15.196:0/80281784 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c0caa40 con 0x7f055c0cb4e0 2024-04-28T13:32:11.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.246+0000 7f0561e00640 1 --2- 172.21.15.196:0/80281784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f055c14c360 0x7f055c0b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:11.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.246+0000 7f0560a00640 1 --2- 172.21.15.196:0/80281784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f055c0cb4e0 0x7f055c15f6d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0548001b40 tx=0x7f055c0d3390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:11.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.246+0000 7f0559600640 1 -- 172.21.15.196:0/80281784 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548007480 con 0x7f055c0cb4e0 2024-04-28T13:32:11.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.246+0000 7f0559600640 1 -- 172.21.15.196:0/80281784 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f054803f030 con 0x7f055c0cb4e0 2024-04-28T13:32:11.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.246+0000 7f0562800640 1 -- 172.21.15.196:0/80281784 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055c0ca350 con 0x7f055c0cb4e0 2024-04-28T13:32:11.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.246+0000 7f0559600640 1 -- 172.21.15.196:0/80281784 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548007820 con 0x7f055c0cb4e0 2024-04-28T13:32:11.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.246+0000 7f0562800640 1 -- 172.21.15.196:0/80281784 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055c0cafc0 con 0x7f055c0cb4e0 2024-04-28T13:32:11.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.247+0000 7f0562800640 1 -- 172.21.15.196:0/80281784 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f055c050730 con 0x7f055c0cb4e0 2024-04-28T13:32:11.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.249+0000 7f0559600640 1 -- 172.21.15.196:0/80281784 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f054803a440 con 0x7f055c0cb4e0 2024-04-28T13:32:11.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.249+0000 7f0559600640 1 --2- 172.21.15.196:0/80281784 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f053c06c920 0x7f053c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.249+0000 7f0559600640 1 -- 172.21.15.196:0/80281784 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f05480ac2f0 con 0x7f055c0cb4e0 2024-04-28T13:32:11.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.250+0000 7f0561400640 1 --2- 172.21.15.196:0/80281784 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f053c06c920 0x7f053c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:11.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.251+0000 7f0561400640 1 --2- 172.21.15.196:0/80281784 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f053c06c920 0x7f053c06ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f055c06aa70 tx=0x7f054c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:11.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.254+0000 7f0559600640 1 -- 172.21.15.196:0/80281784 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0548049070 con 0x7f055c0cb4e0 2024-04-28T13:32:11.453 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.453+0000 7f0562800640 1 -- 172.21.15.196:0/80281784 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f055c0d17d0 con 0x7f055c0cb4e0 2024-04-28T13:32:11.454 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.454+0000 7f0559600640 1 -- 172.21.15.196:0/80281784 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f055c0cfb80 con 0x7f055c0cb4e0 2024-04-28T13:32:11.454 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:32:11.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.457+0000 7f0562800640 1 -- 172.21.15.196:0/80281784 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f053c06c920 msgr2=0x7f053c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:11.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.457+0000 7f0562800640 1 --2- 172.21.15.196:0/80281784 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f053c06c920 0x7f053c06ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f055c06aa70 tx=0x7f054c009040 comp rx=0 tx=0).stop 2024-04-28T13:32:11.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.457+0000 7f0562800640 1 -- 172.21.15.196:0/80281784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f055c0cb4e0 msgr2=0x7f055c15f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:11.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.457+0000 7f0562800640 1 --2- 172.21.15.196:0/80281784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f055c0cb4e0 0x7f055c15f6d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0548001b40 tx=0x7f055c0d3390 comp rx=0 tx=0).stop 2024-04-28T13:32:11.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.457+0000 7f0562800640 1 -- 172.21.15.196:0/80281784 shutdown_connections 2024-04-28T13:32:11.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.457+0000 7f0562800640 1 --2- 172.21.15.196:0/80281784 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f053c06c920 0x7f053c06ede0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.457+0000 7f0562800640 1 --2- 172.21.15.196:0/80281784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f055c14c360 0x7f055c0b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.457+0000 7f0562800640 1 --2- 172.21.15.196:0/80281784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f055c0cb4e0 0x7f055c15f6d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.457+0000 7f0562800640 1 --2- 172.21.15.196:0/80281784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0540004800 0x7f055c15e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.457+0000 7f0562800640 1 -- 172.21.15.196:0/80281784 >> 172.21.15.196:0/80281784 conn(0x7f055c0d4fe0 msgr2=0x7f055c163970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:11.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.457+0000 7f0562800640 1 -- 172.21.15.196:0/80281784 shutdown_connections 2024-04-28T13:32:11.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.457+0000 7f0562800640 1 -- 172.21.15.196:0/80281784 wait complete. 2024-04-28T13:32:11.476 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:32:11.684 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.684+0000 7fdc22600640 1 Processor -- start 2024-04-28T13:32:11.684 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.684+0000 7fdc22600640 1 -- start start 2024-04-28T13:32:11.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.686+0000 7fdc22600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdc1c0d0300 0x7fdc1c0b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.686+0000 7fdc22600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c0b3e50 0x7fdc1c0720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.686+0000 7fdc22600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc1c0b46a0 con 0x7fdc1c06f340 2024-04-28T13:32:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.686+0000 7fdc22600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdc1c0b5db0 con 0x7fdc1c0d0300 2024-04-28T13:32:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.686+0000 7fdc22600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdc1c0b6090 con 0x7fdc1c0b3e50 2024-04-28T13:32:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.686+0000 7fdc1b400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fdc1c06f340 0x7fdc1c06f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43424/0 (socket says 172.21.15.196:43424) 2024-04-28T13:32:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.686+0000 7fdc1b400640 1 -- 172.21.15.196:0/2420031343 learned_addr learned my addr 172.21.15.196:0/2420031343 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:11.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.686+0000 7fdc1be00640 1 --2- 172.21.15.196:0/2420031343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c0b3e50 0x7fdc1c0720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:11.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.686+0000 7fdc1aa00640 1 --2- 172.21.15.196:0/2420031343 >> v2:172.21.15.203:3300/0 conn(0x7fdc1c0d0300 0x7fdc1c0b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:11.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.686+0000 7fdc1a000640 1 -- 172.21.15.196:0/2420031343 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2011205795 0 0) 0x7fdc1c0b46a0 con 0x7fdc1c06f340 2024-04-28T13:32:11.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.686+0000 7fdc1a000640 1 -- 172.21.15.196:0/2420031343 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc00003520 con 0x7fdc1c06f340 2024-04-28T13:32:11.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.687+0000 7fdc1a000640 1 -- 172.21.15.196:0/2420031343 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 888148865 0 0) 0x7fdc00003520 con 0x7fdc1c06f340 2024-04-28T13:32:11.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.687+0000 7fdc1a000640 1 -- 172.21.15.196:0/2420031343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c0b3e50 msgr2=0x7fdc1c0720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:11.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.687+0000 7fdc1a000640 1 --2- 172.21.15.196:0/2420031343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c0b3e50 0x7fdc1c0720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.687+0000 7fdc1a000640 1 -- 172.21.15.196:0/2420031343 >> v2:172.21.15.203:3300/0 conn(0x7fdc1c0d0300 msgr2=0x7fdc1c0b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:11.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.687+0000 7fdc1a000640 1 --2- 172.21.15.196:0/2420031343 >> v2:172.21.15.203:3300/0 conn(0x7fdc1c0d0300 0x7fdc1c0b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.687+0000 7fdc1a000640 1 -- 172.21.15.196:0/2420031343 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c0b6370 con 0x7fdc1c06f340 2024-04-28T13:32:11.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.687+0000 7fdc1a000640 1 -- 172.21.15.196:0/2420031343 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fdc0c001670 con 0x7fdc1c06f340 2024-04-28T13:32:11.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.687+0000 7fdc1a000640 1 -- 172.21.15.196:0/2420031343 >> v1:172.21.15.196:6789/0 conn(0x7fdc1c06f340 legacy=0x7fdc1c06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:11.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.687+0000 7fdc1a000640 1 --2- 172.21.15.196:0/2420031343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc000039b0 0x7fdc00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.687+0000 7fdc1a000640 1 --2- 172.21.15.196:0/2420031343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c0b3e50 0x7fdc00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.687+0000 7fdc1a000640 1 --2- 172.21.15.196:0/2420031343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc00004d40 0x7fdc000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.687+0000 7fdc1a000640 1 -- 172.21.15.196:0/2420031343 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c0b5db0 con 0x7fdc000039b0 2024-04-28T13:32:11.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.687+0000 7fdc1a000640 1 -- 172.21.15.196:0/2420031343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c0b6090 con 0x7fdc00004d40 2024-04-28T13:32:11.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.687+0000 7fdc1a000640 1 -- 172.21.15.196:0/2420031343 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdc0000bac0 con 0x7fdc1c0b3e50 2024-04-28T13:32:11.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.687+0000 7fdc1b400640 1 --2- 172.21.15.196:0/2420031343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc000039b0 0x7fdc00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:11.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.688+0000 7fdc1be00640 1 --2- 172.21.15.196:0/2420031343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc00004d40 0x7fdc000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:11.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.688+0000 7fdc1aa00640 1 --2- 172.21.15.196:0/2420031343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c0b3e50 0x7fdc00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:11.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.688+0000 7fdc1b400640 1 -- 172.21.15.196:0/2420031343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c0b3e50 msgr2=0x7fdc00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:11.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.688+0000 7fdc1b400640 1 --2- 172.21.15.196:0/2420031343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c0b3e50 0x7fdc00004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.688+0000 7fdc1b400640 1 -- 172.21.15.196:0/2420031343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc00004d40 msgr2=0x7fdc000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:11.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.688+0000 7fdc1b400640 1 --2- 172.21.15.196:0/2420031343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc00004d40 0x7fdc000051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.688+0000 7fdc1b400640 1 -- 172.21.15.196:0/2420031343 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc00003520 con 0x7fdc000039b0 2024-04-28T13:32:11.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.689+0000 7fdc1b400640 1 --2- 172.21.15.196:0/2420031343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc000039b0 0x7fdc00003dd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c007c40 tx=0x7fdc00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e871e3e97c00bc server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:11.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.689+0000 7fdc1a000640 1 -- 172.21.15.196:0/2420031343 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c007c80 con 0x7fdc000039b0 2024-04-28T13:32:11.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.689+0000 7fdc1a000640 1 -- 172.21.15.196:0/2420031343 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc0c036070 con 0x7fdc000039b0 2024-04-28T13:32:11.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.689+0000 7fdc1a000640 1 -- 172.21.15.196:0/2420031343 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c005d30 con 0x7fdc000039b0 2024-04-28T13:32:11.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.689+0000 7fdc22600640 1 -- 172.21.15.196:0/2420031343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc000039b0 msgr2=0x7fdc00003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:11.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.689+0000 7fdc22600640 1 --2- 172.21.15.196:0/2420031343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc000039b0 0x7fdc00003dd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c007c40 tx=0x7fdc00005830 comp rx=0 tx=0).stop 2024-04-28T13:32:11.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.689+0000 7fdc1be00640 1 -- 172.21.15.196:0/2420031343 reap_dead start 2024-04-28T13:32:11.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.690+0000 7fdc22600640 1 -- 172.21.15.196:0/2420031343 shutdown_connections 2024-04-28T13:32:11.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.690+0000 7fdc22600640 1 -- 172.21.15.196:0/2420031343 >> 172.21.15.196:0/2420031343 conn(0x7fdc1c0cd8a0 msgr2=0x7fdc1c0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:11.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.690+0000 7fdc22600640 1 -- 172.21.15.196:0/2420031343 shutdown_connections 2024-04-28T13:32:11.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.690+0000 7fdc22600640 1 -- 172.21.15.196:0/2420031343 wait complete. 2024-04-28T13:32:11.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.691+0000 7fdc22600640 1 Processor -- start 2024-04-28T13:32:11.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.691+0000 7fdc22600640 1 -- start start 2024-04-28T13:32:11.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.692+0000 7fdc22600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc000039b0 0x7fdc1c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.692+0000 7fdc22600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c1f5500 0x7fdc1c144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.692+0000 7fdc22600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc1c0ca1d0 0x7fdc1c13d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.692+0000 7fdc22600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c0b4460 con 0x7fdc1c0ca1d0 2024-04-28T13:32:11.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.692+0000 7fdc22600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c0b42e0 con 0x7fdc000039b0 2024-04-28T13:32:11.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.692+0000 7fdc22600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdc1c142a30 con 0x7fdc1c1f5500 2024-04-28T13:32:11.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.693+0000 7fdc1be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc1c0ca1d0 0x7fdc1c13d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:11.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.693+0000 7fdc1be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc1c0ca1d0 0x7fdc1c13d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34334/0 (socket says 172.21.15.196:34334) 2024-04-28T13:32:11.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.693+0000 7fdc1be00640 1 -- 172.21.15.196:0/249026584 learned_addr learned my addr 172.21.15.196:0/249026584 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:11.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.693+0000 7fdc1be00640 1 -- 172.21.15.196:0/249026584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c1f5500 msgr2=0x7fdc1c144de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:11.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.693+0000 7fdc1aa00640 1 --2- 172.21.15.196:0/249026584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c1f5500 0x7fdc1c144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:11.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.693+0000 7fdc1be00640 1 --2- 172.21.15.196:0/249026584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c1f5500 0x7fdc1c144de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.693+0000 7fdc1b400640 1 --2- 172.21.15.196:0/249026584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc000039b0 0x7fdc1c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:11.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.693+0000 7fdc1be00640 1 -- 172.21.15.196:0/249026584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc000039b0 msgr2=0x7fdc1c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:11.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.693+0000 7fdc1be00640 1 --2- 172.21.15.196:0/249026584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc000039b0 0x7fdc1c0cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.693+0000 7fdc1be00640 1 -- 172.21.15.196:0/249026584 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c1f5a10 con 0x7fdc1c0ca1d0 2024-04-28T13:32:11.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.693+0000 7fdc1be00640 1 --2- 172.21.15.196:0/249026584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc1c0ca1d0 0x7fdc1c13d500 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0b56a0 tx=0x7fdc1400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:11.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.694+0000 7fdc13400640 1 -- 172.21.15.196:0/249026584 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1401c070 con 0x7fdc1c0ca1d0 2024-04-28T13:32:11.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.694+0000 7fdc13400640 1 -- 172.21.15.196:0/249026584 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc14017400 con 0x7fdc1c0ca1d0 2024-04-28T13:32:11.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.694+0000 7fdc13400640 1 -- 172.21.15.196:0/249026584 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc140177f0 con 0x7fdc1c0ca1d0 2024-04-28T13:32:11.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.694+0000 7fdc22600640 1 -- 172.21.15.196:0/249026584 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1c1f5cf0 con 0x7fdc1c0ca1d0 2024-04-28T13:32:11.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.694+0000 7fdc22600640 1 -- 172.21.15.196:0/249026584 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc1c1f62b0 con 0x7fdc1c0ca1d0 2024-04-28T13:32:11.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.695+0000 7fdc22600640 1 -- 172.21.15.196:0/249026584 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc1c06a290 con 0x7fdc1c0ca1d0 2024-04-28T13:32:11.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.696+0000 7fdc13400640 1 -- 172.21.15.196:0/249026584 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc1401f070 con 0x7fdc1c0ca1d0 2024-04-28T13:32:11.697 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.697+0000 7fdc13400640 1 --2- 172.21.15.196:0/249026584 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdbfc06c8f0 0x7fdbfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:11.697 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.697+0000 7fdc13400640 1 -- 172.21.15.196:0/249026584 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fdc1408bb80 con 0x7fdc1c0ca1d0 2024-04-28T13:32:11.697 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.698+0000 7fdc1b400640 1 --2- 172.21.15.196:0/249026584 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdbfc06c8f0 0x7fdbfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:11.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.699+0000 7fdc1b400640 1 --2- 172.21.15.196:0/249026584 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdbfc06c8f0 0x7fdbfc06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c035000 tx=0x7fdc0c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:11.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.701+0000 7fdc13400640 1 -- 172.21.15.196:0/249026584 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc14012070 con 0x7fdc1c0ca1d0 2024-04-28T13:32:11.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.892+0000 7fdc22600640 1 -- 172.21.15.196:0/249026584 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdc1c15b800 con 0x7fdc1c0ca1d0 2024-04-28T13:32:11.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.893+0000 7fdc13400640 1 -- 172.21.15.196:0/249026584 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fdc14059f30 con 0x7fdc1c0ca1d0 2024-04-28T13:32:11.893 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:32:11.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.895+0000 7fdc22600640 1 -- 172.21.15.196:0/249026584 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdbfc06c8f0 msgr2=0x7fdbfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:11.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.895+0000 7fdc22600640 1 --2- 172.21.15.196:0/249026584 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdbfc06c8f0 0x7fdbfc06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c035000 tx=0x7fdc0c009030 comp rx=0 tx=0).stop 2024-04-28T13:32:11.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.895+0000 7fdc22600640 1 -- 172.21.15.196:0/249026584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc1c0ca1d0 msgr2=0x7fdc1c13d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:11.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.895+0000 7fdc22600640 1 --2- 172.21.15.196:0/249026584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc1c0ca1d0 0x7fdc1c13d500 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0b56a0 tx=0x7fdc1400f4a0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.895+0000 7fdc22600640 1 -- 172.21.15.196:0/249026584 shutdown_connections 2024-04-28T13:32:11.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.895+0000 7fdc22600640 1 --2- 172.21.15.196:0/249026584 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdbfc06c8f0 0x7fdbfc06edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.895+0000 7fdc22600640 1 --2- 172.21.15.196:0/249026584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc1c0ca1d0 0x7fdc1c13d500 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.895+0000 7fdc22600640 1 --2- 172.21.15.196:0/249026584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c1f5500 0x7fdc1c144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.895+0000 7fdc22600640 1 --2- 172.21.15.196:0/249026584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc000039b0 0x7fdc1c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:11.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.895+0000 7fdc22600640 1 -- 172.21.15.196:0/249026584 >> 172.21.15.196:0/249026584 conn(0x7fdc1c0cd8a0 msgr2=0x7fdc1c0b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:11.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.896+0000 7fdc22600640 1 -- 172.21.15.196:0/249026584 shutdown_connections 2024-04-28T13:32:11.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:11.896+0000 7fdc22600640 1 -- 172.21.15.196:0/249026584 wait complete. 2024-04-28T13:32:11.914 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T13:32:12.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.100+0000 7f2126e00640 1 Processor -- start 2024-04-28T13:32:12.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.100+0000 7f2126e00640 1 -- start start 2024-04-28T13:32:12.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.102+0000 7f2126e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f21200b5ee0 0x7f21200b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:12.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.102+0000 7f2126e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f212009ec80 0x7f21200d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:12.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.102+0000 7f2126e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f212015e380 con 0x7f21200add70 2024-04-28T13:32:12.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.102+0000 7f2126e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f21200b4b20 con 0x7f21200b5ee0 2024-04-28T13:32:12.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.102+0000 7f2126e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f21200b3f30 con 0x7f212009ec80 2024-04-28T13:32:12.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.102+0000 7f2125a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f21200b5ee0 0x7f21200b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:12.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.102+0000 7f2125a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f21200b5ee0 0x7f21200b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49432/0 (socket says 172.21.15.196:49432) 2024-04-28T13:32:12.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.102+0000 7f2125a00640 1 -- 172.21.15.196:0/4057882656 learned_addr learned my addr 172.21.15.196:0/4057882656 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:12.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.103+0000 7f2125000640 1 --2- 172.21.15.196:0/4057882656 >> v2:172.21.15.203:3300/0 conn(0x7f212009ec80 0x7f21200d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:12.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.103+0000 7f2125a00640 1 -- 172.21.15.196:0/4057882656 >> v2:172.21.15.203:3300/0 conn(0x7f212009ec80 msgr2=0x7f21200d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:12.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.103+0000 7f2125a00640 1 --2- 172.21.15.196:0/4057882656 >> v2:172.21.15.203:3300/0 conn(0x7f212009ec80 0x7f21200d1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.103+0000 7f2125a00640 1 -- 172.21.15.196:0/4057882656 >> v1:172.21.15.203:6789/0 conn(0x7f21200add70 legacy=0x7f212006df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:12.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.103+0000 7f2125a00640 1 -- 172.21.15.196:0/4057882656 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21200b41f0 con 0x7f21200b5ee0 2024-04-28T13:32:12.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.104+0000 7f2125a00640 1 --2- 172.21.15.196:0/4057882656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21200b5ee0 0x7f21200b4f40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f21200cd5d0 tx=0x7f211402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c483908e22abf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:12.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.104+0000 7f211fe00640 1 -- 172.21.15.196:0/4057882656 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114037070 con 0x7f21200b5ee0 2024-04-28T13:32:12.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.104+0000 7f211fe00640 1 -- 172.21.15.196:0/4057882656 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2114032440 con 0x7f21200b5ee0 2024-04-28T13:32:12.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.104+0000 7f211fe00640 1 -- 172.21.15.196:0/4057882656 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114032810 con 0x7f21200b5ee0 2024-04-28T13:32:12.105 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.105+0000 7f2126e00640 1 -- 172.21.15.196:0/4057882656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21200b5ee0 msgr2=0x7f21200b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:12.105 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.105+0000 7f2126e00640 1 --2- 172.21.15.196:0/4057882656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21200b5ee0 0x7f21200b4f40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f21200cd5d0 tx=0x7f211402a230 comp rx=0 tx=0).stop 2024-04-28T13:32:12.105 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.105+0000 7f2126e00640 1 -- 172.21.15.196:0/4057882656 shutdown_connections 2024-04-28T13:32:12.105 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.105+0000 7f2126e00640 1 --2- 172.21.15.196:0/4057882656 >> v2:172.21.15.203:3300/0 conn(0x7f212009ec80 0x7f21200d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.105 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.105+0000 7f2126e00640 1 --2- 172.21.15.196:0/4057882656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21200b5ee0 0x7f21200b4f40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.105 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.105+0000 7f2126e00640 1 -- 172.21.15.196:0/4057882656 >> 172.21.15.196:0/4057882656 conn(0x7f212006c6c0 msgr2=0x7f212006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:12.105 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.105+0000 7f2126e00640 1 -- 172.21.15.196:0/4057882656 shutdown_connections 2024-04-28T13:32:12.105 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.105+0000 7f2126e00640 1 -- 172.21.15.196:0/4057882656 wait complete. 2024-04-28T13:32:12.107 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.107+0000 7f2126e00640 1 Processor -- start 2024-04-28T13:32:12.107 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.107+0000 7f2126e00640 1 -- start start 2024-04-28T13:32:12.108 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.108+0000 7f2126e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f212009ec80 0x7f212014f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:12.108 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.108+0000 7f2126e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21200add70 0x7f212009d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:12.108 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.108+0000 7f2126e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21200b5ee0 0x7f212009ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:12.108 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.108+0000 7f2126e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f21200b3c20 con 0x7f21200b5ee0 2024-04-28T13:32:12.108 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.108+0000 7f2126e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f21200b47b0 con 0x7f212009ec80 2024-04-28T13:32:12.108 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.108+0000 7f2126e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f21200d1590 con 0x7f21200add70 2024-04-28T13:32:12.108 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.108+0000 7f2126400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21200b5ee0 0x7f212009ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:12.108 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.108+0000 7f2126400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21200b5ee0 0x7f212009ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49438/0 (socket says 172.21.15.196:49438) 2024-04-28T13:32:12.109 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.108+0000 7f2126400640 1 -- 172.21.15.196:0/869427309 learned_addr learned my addr 172.21.15.196:0/869427309 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:12.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.109+0000 7f2125a00640 1 --2- 172.21.15.196:0/869427309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f212009ec80 0x7f212014f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:12.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.109+0000 7f2126400640 1 -- 172.21.15.196:0/869427309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21200add70 msgr2=0x7f212009d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:12.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.109+0000 7f2126400640 1 --2- 172.21.15.196:0/869427309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21200add70 0x7f212009d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.109+0000 7f2126400640 1 -- 172.21.15.196:0/869427309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f212009ec80 msgr2=0x7f212014f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:12.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.109+0000 7f2126400640 1 --2- 172.21.15.196:0/869427309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f212009ec80 0x7f212014f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.109+0000 7f2126400640 1 -- 172.21.15.196:0/869427309 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21200ca0d0 con 0x7f21200b5ee0 2024-04-28T13:32:12.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.109+0000 7f2126400640 1 --2- 172.21.15.196:0/869427309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21200b5ee0 0x7f212009ddc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f211800c780 tx=0x7f211800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:12.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.109+0000 7f211d600640 1 -- 172.21.15.196:0/869427309 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118017070 con 0x7f21200b5ee0 2024-04-28T13:32:12.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.109+0000 7f211d600640 1 -- 172.21.15.196:0/869427309 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2118003da0 con 0x7f21200b5ee0 2024-04-28T13:32:12.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.109+0000 7f211d600640 1 -- 172.21.15.196:0/869427309 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21180052b0 con 0x7f21200b5ee0 2024-04-28T13:32:12.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.109+0000 7f2126e00640 1 -- 172.21.15.196:0/869427309 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21200ca3b0 con 0x7f21200b5ee0 2024-04-28T13:32:12.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.109+0000 7f2126e00640 1 -- 172.21.15.196:0/869427309 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21200ca970 con 0x7f21200b5ee0 2024-04-28T13:32:12.111 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.110+0000 7f2126e00640 1 -- 172.21.15.196:0/869427309 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f212006a290 con 0x7f21200b5ee0 2024-04-28T13:32:12.111 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.111+0000 7f211d600640 1 -- 172.21.15.196:0/869427309 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2118005450 con 0x7f21200b5ee0 2024-04-28T13:32:12.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.113+0000 7f211d600640 1 --2- 172.21.15.196:0/869427309 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f210406c8f0 0x7f210406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:12.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.113+0000 7f211d600640 1 -- 172.21.15.196:0/869427309 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f211808b1d0 con 0x7f21200b5ee0 2024-04-28T13:32:12.114 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.114+0000 7f2125a00640 1 --2- 172.21.15.196:0/869427309 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f210406c8f0 0x7f210406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:12.115 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.115+0000 7f2125a00640 1 --2- 172.21.15.196:0/869427309 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f210406c8f0 0x7f210406edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2114034000 tx=0x7f211402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:12.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.118+0000 7f211d600640 1 -- 172.21.15.196:0/869427309 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2118059570 con 0x7f21200b5ee0 2024-04-28T13:32:12.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.321+0000 7f2126e00640 1 -- 172.21.15.196:0/869427309 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f212006ee60 con 0x7f21200b5ee0 2024-04-28T13:32:12.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.322+0000 7f211d600640 1 -- 172.21.15.196:0/869427309 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f2118058e70 con 0x7f21200b5ee0 2024-04-28T13:32:12.322 INFO:teuthology.orchestra.run.smithi196.stderr:listed 0 entries 2024-04-28T13:32:12.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.325+0000 7f2126e00640 1 -- 172.21.15.196:0/869427309 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f210406c8f0 msgr2=0x7f210406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:12.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.325+0000 7f2126e00640 1 --2- 172.21.15.196:0/869427309 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f210406c8f0 0x7f210406edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2114034000 tx=0x7f211402e000 comp rx=0 tx=0).stop 2024-04-28T13:32:12.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.325+0000 7f2126e00640 1 -- 172.21.15.196:0/869427309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21200b5ee0 msgr2=0x7f212009ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:12.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.325+0000 7f2126e00640 1 --2- 172.21.15.196:0/869427309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21200b5ee0 0x7f212009ddc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f211800c780 tx=0x7f211800cc40 comp rx=0 tx=0).stop 2024-04-28T13:32:12.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.325+0000 7f2126e00640 1 -- 172.21.15.196:0/869427309 shutdown_connections 2024-04-28T13:32:12.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.325+0000 7f2126e00640 1 --2- 172.21.15.196:0/869427309 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f210406c8f0 0x7f210406edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.325+0000 7f2126e00640 1 --2- 172.21.15.196:0/869427309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21200b5ee0 0x7f212009ddc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.325+0000 7f2126e00640 1 --2- 172.21.15.196:0/869427309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21200add70 0x7f212009d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.325+0000 7f2126e00640 1 --2- 172.21.15.196:0/869427309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f212009ec80 0x7f212014f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.325+0000 7f2126e00640 1 -- 172.21.15.196:0/869427309 >> 172.21.15.196:0/869427309 conn(0x7f212006c6c0 msgr2=0x7f212015e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:12.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.325+0000 7f2126e00640 1 -- 172.21.15.196:0/869427309 shutdown_connections 2024-04-28T13:32:12.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.326+0000 7f2126e00640 1 -- 172.21.15.196:0/869427309 wait complete. 2024-04-28T13:32:12.344 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-28T13:32:12.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:12.344 DEBUG:teuthology.orchestra.run.smithi196:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-28T13:32:12.429 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:32:12.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:12.430 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-28T13:32:12.476 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:32:12.476 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:12.533 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:32:12.533 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:12.585 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:32:12.585 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:12.638 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:32:12.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.843+0000 7f53b2a00640 1 Processor -- start 2024-04-28T13:32:12.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.843+0000 7f53b2a00640 1 -- start start 2024-04-28T13:32:12.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.844+0000 7f53b2a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53ac145870 0x7f53ac06f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:12.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.844+0000 7f53b2a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f53ac06f980 0x7f53ac072d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.844+0000 7f53b2a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f53ac073250 0x7f53ac15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.844+0000 7f53b2a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f53ac06fde0 con 0x7f53ac06f980 2024-04-28T13:32:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.844+0000 7f53b2a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f53ac073a40 con 0x7f53ac073250 2024-04-28T13:32:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.844+0000 7f53b2a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f53ac15d8d0 con 0x7f53ac145870 2024-04-28T13:32:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.845+0000 7f53ab400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f53ac06f980 0x7f53ac072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.845+0000 7f53ab400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f53ac06f980 0x7f53ac072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49448/0 (socket says 172.21.15.196:49448) 2024-04-28T13:32:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.845+0000 7f53ab400640 1 -- 172.21.15.196:0/3703590120 learned_addr learned my addr 172.21.15.196:0/3703590120 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.845+0000 7f53ab400640 1 -- 172.21.15.196:0/3703590120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53ac145870 msgr2=0x7f53ac06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:12.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.845+0000 7f53ab400640 1 --2- 172.21.15.196:0/3703590120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53ac145870 0x7f53ac06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.845+0000 7f53ab400640 1 -- 172.21.15.196:0/3703590120 >> v2:172.21.15.203:3300/0 conn(0x7f53ac073250 msgr2=0x7f53ac15fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:12.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.845+0000 7f53ab400640 1 --2- 172.21.15.196:0/3703590120 >> v2:172.21.15.203:3300/0 conn(0x7f53ac073250 0x7f53ac15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.845+0000 7f53ab400640 1 -- 172.21.15.196:0/3703590120 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ac15dbb0 con 0x7f53ac06f980 2024-04-28T13:32:12.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.846+0000 7f53ab400640 1 --2- 172.21.15.196:0/3703590120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac06f980 0x7f53ac072d10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0cf140 tx=0x7f539802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55bea9855a04ebe server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:12.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.847+0000 7f53aaa00640 1 -- 172.21.15.196:0/3703590120 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398037070 con 0x7f53ac06f980 2024-04-28T13:32:12.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.847+0000 7f53aaa00640 1 -- 172.21.15.196:0/3703590120 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5398032440 con 0x7f53ac06f980 2024-04-28T13:32:12.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.847+0000 7f53aaa00640 1 -- 172.21.15.196:0/3703590120 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398032810 con 0x7f53ac06f980 2024-04-28T13:32:12.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.847+0000 7f53b2a00640 1 -- 172.21.15.196:0/3703590120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac06f980 msgr2=0x7f53ac072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:12.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.847+0000 7f53b2a00640 1 --2- 172.21.15.196:0/3703590120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac06f980 0x7f53ac072d10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0cf140 tx=0x7f539802a1e0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.847+0000 7f53b2a00640 1 -- 172.21.15.196:0/3703590120 shutdown_connections 2024-04-28T13:32:12.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.847+0000 7f53b2a00640 1 --2- 172.21.15.196:0/3703590120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac06f980 0x7f53ac072d10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.847+0000 7f53b2a00640 1 --2- 172.21.15.196:0/3703590120 >> v2:172.21.15.203:3300/0 conn(0x7f53ac073250 0x7f53ac15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.847+0000 7f53b2a00640 1 --2- 172.21.15.196:0/3703590120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53ac145870 0x7f53ac06f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.847+0000 7f53b2a00640 1 -- 172.21.15.196:0/3703590120 >> 172.21.15.196:0/3703590120 conn(0x7f53ac0cd830 msgr2=0x7f53ac0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:12.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.847+0000 7f53b2a00640 1 -- 172.21.15.196:0/3703590120 shutdown_connections 2024-04-28T13:32:12.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.847+0000 7f53b2a00640 1 -- 172.21.15.196:0/3703590120 wait complete. 2024-04-28T13:32:12.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.849+0000 7f53b2a00640 1 Processor -- start 2024-04-28T13:32:12.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.849+0000 7f53b2a00640 1 -- start start 2024-04-28T13:32:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.850+0000 7f53b2a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53ac06f980 0x7f53ac13b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.850+0000 7f53b2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53ac073250 0x7f53ac0b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.850+0000 7f53b2a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac145870 0x7f53ac0b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.850+0000 7f53b2a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac145c50 con 0x7f53ac145870 2024-04-28T13:32:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.850+0000 7f53b2a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac15d5d0 con 0x7f53ac073250 2024-04-28T13:32:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.850+0000 7f53b2a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f53ac0736e0 con 0x7f53ac06f980 2024-04-28T13:32:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.850+0000 7f53b0a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac145870 0x7f53ac0b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.850+0000 7f53b0a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac145870 0x7f53ac0b28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49464/0 (socket says 172.21.15.196:49464) 2024-04-28T13:32:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.850+0000 7f53b0a00640 1 -- 172.21.15.196:0/2609468577 learned_addr learned my addr 172.21.15.196:0/2609468577 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.851+0000 7f53b0a00640 1 -- 172.21.15.196:0/2609468577 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53ac06f980 msgr2=0x7f53ac13b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.851+0000 7f53b0a00640 1 --2- 172.21.15.196:0/2609468577 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53ac06f980 0x7f53ac13b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.851+0000 7f53b0a00640 1 -- 172.21.15.196:0/2609468577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53ac073250 msgr2=0x7f53ac0b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.851+0000 7f53ab400640 1 --2- 172.21.15.196:0/2609468577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53ac073250 0x7f53ac0b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.851+0000 7f53b0a00640 1 --2- 172.21.15.196:0/2609468577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53ac073250 0x7f53ac0b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.851+0000 7f53b0a00640 1 -- 172.21.15.196:0/2609468577 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ac13bcb0 con 0x7f53ac145870 2024-04-28T13:32:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.851+0000 7f53b0a00640 1 --2- 172.21.15.196:0/2609468577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac145870 0x7f53ac0b28e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f53a400b9e0 tx=0x7f53a400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.851+0000 7f53a3e00640 1 -- 172.21.15.196:0/2609468577 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4019070 con 0x7f53ac145870 2024-04-28T13:32:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.851+0000 7f53a3e00640 1 -- 172.21.15.196:0/2609468577 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53a400cac0 con 0x7f53ac145870 2024-04-28T13:32:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.851+0000 7f53b2a00640 1 -- 172.21.15.196:0/2609468577 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ac13b780 con 0x7f53ac145870 2024-04-28T13:32:12.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.851+0000 7f53a3e00640 1 -- 172.21.15.196:0/2609468577 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4014440 con 0x7f53ac145870 2024-04-28T13:32:12.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.852+0000 7f53b2a00640 1 -- 172.21.15.196:0/2609468577 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ac0d0490 con 0x7f53ac145870 2024-04-28T13:32:12.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.853+0000 7f53b2a00640 1 -- 172.21.15.196:0/2609468577 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53ac06a290 con 0x7f53ac145870 2024-04-28T13:32:12.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.854+0000 7f53a3e00640 1 -- 172.21.15.196:0/2609468577 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53a401c070 con 0x7f53ac145870 2024-04-28T13:32:12.854 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.854+0000 7f53a3e00640 1 --2- 172.21.15.196:0/2609468577 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f538c06c920 0x7f538c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:12.854 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.855+0000 7f53a3e00640 1 -- 172.21.15.196:0/2609468577 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f53a408b930 con 0x7f53ac145870 2024-04-28T13:32:12.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.855+0000 7f53abe00640 1 --2- 172.21.15.196:0/2609468577 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f538c06c920 0x7f538c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:12.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.856+0000 7f53abe00640 1 --2- 172.21.15.196:0/2609468577 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f538c06c920 0x7f538c06ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0b2ef0 tx=0x7f539c006c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:12.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:12.859+0000 7f53a3e00640 1 -- 172.21.15.196:0/2609468577 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f53ac13bcb0 con 0x7f53ac145870 2024-04-28T13:32:13.060 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.060+0000 7f53b2a00640 1 -- 172.21.15.196:0/2609468577 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53ac0b3520 con 0x7f53ac145870 2024-04-28T13:32:13.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.064+0000 7f53a3e00640 1 -- 172.21.15.196:0/2609468577 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16890 (secure 0 0 0) 0x7f53a4059c40 con 0x7f53ac145870 2024-04-28T13:32:13.064 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:13.064 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:03.882452+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:32:13.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.066+0000 7f53b2a00640 1 -- 172.21.15.196:0/2609468577 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f538c06c920 msgr2=0x7f538c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.066+0000 7f53b2a00640 1 --2- 172.21.15.196:0/2609468577 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f538c06c920 0x7f538c06ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0b2ef0 tx=0x7f539c006c90 comp rx=0 tx=0).stop 2024-04-28T13:32:13.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.066+0000 7f53b2a00640 1 -- 172.21.15.196:0/2609468577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac145870 msgr2=0x7f53ac0b28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.066+0000 7f53b2a00640 1 --2- 172.21.15.196:0/2609468577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac145870 0x7f53ac0b28e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f53a400b9e0 tx=0x7f53a400bea0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.067+0000 7f53b2a00640 1 -- 172.21.15.196:0/2609468577 shutdown_connections 2024-04-28T13:32:13.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.067+0000 7f53b2a00640 1 --2- 172.21.15.196:0/2609468577 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f538c06c920 0x7f538c06ede0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.067+0000 7f53b2a00640 1 --2- 172.21.15.196:0/2609468577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53ac145870 0x7f53ac0b28e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.067+0000 7f53b2a00640 1 --2- 172.21.15.196:0/2609468577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53ac073250 0x7f53ac0b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.067+0000 7f53b2a00640 1 --2- 172.21.15.196:0/2609468577 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53ac06f980 0x7f53ac13b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.067+0000 7f53b2a00640 1 -- 172.21.15.196:0/2609468577 >> 172.21.15.196:0/2609468577 conn(0x7f53ac0cd830 msgr2=0x7f53ac0b5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:13.068 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.067+0000 7f53b2a00640 1 -- 172.21.15.196:0/2609468577 shutdown_connections 2024-04-28T13:32:13.068 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.067+0000 7f53b2a00640 1 -- 172.21.15.196:0/2609468577 wait complete. 2024-04-28T13:32:13.086 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:31:19.280975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:31:20.037066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:32:13.087 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:32:13.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.296+0000 7fa6dba00640 1 Processor -- start 2024-04-28T13:32:13.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.296+0000 7fa6dba00640 1 -- start start 2024-04-28T13:32:13.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.297+0000 7fa6dba00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d40d3820 0x7fa6d40b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.298+0000 7fa6dba00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6d40d3f50 con 0x7fa6d415f680 2024-04-28T13:32:13.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.298+0000 7fa6dba00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6d40b5090 con 0x7fa6d40b60b0 2024-04-28T13:32:13.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.298+0000 7fa6dba00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa6d415fa60 con 0x7fa6d40d3820 2024-04-28T13:32:13.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.298+0000 7fa6d9000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa6d415f680 0x7fa6d4163aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:44308/0 (socket says 172.21.15.196:44308) 2024-04-28T13:32:13.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.298+0000 7fa6d9000640 1 -- 172.21.15.196:0/964015775 learned_addr learned my addr 172.21.15.196:0/964015775 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:13.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.298+0000 7fa6d3e00640 1 --2- 172.21.15.196:0/964015775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d40d3820 0x7fa6d40b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:13.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.298+0000 7fa6d3400640 1 -- 172.21.15.196:0/964015775 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3111773758 0 0) 0x7fa6d40d3f50 con 0x7fa6d415f680 2024-04-28T13:32:13.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.298+0000 7fa6d3400640 1 -- 172.21.15.196:0/964015775 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6b8003520 con 0x7fa6d415f680 2024-04-28T13:32:13.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.299+0000 7fa6d3400640 1 -- 172.21.15.196:0/964015775 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1225759370 0 0) 0x7fa6b8003520 con 0x7fa6d415f680 2024-04-28T13:32:13.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.299+0000 7fa6d3400640 1 -- 172.21.15.196:0/964015775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d40d3820 msgr2=0x7fa6d40b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:13.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.299+0000 7fa6d3400640 1 --2- 172.21.15.196:0/964015775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d40d3820 0x7fa6d40b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.299+0000 7fa6d3400640 1 -- 172.21.15.196:0/964015775 >> v1:172.21.15.203:6789/0 conn(0x7fa6d40b60b0 legacy=0x7fa6d40d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.299+0000 7fa6d3400640 1 -- 172.21.15.196:0/964015775 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d40b5370 con 0x7fa6d415f680 2024-04-28T13:32:13.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.299+0000 7fa6d3400640 1 -- 172.21.15.196:0/964015775 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fa6c4001670 con 0x7fa6d415f680 2024-04-28T13:32:13.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.299+0000 7fa6d3400640 1 -- 172.21.15.196:0/964015775 >> v1:172.21.15.196:6789/0 conn(0x7fa6d415f680 legacy=0x7fa6d4163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.299+0000 7fa6d3400640 1 --2- 172.21.15.196:0/964015775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b80039e0 0x7fa6b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.299+0000 7fa6d3400640 1 --2- 172.21.15.196:0/964015775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b8004800 0x7fa6b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.299+0000 7fa6d3400640 1 --2- 172.21.15.196:0/964015775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d40d3820 0x7fa6b8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.299+0000 7fa6d3400640 1 -- 172.21.15.196:0/964015775 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d415fa60 con 0x7fa6b80039e0 2024-04-28T13:32:13.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.299+0000 7fa6d3400640 1 -- 172.21.15.196:0/964015775 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b800baf0 con 0x7fa6b8004800 2024-04-28T13:32:13.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.299+0000 7fa6d3400640 1 -- 172.21.15.196:0/964015775 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b800bcf0 con 0x7fa6d40d3820 2024-04-28T13:32:13.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.299+0000 7fa6d9000640 1 --2- 172.21.15.196:0/964015775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b80039e0 0x7fa6b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:13.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.299+0000 7fa6d3e00640 1 --2- 172.21.15.196:0/964015775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b8004800 0x7fa6b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:13.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.300+0000 7fa6d9a00640 1 --2- 172.21.15.196:0/964015775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d40d3820 0x7fa6b8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:13.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.300+0000 7fa6d9000640 1 -- 172.21.15.196:0/964015775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d40d3820 msgr2=0x7fa6b8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:13.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.300+0000 7fa6d9000640 1 --2- 172.21.15.196:0/964015775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d40d3820 0x7fa6b8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.300+0000 7fa6d9000640 1 -- 172.21.15.196:0/964015775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b8004800 msgr2=0x7fa6b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:13.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.300+0000 7fa6d9000640 1 --2- 172.21.15.196:0/964015775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b8004800 0x7fa6b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.300+0000 7fa6d9000640 1 -- 172.21.15.196:0/964015775 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b8003520 con 0x7fa6b80039e0 2024-04-28T13:32:13.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.300+0000 7fa6d9a00640 1 --2- 172.21.15.196:0/964015775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d40d3820 0x7fa6b8005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:13.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.301+0000 7fa6d9000640 1 --2- 172.21.15.196:0/964015775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b80039e0 0x7fa6b8003e00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c210 tx=0x7fa6b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4b83a8a63760d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:13.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.301+0000 7fa6d3400640 1 -- 172.21.15.196:0/964015775 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4007d30 con 0x7fa6b80039e0 2024-04-28T13:32:13.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.301+0000 7fa6d3400640 1 -- 172.21.15.196:0/964015775 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c4005ad0 con 0x7fa6b80039e0 2024-04-28T13:32:13.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.301+0000 7fa6d3400640 1 -- 172.21.15.196:0/964015775 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4007070 con 0x7fa6b80039e0 2024-04-28T13:32:13.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.302+0000 7fa6dba00640 1 -- 172.21.15.196:0/964015775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b80039e0 msgr2=0x7fa6b8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.302+0000 7fa6dba00640 1 --2- 172.21.15.196:0/964015775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b80039e0 0x7fa6b8003e00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c210 tx=0x7fa6b8005830 comp rx=0 tx=0).stop 2024-04-28T13:32:13.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.302+0000 7fa6d9a00640 1 -- 172.21.15.196:0/964015775 reap_dead start 2024-04-28T13:32:13.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.302+0000 7fa6dba00640 1 -- 172.21.15.196:0/964015775 shutdown_connections 2024-04-28T13:32:13.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.302+0000 7fa6dba00640 1 -- 172.21.15.196:0/964015775 >> 172.21.15.196:0/964015775 conn(0x7fa6d408eca0 msgr2=0x7fa6d409e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:13.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.302+0000 7fa6dba00640 1 -- 172.21.15.196:0/964015775 shutdown_connections 2024-04-28T13:32:13.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.302+0000 7fa6dba00640 1 -- 172.21.15.196:0/964015775 wait complete. 2024-04-28T13:32:13.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.304+0000 7fa6dba00640 1 Processor -- start 2024-04-28T13:32:13.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.304+0000 7fa6dba00640 1 -- start start 2024-04-28T13:32:13.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6dba00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b80039e0 0x7fa6d415b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6dba00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d4161ff0 0x7fa6d4162470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6dba00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d415fcc0 0x7fa6d4160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6dba00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d40d57b0 con 0x7fa6d4161ff0 2024-04-28T13:32:13.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6dba00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d40d5630 con 0x7fa6b80039e0 2024-04-28T13:32:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6dba00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa6d40d5930 con 0x7fa6d415fcc0 2024-04-28T13:32:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6d3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d4161ff0 0x7fa6d4162470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6d3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d4161ff0 0x7fa6d4162470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49490/0 (socket says 172.21.15.196:49490) 2024-04-28T13:32:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6d3e00640 1 -- 172.21.15.196:0/2057957448 learned_addr learned my addr 172.21.15.196:0/2057957448 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6d9000640 1 --2- 172.21.15.196:0/2057957448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b80039e0 0x7fa6d415b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6d9a00640 1 --2- 172.21.15.196:0/2057957448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d415fcc0 0x7fa6d4160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6d3e00640 1 -- 172.21.15.196:0/2057957448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d415fcc0 msgr2=0x7fa6d4160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6d3e00640 1 --2- 172.21.15.196:0/2057957448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d415fcc0 0x7fa6d4160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6d3e00640 1 -- 172.21.15.196:0/2057957448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b80039e0 msgr2=0x7fa6d415b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6d3e00640 1 --2- 172.21.15.196:0/2057957448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b80039e0 0x7fa6d415b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.305+0000 7fa6d3e00640 1 -- 172.21.15.196:0/2057957448 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d4160a10 con 0x7fa6d4161ff0 2024-04-28T13:32:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.306+0000 7fa6d3e00640 1 --2- 172.21.15.196:0/2057957448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d4161ff0 0x7fa6d4162470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00f580 tx=0x7fa6bc00fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.306+0000 7fa6d0c00640 1 -- 172.21.15.196:0/2057957448 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc019020 con 0x7fa6d4161ff0 2024-04-28T13:32:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.306+0000 7fa6d0c00640 1 -- 172.21.15.196:0/2057957448 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6bc014870 con 0x7fa6d4161ff0 2024-04-28T13:32:13.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.306+0000 7fa6dba00640 1 -- 172.21.15.196:0/2057957448 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d4160cf0 con 0x7fa6d4161ff0 2024-04-28T13:32:13.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.306+0000 7fa6d0c00640 1 -- 172.21.15.196:0/2057957448 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc014c60 con 0x7fa6d4161ff0 2024-04-28T13:32:13.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.306+0000 7fa6dba00640 1 -- 172.21.15.196:0/2057957448 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d41612b0 con 0x7fa6d4161ff0 2024-04-28T13:32:13.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.307+0000 7fa6dba00640 1 -- 172.21.15.196:0/2057957448 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d406a290 con 0x7fa6d4161ff0 2024-04-28T13:32:13.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.308+0000 7fa6d0c00640 1 -- 172.21.15.196:0/2057957448 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6bc01c020 con 0x7fa6d4161ff0 2024-04-28T13:32:13.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.309+0000 7fa6d0c00640 1 --2- 172.21.15.196:0/2057957448 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa6b006c8f0 0x7fa6b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.309+0000 7fa6d0c00640 1 -- 172.21.15.196:0/2057957448 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa6bc08c350 con 0x7fa6d4161ff0 2024-04-28T13:32:13.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.310+0000 7fa6d9000640 1 --2- 172.21.15.196:0/2057957448 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa6b006c8f0 0x7fa6b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:13.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.311+0000 7fa6d9000640 1 --2- 172.21.15.196:0/2057957448 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa6b006c8f0 0x7fa6b006edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4033000 tx=0x7fa6c402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:13.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.314+0000 7fa6d0c00640 1 -- 172.21.15.196:0/2057957448 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa6bc023020 con 0x7fa6d4161ff0 2024-04-28T13:32:13.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.512+0000 7fa6dba00640 1 -- 172.21.15.196:0/2057957448 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa6d40d4f70 con 0x7fa6d4161ff0 2024-04-28T13:32:13.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.512+0000 7fa6d0c00640 1 -- 172.21.15.196:0/2057957448 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fa6bc026350 con 0x7fa6d4161ff0 2024-04-28T13:32:13.512 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:32:13.515 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.515+0000 7fa6dba00640 1 -- 172.21.15.196:0/2057957448 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa6b006c8f0 msgr2=0x7fa6b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.515 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.515+0000 7fa6dba00640 1 --2- 172.21.15.196:0/2057957448 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa6b006c8f0 0x7fa6b006edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4033000 tx=0x7fa6c402e000 comp rx=0 tx=0).stop 2024-04-28T13:32:13.515 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.515+0000 7fa6dba00640 1 -- 172.21.15.196:0/2057957448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d4161ff0 msgr2=0x7fa6d4162470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.515 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.515+0000 7fa6dba00640 1 --2- 172.21.15.196:0/2057957448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d4161ff0 0x7fa6d4162470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00f580 tx=0x7fa6bc00fa40 comp rx=0 tx=0).stop 2024-04-28T13:32:13.516 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.515+0000 7fa6dba00640 1 -- 172.21.15.196:0/2057957448 shutdown_connections 2024-04-28T13:32:13.516 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.515+0000 7fa6dba00640 1 --2- 172.21.15.196:0/2057957448 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa6b006c8f0 0x7fa6b006edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.516 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.515+0000 7fa6dba00640 1 --2- 172.21.15.196:0/2057957448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d415fcc0 0x7fa6d4160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.516 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.515+0000 7fa6dba00640 1 --2- 172.21.15.196:0/2057957448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d4161ff0 0x7fa6d4162470 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.516 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.515+0000 7fa6dba00640 1 --2- 172.21.15.196:0/2057957448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b80039e0 0x7fa6d415b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.516 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.515+0000 7fa6dba00640 1 -- 172.21.15.196:0/2057957448 >> 172.21.15.196:0/2057957448 conn(0x7fa6d408eca0 msgr2=0x7fa6d40b3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:13.516 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.515+0000 7fa6dba00640 1 -- 172.21.15.196:0/2057957448 shutdown_connections 2024-04-28T13:32:13.516 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.515+0000 7fa6dba00640 1 -- 172.21.15.196:0/2057957448 wait complete. 2024-04-28T13:32:13.535 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:32:13.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.737+0000 7f558e200640 1 Processor -- start 2024-04-28T13:32:13.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.737+0000 7f558e200640 1 -- start start 2024-04-28T13:32:13.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.738+0000 7f558e200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880d2070 0x7f55880baab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.738+0000 7f558e200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55880d4240 con 0x7f558815f490 2024-04-28T13:32:13.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.738+0000 7f558e200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55880d31a0 con 0x7f55880d1be0 2024-04-28T13:32:13.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.738+0000 7f558e200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f55880d3480 con 0x7f55880d2070 2024-04-28T13:32:13.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.739+0000 7f5587400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f558815f490 0x7f55880d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:44314/0 (socket says 172.21.15.196:44314) 2024-04-28T13:32:13.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.739+0000 7f5587400640 1 -- 172.21.15.196:0/4050867368 learned_addr learned my addr 172.21.15.196:0/4050867368 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:13.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.739+0000 7f5587e00640 1 --2- 172.21.15.196:0/4050867368 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880d2070 0x7f55880baab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:13.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.739+0000 7f5586000640 1 -- 172.21.15.196:0/4050867368 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3011627131 0 0) 0x7f55880d4240 con 0x7f558815f490 2024-04-28T13:32:13.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.739+0000 7f5586000640 1 -- 172.21.15.196:0/4050867368 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5568003520 con 0x7f558815f490 2024-04-28T13:32:13.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.739+0000 7f5586000640 1 -- 172.21.15.196:0/4050867368 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1267909489 0 0) 0x7f5568003520 con 0x7f558815f490 2024-04-28T13:32:13.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5586000640 1 -- 172.21.15.196:0/4050867368 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880d2070 msgr2=0x7f55880baab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:13.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5586000640 1 --2- 172.21.15.196:0/4050867368 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880d2070 0x7f55880baab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5586000640 1 -- 172.21.15.196:0/4050867368 >> v1:172.21.15.203:6789/0 conn(0x7f55880d1be0 legacy=0x7f55880ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5586000640 1 -- 172.21.15.196:0/4050867368 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55880d3760 con 0x7f558815f490 2024-04-28T13:32:13.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5586000640 1 -- 172.21.15.196:0/4050867368 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f5578001670 con 0x7f558815f490 2024-04-28T13:32:13.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5586000640 1 -- 172.21.15.196:0/4050867368 >> v1:172.21.15.196:6789/0 conn(0x7f558815f490 legacy=0x7f55880d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5586000640 1 --2- 172.21.15.196:0/4050867368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55680039e0 0x7f5568003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5586000640 1 --2- 172.21.15.196:0/4050867368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5568004800 0x7f5568004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5586000640 1 --2- 172.21.15.196:0/4050867368 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880d2070 0x7f55680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5586000640 1 -- 172.21.15.196:0/4050867368 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55880d3480 con 0x7f55680039e0 2024-04-28T13:32:13.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5586000640 1 -- 172.21.15.196:0/4050867368 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f556800ba70 con 0x7f5568004800 2024-04-28T13:32:13.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5586000640 1 -- 172.21.15.196:0/4050867368 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f556800bc70 con 0x7f55880d2070 2024-04-28T13:32:13.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5587400640 1 --2- 172.21.15.196:0/4050867368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55680039e0 0x7f5568003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:13.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5586a00640 1 --2- 172.21.15.196:0/4050867368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5568004800 0x7f5568004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:13.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5587400640 1 -- 172.21.15.196:0/4050867368 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880d2070 msgr2=0x7f55680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:13.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5587400640 1 --2- 172.21.15.196:0/4050867368 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880d2070 0x7f55680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5587400640 1 -- 172.21.15.196:0/4050867368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5568004800 msgr2=0x7f5568004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:13.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5587400640 1 --2- 172.21.15.196:0/4050867368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5568004800 0x7f5568004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5587400640 1 -- 172.21.15.196:0/4050867368 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5568003520 con 0x7f55680039e0 2024-04-28T13:32:13.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.740+0000 7f5586a00640 1 --2- 172.21.15.196:0/4050867368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5568004800 0x7f5568004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:13.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.741+0000 7f5587400640 1 --2- 172.21.15.196:0/4050867368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55680039e0 0x7f5568003e00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5578001eb0 tx=0x7f5568005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9286540717d640dd server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:13.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.742+0000 7f5586000640 1 -- 172.21.15.196:0/4050867368 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578007d30 con 0x7f55680039e0 2024-04-28T13:32:13.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.742+0000 7f5586000640 1 -- 172.21.15.196:0/4050867368 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5578005af0 con 0x7f55680039e0 2024-04-28T13:32:13.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.742+0000 7f5586000640 1 -- 172.21.15.196:0/4050867368 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578007070 con 0x7f55680039e0 2024-04-28T13:32:13.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.742+0000 7f558e200640 1 -- 172.21.15.196:0/4050867368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55680039e0 msgr2=0x7f5568003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.742+0000 7f558e200640 1 --2- 172.21.15.196:0/4050867368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55680039e0 0x7f5568003e00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5578001eb0 tx=0x7f5568005830 comp rx=0 tx=0).stop 2024-04-28T13:32:13.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.742+0000 7f5587e00640 1 -- 172.21.15.196:0/4050867368 reap_dead start 2024-04-28T13:32:13.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.742+0000 7f558e200640 1 -- 172.21.15.196:0/4050867368 shutdown_connections 2024-04-28T13:32:13.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.742+0000 7f558e200640 1 -- 172.21.15.196:0/4050867368 >> 172.21.15.196:0/4050867368 conn(0x7f558809e530 msgr2=0x7f55880d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:13.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.743+0000 7f558e200640 1 -- 172.21.15.196:0/4050867368 shutdown_connections 2024-04-28T13:32:13.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.743+0000 7f558e200640 1 -- 172.21.15.196:0/4050867368 wait complete. 2024-04-28T13:32:13.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.744+0000 7f558e200640 1 Processor -- start 2024-04-28T13:32:13.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.744+0000 7f558e200640 1 -- start start 2024-04-28T13:32:13.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.745+0000 7f558e200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55680039e0 0x7f5588151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.745+0000 7f558e200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f558815e220 0x7f55880b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.745+0000 7f558e200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b4160 0x7f5588161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.745+0000 7f558e200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55880d24a0 con 0x7f55680039e0 2024-04-28T13:32:13.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.745+0000 7f558e200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f55880d3d20 con 0x7f558815e220 2024-04-28T13:32:13.746 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.745+0000 7f558e200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f55880d11c0 con 0x7f55880b4160 2024-04-28T13:32:13.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.746+0000 7f5587400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55680039e0 0x7f5588151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:13.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.746+0000 7f5587400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55680039e0 0x7f5588151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49516/0 (socket says 172.21.15.196:49516) 2024-04-28T13:32:13.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.746+0000 7f5587400640 1 -- 172.21.15.196:0/1550791690 learned_addr learned my addr 172.21.15.196:0/1550791690 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:13.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.746+0000 7f5587e00640 1 --2- 172.21.15.196:0/1550791690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b4160 0x7f5588161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:13.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.746+0000 7f5586a00640 1 --2- 172.21.15.196:0/1550791690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f558815e220 0x7f55880b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:13.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.746+0000 7f5587400640 1 -- 172.21.15.196:0/1550791690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b4160 msgr2=0x7f5588161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.746+0000 7f5587400640 1 --2- 172.21.15.196:0/1550791690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b4160 0x7f5588161fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.746+0000 7f5587400640 1 -- 172.21.15.196:0/1550791690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f558815e220 msgr2=0x7f55880b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.746+0000 7f5587400640 1 --2- 172.21.15.196:0/1550791690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f558815e220 0x7f55880b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.746+0000 7f5587400640 1 -- 172.21.15.196:0/1550791690 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5588152390 con 0x7f55680039e0 2024-04-28T13:32:13.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.746+0000 7f5587e00640 1 --2- 172.21.15.196:0/1550791690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b4160 0x7f5588161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:13.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.746+0000 7f5587400640 1 --2- 172.21.15.196:0/1550791690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55680039e0 0x7f5588151e50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f557800c300 tx=0x7f5578001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:13.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.747+0000 7f557f400640 1 -- 172.21.15.196:0/1550791690 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578041030 con 0x7f55680039e0 2024-04-28T13:32:13.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.747+0000 7f557f400640 1 -- 172.21.15.196:0/1550791690 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5578038040 con 0x7f55680039e0 2024-04-28T13:32:13.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.747+0000 7f557f400640 1 -- 172.21.15.196:0/1550791690 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578001630 con 0x7f55680039e0 2024-04-28T13:32:13.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.747+0000 7f558e200640 1 -- 172.21.15.196:0/1550791690 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558814f6e0 con 0x7f55680039e0 2024-04-28T13:32:13.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.747+0000 7f558e200640 1 -- 172.21.15.196:0/1550791690 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558814fca0 con 0x7f55680039e0 2024-04-28T13:32:13.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.748+0000 7f558e200640 1 -- 172.21.15.196:0/1550791690 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558806a290 con 0x7f55680039e0 2024-04-28T13:32:13.749 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.749+0000 7f557f400640 1 -- 172.21.15.196:0/1550791690 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5578044070 con 0x7f55680039e0 2024-04-28T13:32:13.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.750+0000 7f557f400640 1 --2- 172.21.15.196:0/1550791690 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f556406c8f0 0x7f556406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:13.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.751+0000 7f557f400640 1 -- 172.21.15.196:0/1550791690 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f55780abc00 con 0x7f55680039e0 2024-04-28T13:32:13.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.751+0000 7f5586a00640 1 --2- 172.21.15.196:0/1550791690 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f556406c8f0 0x7f556406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:13.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.752+0000 7f5586a00640 1 --2- 172.21.15.196:0/1550791690 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f556406c8f0 0x7f556406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f55880c8f90 tx=0x7f5570009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:13.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.755+0000 7f557f400640 1 -- 172.21.15.196:0/1550791690 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5588152390 con 0x7f55680039e0 2024-04-28T13:32:13.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.958+0000 7f558e200640 1 -- 172.21.15.196:0/1550791690 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f558806cc10 con 0x7f55680039e0 2024-04-28T13:32:13.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.960+0000 7f557f400640 1 -- 172.21.15.196:0/1550791690 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f557803c380 con 0x7f55680039e0 2024-04-28T13:32:13.960 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:32:13.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.963+0000 7f558e200640 1 -- 172.21.15.196:0/1550791690 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f556406c8f0 msgr2=0x7f556406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.963+0000 7f558e200640 1 --2- 172.21.15.196:0/1550791690 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f556406c8f0 0x7f556406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f55880c8f90 tx=0x7f5570009040 comp rx=0 tx=0).stop 2024-04-28T13:32:13.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.963+0000 7f558e200640 1 -- 172.21.15.196:0/1550791690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55680039e0 msgr2=0x7f5588151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:13.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.963+0000 7f558e200640 1 --2- 172.21.15.196:0/1550791690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55680039e0 0x7f5588151e50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f557800c300 tx=0x7f5578001a40 comp rx=0 tx=0).stop 2024-04-28T13:32:13.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.963+0000 7f558e200640 1 -- 172.21.15.196:0/1550791690 shutdown_connections 2024-04-28T13:32:13.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.963+0000 7f558e200640 1 --2- 172.21.15.196:0/1550791690 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f556406c8f0 0x7f556406edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.963+0000 7f558e200640 1 --2- 172.21.15.196:0/1550791690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b4160 0x7f5588161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.963+0000 7f558e200640 1 --2- 172.21.15.196:0/1550791690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f558815e220 0x7f55880b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.963+0000 7f558e200640 1 --2- 172.21.15.196:0/1550791690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55680039e0 0x7f5588151e50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:13.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.963+0000 7f558e200640 1 -- 172.21.15.196:0/1550791690 >> 172.21.15.196:0/1550791690 conn(0x7f558809e530 msgr2=0x7f55880b9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:13.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.963+0000 7f558e200640 1 -- 172.21.15.196:0/1550791690 shutdown_connections 2024-04-28T13:32:13.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:13.963+0000 7f558e200640 1 -- 172.21.15.196:0/1550791690 wait complete. 2024-04-28T13:32:13.982 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:32:14.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.185+0000 7f518a600640 1 Processor -- start 2024-04-28T13:32:14.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.185+0000 7f518a600640 1 -- start start 2024-04-28T13:32:14.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.186+0000 7f518a600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51841628c0 0x7f51840d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:14.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.186+0000 7f518a600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f51840b93e0 0x7f51840d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:14.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.186+0000 7f518a600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51840d0710 con 0x7f51840d1ef0 2024-04-28T13:32:14.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.186+0000 7f518a600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f51840b7f40 con 0x7f51840b93e0 2024-04-28T13:32:14.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.186+0000 7f518a600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f51840d09f0 con 0x7f51841628c0 2024-04-28T13:32:14.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.187+0000 7f5183e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f51840d1ef0 0x7f51840d2350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:44324/0 (socket says 172.21.15.196:44324) 2024-04-28T13:32:14.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.187+0000 7f5183e00640 1 -- 172.21.15.196:0/1365601845 learned_addr learned my addr 172.21.15.196:0/1365601845 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:14.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.187+0000 7f5189200640 1 --2- 172.21.15.196:0/1365601845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51841628c0 0x7f51840d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:14.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.187+0000 7f5189c00640 1 --2- 172.21.15.196:0/1365601845 >> v2:172.21.15.203:3300/0 conn(0x7f51840b93e0 0x7f51840d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:14.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.187+0000 7f5183400640 1 -- 172.21.15.196:0/1365601845 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2096765888 0 0) 0x7f51840d0710 con 0x7f51840d1ef0 2024-04-28T13:32:14.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.187+0000 7f5183400640 1 -- 172.21.15.196:0/1365601845 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f516c003520 con 0x7f51840d1ef0 2024-04-28T13:32:14.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5183400640 1 -- 172.21.15.196:0/1365601845 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1399694433 0 0) 0x7f516c003520 con 0x7f51840d1ef0 2024-04-28T13:32:14.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5183400640 1 -- 172.21.15.196:0/1365601845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51841628c0 msgr2=0x7f51840d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:14.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5183400640 1 --2- 172.21.15.196:0/1365601845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51841628c0 0x7f51840d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5183400640 1 -- 172.21.15.196:0/1365601845 >> v2:172.21.15.203:3300/0 conn(0x7f51840b93e0 msgr2=0x7f51840d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:14.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5183400640 1 --2- 172.21.15.196:0/1365601845 >> v2:172.21.15.203:3300/0 conn(0x7f51840b93e0 0x7f51840d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5183400640 1 -- 172.21.15.196:0/1365601845 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51840d0cd0 con 0x7f51840d1ef0 2024-04-28T13:32:14.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5183400640 1 -- 172.21.15.196:0/1365601845 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f5170001670 con 0x7f51840d1ef0 2024-04-28T13:32:14.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5189c00640 1 --2- 172.21.15.196:0/1365601845 >> v2:172.21.15.203:3300/0 conn(0x7f51840b93e0 0x7f51840d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:32:14.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5183400640 1 -- 172.21.15.196:0/1365601845 >> v1:172.21.15.196:6789/0 conn(0x7f51840d1ef0 legacy=0x7f51840d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:14.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5183400640 1 --2- 172.21.15.196:0/1365601845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f516c0039b0 0x7f516c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:14.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5183400640 1 --2- 172.21.15.196:0/1365601845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f516c004800 0x7f516c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:14.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5183400640 1 --2- 172.21.15.196:0/1365601845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51841628c0 0x7f516c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:14.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5183400640 1 -- 172.21.15.196:0/1365601845 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51840b7f40 con 0x7f516c0039b0 2024-04-28T13:32:14.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5183400640 1 -- 172.21.15.196:0/1365601845 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f51840d09f0 con 0x7f516c004800 2024-04-28T13:32:14.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5183400640 1 -- 172.21.15.196:0/1365601845 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f516c00ba10 con 0x7f51841628c0 2024-04-28T13:32:14.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5189200640 1 --2- 172.21.15.196:0/1365601845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f516c0039b0 0x7f516c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:14.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5189200640 1 -- 172.21.15.196:0/1365601845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51841628c0 msgr2=0x7f516c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:14.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5189200640 1 --2- 172.21.15.196:0/1365601845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51841628c0 0x7f516c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5189200640 1 -- 172.21.15.196:0/1365601845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f516c004800 msgr2=0x7f516c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:14.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.188+0000 7f5189200640 1 --2- 172.21.15.196:0/1365601845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f516c004800 0x7f516c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.189+0000 7f5189200640 1 -- 172.21.15.196:0/1365601845 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516c003520 con 0x7f516c0039b0 2024-04-28T13:32:14.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.190+0000 7f5189200640 1 --2- 172.21.15.196:0/1365601845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f516c0039b0 0x7f516c003dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5178009ac0 tx=0x7f516c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fa22093bbc7e9e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:14.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.190+0000 7f5183400640 1 -- 172.21.15.196:0/1365601845 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178036070 con 0x7f516c0039b0 2024-04-28T13:32:14.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.190+0000 7f5183400640 1 -- 172.21.15.196:0/1365601845 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5178031400 con 0x7f516c0039b0 2024-04-28T13:32:14.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.190+0000 7f5183400640 1 -- 172.21.15.196:0/1365601845 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178009690 con 0x7f516c0039b0 2024-04-28T13:32:14.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.190+0000 7f518a600640 1 -- 172.21.15.196:0/1365601845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f516c0039b0 msgr2=0x7f516c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:14.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.190+0000 7f518a600640 1 --2- 172.21.15.196:0/1365601845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f516c0039b0 0x7f516c003dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5178009ac0 tx=0x7f516c005830 comp rx=0 tx=0).stop 2024-04-28T13:32:14.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.190+0000 7f5189c00640 1 -- 172.21.15.196:0/1365601845 reap_dead start 2024-04-28T13:32:14.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.191+0000 7f518a600640 1 -- 172.21.15.196:0/1365601845 shutdown_connections 2024-04-28T13:32:14.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.191+0000 7f518a600640 1 -- 172.21.15.196:0/1365601845 >> 172.21.15.196:0/1365601845 conn(0x7f51840d4fc0 msgr2=0x7f51840d53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:14.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.191+0000 7f518a600640 1 -- 172.21.15.196:0/1365601845 shutdown_connections 2024-04-28T13:32:14.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.191+0000 7f518a600640 1 -- 172.21.15.196:0/1365601845 wait complete. 2024-04-28T13:32:14.193 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.193+0000 7f518a600640 1 Processor -- start 2024-04-28T13:32:14.193 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.193+0000 7f518a600640 1 -- start start 2024-04-28T13:32:14.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.194+0000 7f518a600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f516c0039b0 0x7f51840caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:14.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.194+0000 7f518a600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51840c8650 0x7f51840c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:14.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.194+0000 7f518a600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51840cdc50 0x7f51840ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:14.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.194+0000 7f518a600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5184162e20 con 0x7f51840cdc50 2024-04-28T13:32:14.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.194+0000 7f518a600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5184162ca0 con 0x7f51840c8650 2024-04-28T13:32:14.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.194+0000 7f518a600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f51840d19b0 con 0x7f516c0039b0 2024-04-28T13:32:14.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.194+0000 7f5189c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51840cdc50 0x7f51840ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:14.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.194+0000 7f5189c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51840cdc50 0x7f51840ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49532/0 (socket says 172.21.15.196:49532) 2024-04-28T13:32:14.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.194+0000 7f5189c00640 1 -- 172.21.15.196:0/1701063829 learned_addr learned my addr 172.21.15.196:0/1701063829 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:14.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.195+0000 7f5183e00640 1 --2- 172.21.15.196:0/1701063829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51840c8650 0x7f51840c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:14.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.195+0000 7f5189200640 1 --2- 172.21.15.196:0/1701063829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f516c0039b0 0x7f51840caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:14.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.195+0000 7f5189c00640 1 -- 172.21.15.196:0/1701063829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f516c0039b0 msgr2=0x7f51840caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:14.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.195+0000 7f5189c00640 1 --2- 172.21.15.196:0/1701063829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f516c0039b0 0x7f51840caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.195+0000 7f5189c00640 1 -- 172.21.15.196:0/1701063829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51840c8650 msgr2=0x7f51840c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:14.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.195+0000 7f5189c00640 1 --2- 172.21.15.196:0/1701063829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51840c8650 0x7f51840c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.195+0000 7f5189c00640 1 -- 172.21.15.196:0/1701063829 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51840cd360 con 0x7f51840cdc50 2024-04-28T13:32:14.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.195+0000 7f5189c00640 1 --2- 172.21.15.196:0/1701063829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51840cdc50 0x7f51840ce0d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f51840b9870 tx=0x7f517c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:14.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.195+0000 7f5180c00640 1 -- 172.21.15.196:0/1701063829 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c01a070 con 0x7f51840cdc50 2024-04-28T13:32:14.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.195+0000 7f5180c00640 1 -- 172.21.15.196:0/1701063829 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f517c00dcd0 con 0x7f51840cdc50 2024-04-28T13:32:14.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.195+0000 7f5180c00640 1 -- 172.21.15.196:0/1701063829 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c015640 con 0x7f51840cdc50 2024-04-28T13:32:14.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.196+0000 7f518a600640 1 -- 172.21.15.196:0/1701063829 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51840cb480 con 0x7f51840cdc50 2024-04-28T13:32:14.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.196+0000 7f518a600640 1 -- 172.21.15.196:0/1701063829 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51840c94a0 con 0x7f51840cdc50 2024-04-28T13:32:14.198 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.197+0000 7f518a600640 1 -- 172.21.15.196:0/1701063829 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5184050730 con 0x7f51840cdc50 2024-04-28T13:32:14.198 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.198+0000 7f5180c00640 1 -- 172.21.15.196:0/1701063829 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f517c01d070 con 0x7f51840cdc50 2024-04-28T13:32:14.198 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.199+0000 7f5180c00640 1 --2- 172.21.15.196:0/1701063829 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f516406c920 0x7f516406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:14.198 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.199+0000 7f5180c00640 1 -- 172.21.15.196:0/1701063829 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f517c08ba10 con 0x7f51840cdc50 2024-04-28T13:32:14.199 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.199+0000 7f5189200640 1 --2- 172.21.15.196:0/1701063829 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f516406c920 0x7f516406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:14.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.201+0000 7f5189200640 1 --2- 172.21.15.196:0/1701063829 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f516406c920 0x7f516406ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5178033000 tx=0x7f5178008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:14.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.203+0000 7f5180c00640 1 -- 172.21.15.196:0/1701063829 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f517c010070 con 0x7f51840cdc50 2024-04-28T13:32:14.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.402+0000 7f518a600640 1 -- 172.21.15.196:0/1701063829 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f51840b60a0 con 0x7f51840cdc50 2024-04-28T13:32:14.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.403+0000 7f5180c00640 1 -- 172.21.15.196:0/1701063829 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f517c059c90 con 0x7f51840cdc50 2024-04-28T13:32:14.403 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:32:14.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.406+0000 7f518a600640 1 -- 172.21.15.196:0/1701063829 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f516406c920 msgr2=0x7f516406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:14.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.406+0000 7f518a600640 1 --2- 172.21.15.196:0/1701063829 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f516406c920 0x7f516406ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5178033000 tx=0x7f5178008030 comp rx=0 tx=0).stop 2024-04-28T13:32:14.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.406+0000 7f518a600640 1 -- 172.21.15.196:0/1701063829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51840cdc50 msgr2=0x7f51840ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:14.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.406+0000 7f518a600640 1 --2- 172.21.15.196:0/1701063829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51840cdc50 0x7f51840ce0d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f51840b9870 tx=0x7f517c00d4a0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.406+0000 7f518a600640 1 -- 172.21.15.196:0/1701063829 shutdown_connections 2024-04-28T13:32:14.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.406+0000 7f518a600640 1 --2- 172.21.15.196:0/1701063829 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f516406c920 0x7f516406ede0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.406+0000 7f518a600640 1 --2- 172.21.15.196:0/1701063829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51840cdc50 0x7f51840ce0d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.406+0000 7f518a600640 1 --2- 172.21.15.196:0/1701063829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51840c8650 0x7f51840c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.406+0000 7f518a600640 1 --2- 172.21.15.196:0/1701063829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f516c0039b0 0x7f51840caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.406+0000 7f518a600640 1 -- 172.21.15.196:0/1701063829 >> 172.21.15.196:0/1701063829 conn(0x7f51840d4fc0 msgr2=0x7f51840d2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:14.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.406+0000 7f518a600640 1 -- 172.21.15.196:0/1701063829 shutdown_connections 2024-04-28T13:32:14.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.406+0000 7f518a600640 1 -- 172.21.15.196:0/1701063829 wait complete. 2024-04-28T13:32:14.423 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T13:32:14.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.613+0000 7febb3e00640 1 Processor -- start 2024-04-28T13:32:14.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.613+0000 7febb3e00640 1 -- start start 2024-04-28T13:32:14.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.615+0000 7febb3e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7febb40d1760 0x7febb40b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:14.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.615+0000 7febb3e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7febb409ec80 0x7febb406df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:14.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.615+0000 7febb3e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febb415e380 con 0x7febb40b5ee0 2024-04-28T13:32:14.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.615+0000 7febb3e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7febb40b4b20 con 0x7febb409ec80 2024-04-28T13:32:14.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.615+0000 7febb3e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7febb40b3f30 con 0x7febb40d1760 2024-04-28T13:32:14.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.615+0000 7febb3400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7febb409ec80 0x7febb406df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:14.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.615+0000 7febb2a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7febb40b5ee0 0x7febb40d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:44332/0 (socket says 172.21.15.196:44332) 2024-04-28T13:32:14.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.615+0000 7febb2a00640 1 -- 172.21.15.196:0/3185379122 learned_addr learned my addr 172.21.15.196:0/3185379122 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:14.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.615+0000 7febb3400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7febb409ec80 0x7febb406df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49546/0 (socket says 172.21.15.196:49546) 2024-04-28T13:32:14.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.616+0000 7febabe00640 1 --2- 172.21.15.196:0/3185379122 >> v2:172.21.15.203:3300/0 conn(0x7febb40d1760 0x7febb40b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:14.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.616+0000 7febb3400640 1 -- 172.21.15.196:0/3185379122 >> v2:172.21.15.203:3300/0 conn(0x7febb40d1760 msgr2=0x7febb40b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:14.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.616+0000 7febb3400640 1 --2- 172.21.15.196:0/3185379122 >> v2:172.21.15.203:3300/0 conn(0x7febb40d1760 0x7febb40b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.616+0000 7febb3400640 1 -- 172.21.15.196:0/3185379122 >> v1:172.21.15.196:6789/0 conn(0x7febb40b5ee0 legacy=0x7febb40d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:14.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.616+0000 7febb2000640 1 -- 172.21.15.196:0/3185379122 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3638180239 0 0) 0x7febb415e380 con 0x7febb40b5ee0 2024-04-28T13:32:14.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.616+0000 7febb3400640 1 -- 172.21.15.196:0/3185379122 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb40b41f0 con 0x7febb409ec80 2024-04-28T13:32:14.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.617+0000 7febb3400640 1 --2- 172.21.15.196:0/3185379122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febb409ec80 0x7febb406df50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7febb40cd5d0 tx=0x7febac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681736c5f7beda40 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:14.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.618+0000 7febb2000640 1 -- 172.21.15.196:0/3185379122 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac007740 con 0x7febb409ec80 2024-04-28T13:32:14.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.618+0000 7febb2000640 1 -- 172.21.15.196:0/3185379122 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febac0078e0 con 0x7febb409ec80 2024-04-28T13:32:14.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.618+0000 7febb2000640 1 -- 172.21.15.196:0/3185379122 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac007cb0 con 0x7febb409ec80 2024-04-28T13:32:14.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.618+0000 7febb3e00640 1 -- 172.21.15.196:0/3185379122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febb409ec80 msgr2=0x7febb406df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:14.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.618+0000 7febb3e00640 1 --2- 172.21.15.196:0/3185379122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febb409ec80 0x7febb406df50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7febb40cd5d0 tx=0x7febac02a680 comp rx=0 tx=0).stop 2024-04-28T13:32:14.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.618+0000 7febb3e00640 1 -- 172.21.15.196:0/3185379122 shutdown_connections 2024-04-28T13:32:14.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.618+0000 7febb3e00640 1 --2- 172.21.15.196:0/3185379122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febb409ec80 0x7febb406df50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.618+0000 7febb3e00640 1 --2- 172.21.15.196:0/3185379122 >> v2:172.21.15.203:3300/0 conn(0x7febb40d1760 0x7febb40b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.618+0000 7febb3e00640 1 -- 172.21.15.196:0/3185379122 >> 172.21.15.196:0/3185379122 conn(0x7febb406c6c0 msgr2=0x7febb406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:14.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.618+0000 7febb3e00640 1 -- 172.21.15.196:0/3185379122 shutdown_connections 2024-04-28T13:32:14.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.618+0000 7febb3e00640 1 -- 172.21.15.196:0/3185379122 wait complete. 2024-04-28T13:32:14.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.620+0000 7febb3e00640 1 Processor -- start 2024-04-28T13:32:14.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.620+0000 7febb3e00640 1 -- start start 2024-04-28T13:32:14.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.621+0000 7febb3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb409ec80 0x7febb40b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:14.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.621+0000 7febb3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febb40b5ee0 0x7febb4150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:14.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.621+0000 7febb3e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb40d1760 0x7febb4151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:14.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.621+0000 7febb3e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7febb40b3c20 con 0x7febb40b5ee0 2024-04-28T13:32:14.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.621+0000 7febb3e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7febb40b47b0 con 0x7febb409ec80 2024-04-28T13:32:14.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.621+0000 7febb3e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7febb40add70 con 0x7febb40d1760 2024-04-28T13:32:14.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.621+0000 7febabe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febb40b5ee0 0x7febb4150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:14.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.621+0000 7febabe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febb40b5ee0 0x7febb4150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49550/0 (socket says 172.21.15.196:49550) 2024-04-28T13:32:14.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.621+0000 7febabe00640 1 -- 172.21.15.196:0/3131164783 learned_addr learned my addr 172.21.15.196:0/3131164783 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:14.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.622+0000 7febb2a00640 1 --2- 172.21.15.196:0/3131164783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb409ec80 0x7febb40b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:14.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.622+0000 7febb3400640 1 --2- 172.21.15.196:0/3131164783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb40d1760 0x7febb4151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:14.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.622+0000 7febabe00640 1 -- 172.21.15.196:0/3131164783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb40d1760 msgr2=0x7febb4151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:14.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.622+0000 7febabe00640 1 --2- 172.21.15.196:0/3131164783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb40d1760 0x7febb4151210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.622+0000 7febabe00640 1 -- 172.21.15.196:0/3131164783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb409ec80 msgr2=0x7febb40b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:14.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.622+0000 7febabe00640 1 --2- 172.21.15.196:0/3131164783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb409ec80 0x7febb40b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.622+0000 7febabe00640 1 -- 172.21.15.196:0/3131164783 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb40cd1f0 con 0x7febb40b5ee0 2024-04-28T13:32:14.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.622+0000 7febabe00640 1 --2- 172.21.15.196:0/3131164783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febb40b5ee0 0x7febb4150cd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00d620 tx=0x7feb9c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:14.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.623+0000 7febaaa00640 1 -- 172.21.15.196:0/3131164783 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00b620 con 0x7febb40b5ee0 2024-04-28T13:32:14.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.623+0000 7febaaa00640 1 -- 172.21.15.196:0/3131164783 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9c019070 con 0x7febb40b5ee0 2024-04-28T13:32:14.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.623+0000 7febaaa00640 1 -- 172.21.15.196:0/3131164783 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00ba00 con 0x7febb40b5ee0 2024-04-28T13:32:14.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.623+0000 7febb3e00640 1 -- 172.21.15.196:0/3131164783 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb40cd4d0 con 0x7febb40b5ee0 2024-04-28T13:32:14.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.623+0000 7febb3e00640 1 -- 172.21.15.196:0/3131164783 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb40cda90 con 0x7febb40b5ee0 2024-04-28T13:32:14.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.624+0000 7febb3e00640 1 -- 172.21.15.196:0/3131164783 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb406a290 con 0x7febb40b5ee0 2024-04-28T13:32:14.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.625+0000 7febaaa00640 1 -- 172.21.15.196:0/3131164783 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb9c00bba0 con 0x7febb40b5ee0 2024-04-28T13:32:14.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.626+0000 7febaaa00640 1 --2- 172.21.15.196:0/3131164783 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feb9006c920 0x7feb9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:14.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.626+0000 7febaaa00640 1 -- 172.21.15.196:0/3131164783 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7feb9c08c0c0 con 0x7febb40b5ee0 2024-04-28T13:32:14.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.627+0000 7febb2a00640 1 --2- 172.21.15.196:0/3131164783 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feb9006c920 0x7feb9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:14.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.628+0000 7febb2a00640 1 --2- 172.21.15.196:0/3131164783 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feb9006c920 0x7feb9006ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feba4000f60 tx=0x7feba4007b90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:14.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.630+0000 7febaaa00640 1 -- 172.21.15.196:0/3131164783 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb9c05a340 con 0x7febb40b5ee0 2024-04-28T13:32:14.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.845+0000 7febb3e00640 1 -- 172.21.15.196:0/3131164783 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7febb409d940 con 0x7febb40b5ee0 2024-04-28T13:32:14.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.846+0000 7febaaa00640 1 -- 172.21.15.196:0/3131164783 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7feb9c059c40 con 0x7febb40b5ee0 2024-04-28T13:32:14.846 INFO:teuthology.orchestra.run.smithi196.stderr:listed 0 entries 2024-04-28T13:32:14.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.848+0000 7febb3e00640 1 -- 172.21.15.196:0/3131164783 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feb9006c920 msgr2=0x7feb9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:14.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.848+0000 7febb3e00640 1 --2- 172.21.15.196:0/3131164783 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feb9006c920 0x7feb9006ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feba4000f60 tx=0x7feba4007b90 comp rx=0 tx=0).stop 2024-04-28T13:32:14.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.848+0000 7febb3e00640 1 -- 172.21.15.196:0/3131164783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febb40b5ee0 msgr2=0x7febb4150cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:14.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.848+0000 7febb3e00640 1 --2- 172.21.15.196:0/3131164783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febb40b5ee0 0x7febb4150cd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00d620 tx=0x7feb9c00dae0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.848+0000 7febb3e00640 1 -- 172.21.15.196:0/3131164783 shutdown_connections 2024-04-28T13:32:14.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.848+0000 7febb3e00640 1 --2- 172.21.15.196:0/3131164783 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feb9006c920 0x7feb9006ede0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.848+0000 7febb3e00640 1 --2- 172.21.15.196:0/3131164783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb40d1760 0x7febb4151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.848+0000 7febb3e00640 1 --2- 172.21.15.196:0/3131164783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febb40b5ee0 0x7febb4150cd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.848+0000 7febb3e00640 1 --2- 172.21.15.196:0/3131164783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb409ec80 0x7febb40b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:14.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.848+0000 7febb3e00640 1 -- 172.21.15.196:0/3131164783 >> 172.21.15.196:0/3131164783 conn(0x7febb406c6c0 msgr2=0x7febb409c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:14.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.849+0000 7febb3e00640 1 -- 172.21.15.196:0/3131164783 shutdown_connections 2024-04-28T13:32:14.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:14.849+0000 7febb3e00640 1 -- 172.21.15.196:0/3131164783 wait complete. 2024-04-28T13:32:14.868 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-28T13:32:14.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:14.868 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-28T13:32:14.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:32:14.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:14.966 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-28T13:32:15.014 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-28T13:32:15.014 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:32:15.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.223+0000 7f6731e00640 1 Processor -- start 2024-04-28T13:32:15.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.223+0000 7f6731e00640 1 -- start start 2024-04-28T13:32:15.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.225+0000 7f6731e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f672c0b5ee0 0x7f672c0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:15.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.225+0000 7f6731e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c09ec80 0x7f672c0d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:15.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.225+0000 7f6731e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f672c15e380 con 0x7f672c0add70 2024-04-28T13:32:15.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.225+0000 7f6731e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f672c0b4b20 con 0x7f672c0b5ee0 2024-04-28T13:32:15.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.225+0000 7f6731e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f672c0b3f30 con 0x7f672c09ec80 2024-04-28T13:32:15.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.226+0000 7f6730a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f672c0b5ee0 0x7f672c0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:15.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.226+0000 7f6723e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c09ec80 0x7f672c0d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:15.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.226+0000 7f6731400640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f672c0add70 0x7f672c06df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.196:59816/0 (socket says 172.21.15.196:59816) 2024-04-28T13:32:15.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.226+0000 7f6731400640 1 -- 172.21.15.196:0/2530915622 learned_addr learned my addr 172.21.15.196:0/2530915622 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:15.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.226+0000 7f672be00640 1 -- 172.21.15.196:0/2530915622 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3687504328 0 0) 0x7f672c15e380 con 0x7f672c0add70 2024-04-28T13:32:15.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.226+0000 7f672be00640 1 -- 172.21.15.196:0/2530915622 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6710003520 con 0x7f672c0add70 2024-04-28T13:32:15.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.227+0000 7f6730a00640 1 -- 172.21.15.196:0/2530915622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c09ec80 msgr2=0x7f672c0d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:15.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.227+0000 7f6730a00640 1 --2- 172.21.15.196:0/2530915622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c09ec80 0x7f672c0d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.227+0000 7f6730a00640 1 -- 172.21.15.196:0/2530915622 >> v1:172.21.15.203:6789/0 conn(0x7f672c0add70 legacy=0x7f672c06df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:15.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.227+0000 7f6730a00640 1 -- 172.21.15.196:0/2530915622 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672c0b41f0 con 0x7f672c0b5ee0 2024-04-28T13:32:15.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.227+0000 7f6723e00640 1 --2- 172.21.15.196:0/2530915622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c09ec80 0x7f672c0d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:32:15.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.228+0000 7f6730a00640 1 --2- 172.21.15.196:0/2530915622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f672c0b5ee0 0x7f672c0b4f40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f671c0097a0 tx=0x7f671c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dc794e4dce2eb74 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:15.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.229+0000 7f672be00640 1 -- 172.21.15.196:0/2530915622 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c037070 con 0x7f672c0b5ee0 2024-04-28T13:32:15.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.229+0000 7f672be00640 1 -- 172.21.15.196:0/2530915622 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671c032440 con 0x7f672c0b5ee0 2024-04-28T13:32:15.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.229+0000 7f672be00640 1 -- 172.21.15.196:0/2530915622 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c032810 con 0x7f672c0b5ee0 2024-04-28T13:32:15.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.229+0000 7f6731e00640 1 -- 172.21.15.196:0/2530915622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f672c0b5ee0 msgr2=0x7f672c0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:15.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.229+0000 7f6731e00640 1 --2- 172.21.15.196:0/2530915622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f672c0b5ee0 0x7f672c0b4f40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f671c0097a0 tx=0x7f671c02a180 comp rx=0 tx=0).stop 2024-04-28T13:32:15.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.230+0000 7f6731e00640 1 -- 172.21.15.196:0/2530915622 shutdown_connections 2024-04-28T13:32:15.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.230+0000 7f6731e00640 1 --2- 172.21.15.196:0/2530915622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c09ec80 0x7f672c0d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.230+0000 7f6731e00640 1 --2- 172.21.15.196:0/2530915622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f672c0b5ee0 0x7f672c0b4f40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.230+0000 7f6731e00640 1 -- 172.21.15.196:0/2530915622 >> 172.21.15.196:0/2530915622 conn(0x7f672c06c6c0 msgr2=0x7f672c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:15.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.230+0000 7f6731e00640 1 -- 172.21.15.196:0/2530915622 shutdown_connections 2024-04-28T13:32:15.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.230+0000 7f6731e00640 1 -- 172.21.15.196:0/2530915622 wait complete. 2024-04-28T13:32:15.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.231+0000 7f6731e00640 1 Processor -- start 2024-04-28T13:32:15.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.232+0000 7f6731e00640 1 -- start start 2024-04-28T13:32:15.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.232+0000 7f6731e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f672c09ec80 0x7f672c158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:15.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.232+0000 7f6731e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f672c0add70 0x7f672c1f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:15.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.232+0000 7f6731e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0b5ee0 0x7f672c1f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:15.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.232+0000 7f6731e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f672c0b3c20 con 0x7f672c09ec80 2024-04-28T13:32:15.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.232+0000 7f6731e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f672c0b47b0 con 0x7f672c0add70 2024-04-28T13:32:15.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.232+0000 7f6731e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f672c0d1590 con 0x7f672c0b5ee0 2024-04-28T13:32:15.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.233+0000 7f6730a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f672c09ec80 0x7f672c158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:15.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.233+0000 7f6730a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f672c09ec80 0x7f672c158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49560/0 (socket says 172.21.15.196:49560) 2024-04-28T13:32:15.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.233+0000 7f6730a00640 1 -- 172.21.15.196:0/3598480576 learned_addr learned my addr 172.21.15.196:0/3598480576 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:15.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.233+0000 7f6723e00640 1 --2- 172.21.15.196:0/3598480576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f672c0add70 0x7f672c1f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:15.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.233+0000 7f6730a00640 1 -- 172.21.15.196:0/3598480576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0b5ee0 msgr2=0x7f672c1f6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:15.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.233+0000 7f6730a00640 1 --2- 172.21.15.196:0/3598480576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0b5ee0 0x7f672c1f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.233+0000 7f6730a00640 1 -- 172.21.15.196:0/3598480576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f672c0add70 msgr2=0x7f672c1f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:15.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.233+0000 7f6730a00640 1 --2- 172.21.15.196:0/3598480576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f672c0add70 0x7f672c1f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.233+0000 7f6730a00640 1 -- 172.21.15.196:0/3598480576 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672c156500 con 0x7f672c09ec80 2024-04-28T13:32:15.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.234+0000 7f6730a00640 1 --2- 172.21.15.196:0/3598480576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f672c09ec80 0x7f672c158080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f671c0098c0 tx=0x7f671c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:15.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.234+0000 7f6729600640 1 -- 172.21.15.196:0/3598480576 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c041030 con 0x7f672c09ec80 2024-04-28T13:32:15.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.234+0000 7f6729600640 1 -- 172.21.15.196:0/3598480576 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671c03d4e0 con 0x7f672c09ec80 2024-04-28T13:32:15.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.234+0000 7f6731e00640 1 -- 172.21.15.196:0/3598480576 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672c0cdf90 con 0x7f672c09ec80 2024-04-28T13:32:15.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.234+0000 7f6729600640 1 -- 172.21.15.196:0/3598480576 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c037040 con 0x7f672c09ec80 2024-04-28T13:32:15.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.234+0000 7f6731e00640 1 -- 172.21.15.196:0/3598480576 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f672c1f6b20 con 0x7f672c09ec80 2024-04-28T13:32:15.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.235+0000 7f6731e00640 1 -- 172.21.15.196:0/3598480576 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f672c06a290 con 0x7f672c09ec80 2024-04-28T13:32:15.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.236+0000 7f6729600640 1 -- 172.21.15.196:0/3598480576 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f671c044070 con 0x7f672c09ec80 2024-04-28T13:32:15.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.237+0000 7f6729600640 1 --2- 172.21.15.196:0/3598480576 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f670806c8f0 0x7f670806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:15.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.237+0000 7f6729600640 1 -- 172.21.15.196:0/3598480576 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f671c0ac2f0 con 0x7f672c09ec80 2024-04-28T13:32:15.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.238+0000 7f6723e00640 1 --2- 172.21.15.196:0/3598480576 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f670806c8f0 0x7f670806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:15.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.239+0000 7f6723e00640 1 --2- 172.21.15.196:0/3598480576 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f670806c8f0 0x7f670806edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6714009560 tx=0x7f6714008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:15.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.242+0000 7f6729600640 1 -- 172.21.15.196:0/3598480576 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f671c07a610 con 0x7f672c09ec80 2024-04-28T13:32:15.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.484+0000 7f6731e00640 1 -- 172.21.15.196:0/3598480576 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f672c06ee60 con 0x7f672c09ec80 2024-04-28T13:32:15.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.486+0000 7f6729600640 1 -- 172.21.15.196:0/3598480576 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f671c079f10 con 0x7f672c09ec80 2024-04-28T13:32:15.487 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:15.487 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/422653807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":422653807},{"type":"v1","addr":"172.21.15.196:6834","nonce":422653807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2793552376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2793552376},{"type":"v1","addr":"172.21.15.196:6835","nonce":2793552376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1810956193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1810956193},{"type":"v1","addr":"172.21.15.203:6837","nonce":1810956193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:31:21.059586+0000","modified":"2024-04-28T13:31:24.132134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/2114798853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2114798853},{"type":"v1","addr":"172.21.15.203:6836","nonce":2114798853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4253,"qdb_cluster":[4253]},"id":1}]} 2024-04-28T13:32:15.487 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 7 2024-04-28T13:32:15.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.489+0000 7f6731e00640 1 -- 172.21.15.196:0/3598480576 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f670806c8f0 msgr2=0x7f670806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:15.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.489+0000 7f6731e00640 1 --2- 172.21.15.196:0/3598480576 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f670806c8f0 0x7f670806edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6714009560 tx=0x7f6714008040 comp rx=0 tx=0).stop 2024-04-28T13:32:15.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.489+0000 7f6731e00640 1 -- 172.21.15.196:0/3598480576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f672c09ec80 msgr2=0x7f672c158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:15.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.489+0000 7f6731e00640 1 --2- 172.21.15.196:0/3598480576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f672c09ec80 0x7f672c158080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f671c0098c0 tx=0x7f671c0046c0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.489+0000 7f6731e00640 1 -- 172.21.15.196:0/3598480576 shutdown_connections 2024-04-28T13:32:15.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.489+0000 7f6731e00640 1 --2- 172.21.15.196:0/3598480576 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f670806c8f0 0x7f670806edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.489+0000 7f6731e00640 1 --2- 172.21.15.196:0/3598480576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f672c0b5ee0 0x7f672c1f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.489+0000 7f6731e00640 1 --2- 172.21.15.196:0/3598480576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f672c0add70 0x7f672c1f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.489+0000 7f6731e00640 1 --2- 172.21.15.196:0/3598480576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f672c09ec80 0x7f672c158080 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.489+0000 7f6731e00640 1 -- 172.21.15.196:0/3598480576 >> 172.21.15.196:0/3598480576 conn(0x7f672c06c6c0 msgr2=0x7f672c0afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:15.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.489+0000 7f6731e00640 1 -- 172.21.15.196:0/3598480576 shutdown_connections 2024-04-28T13:32:15.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.489+0000 7f6731e00640 1 -- 172.21.15.196:0/3598480576 wait complete. 2024-04-28T13:32:15.510 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:32:15.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.709+0000 7f73e5600640 1 Processor -- start 2024-04-28T13:32:15.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.709+0000 7f73e5600640 1 -- start start 2024-04-28T13:32:15.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.711+0000 7f73e5600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e01469f0 0x7f73e00ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:15.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.711+0000 7f73e5600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f73e00d1410 0x7f73e00b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:15.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.711+0000 7f73e5600640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73e0160f40 con 0x7f73e00cd420 2024-04-28T13:32:15.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.711+0000 7f73e5600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f73e00b5960 con 0x7f73e00d1410 2024-04-28T13:32:15.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.711+0000 7f73e5600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f73e0161ac0 con 0x7f73e01469f0 2024-04-28T13:32:15.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.712+0000 7f73dea00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e01469f0 0x7f73e00ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:15.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.712+0000 7f73dea00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e01469f0 0x7f73e00ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:49218/0 (socket says 172.21.15.196:49218) 2024-04-28T13:32:15.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.712+0000 7f73dea00640 1 -- 172.21.15.196:0/465696716 learned_addr learned my addr 172.21.15.196:0/465696716 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:15.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.712+0000 7f73df400640 1 --2- 172.21.15.196:0/465696716 >> v2:172.21.15.203:3300/0 conn(0x7f73e00d1410 0x7f73e00b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:15.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.712+0000 7f73dea00640 1 -- 172.21.15.196:0/465696716 >> v2:172.21.15.203:3300/0 conn(0x7f73e00d1410 msgr2=0x7f73e00b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:15.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.712+0000 7f73dea00640 1 --2- 172.21.15.196:0/465696716 >> v2:172.21.15.203:3300/0 conn(0x7f73e00d1410 0x7f73e00b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.712+0000 7f73dea00640 1 -- 172.21.15.196:0/465696716 >> v1:172.21.15.203:6789/0 conn(0x7f73e00cd420 legacy=0x7f73e00d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:15.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.712+0000 7f73dea00640 1 -- 172.21.15.196:0/465696716 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e0161220 con 0x7f73e01469f0 2024-04-28T13:32:15.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.714+0000 7f73dea00640 1 --2- 172.21.15.196:0/465696716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e01469f0 0x7f73e00ccee0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f73e00d1de0 tx=0x7f73d002a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=250758edf6a22814 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:15.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.714+0000 7f73de000640 1 -- 172.21.15.196:0/465696716 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0037070 con 0x7f73e01469f0 2024-04-28T13:32:15.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.714+0000 7f73de000640 1 -- 172.21.15.196:0/465696716 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73d0032440 con 0x7f73e01469f0 2024-04-28T13:32:15.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.714+0000 7f73de000640 1 -- 172.21.15.196:0/465696716 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0032810 con 0x7f73e01469f0 2024-04-28T13:32:15.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.715+0000 7f73e5600640 1 -- 172.21.15.196:0/465696716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e01469f0 msgr2=0x7f73e00ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:15.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.715+0000 7f73e5600640 1 --2- 172.21.15.196:0/465696716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e01469f0 0x7f73e00ccee0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f73e00d1de0 tx=0x7f73d002a200 comp rx=0 tx=0).stop 2024-04-28T13:32:15.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.715+0000 7f73e5600640 1 -- 172.21.15.196:0/465696716 shutdown_connections 2024-04-28T13:32:15.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.715+0000 7f73e5600640 1 --2- 172.21.15.196:0/465696716 >> v2:172.21.15.203:3300/0 conn(0x7f73e00d1410 0x7f73e00b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.715+0000 7f73e5600640 1 --2- 172.21.15.196:0/465696716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e01469f0 0x7f73e00ccee0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.715+0000 7f73e5600640 1 -- 172.21.15.196:0/465696716 >> 172.21.15.196:0/465696716 conn(0x7f73e00cd830 msgr2=0x7f73e00cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:15.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.715+0000 7f73e5600640 1 -- 172.21.15.196:0/465696716 shutdown_connections 2024-04-28T13:32:15.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.715+0000 7f73e5600640 1 -- 172.21.15.196:0/465696716 wait complete. 2024-04-28T13:32:15.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.717+0000 7f73e5600640 1 Processor -- start 2024-04-28T13:32:15.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.718+0000 7f73e5600640 1 -- start start 2024-04-28T13:32:15.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.719+0000 7f73e5600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e00cd420 0x7f73e01588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:15.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.719+0000 7f73e5600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e00d1410 0x7f73e012c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:15.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.719+0000 7f73e5600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73e01469f0 0x7f73e00721a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:15.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.719+0000 7f73e5600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0072020 con 0x7f73e00cd420 2024-04-28T13:32:15.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.719+0000 7f73e5600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01617e0 con 0x7f73e01469f0 2024-04-28T13:32:15.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.719+0000 7f73e5600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f73e0160b80 con 0x7f73e00d1410 2024-04-28T13:32:15.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.719+0000 7f73dea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e00cd420 0x7f73e01588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:15.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.719+0000 7f73dea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e00cd420 0x7f73e01588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49568/0 (socket says 172.21.15.196:49568) 2024-04-28T13:32:15.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.719+0000 7f73dea00640 1 -- 172.21.15.196:0/1132941877 learned_addr learned my addr 172.21.15.196:0/1132941877 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:15.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.719+0000 7f73dea00640 1 -- 172.21.15.196:0/1132941877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e00d1410 msgr2=0x7f73e012c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:15.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.719+0000 7f73d6000640 1 --2- 172.21.15.196:0/1132941877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e00d1410 0x7f73e012c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:15.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.720+0000 7f73dea00640 1 --2- 172.21.15.196:0/1132941877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e00d1410 0x7f73e012c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.720+0000 7f73dea00640 1 -- 172.21.15.196:0/1132941877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73e01469f0 msgr2=0x7f73e00721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:15.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.720+0000 7f73df400640 1 --2- 172.21.15.196:0/1132941877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73e01469f0 0x7f73e00721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:15.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.720+0000 7f73dea00640 1 --2- 172.21.15.196:0/1132941877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73e01469f0 0x7f73e00721a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.720+0000 7f73dea00640 1 -- 172.21.15.196:0/1132941877 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e012cd20 con 0x7f73e00cd420 2024-04-28T13:32:15.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.720+0000 7f73d6000640 1 --2- 172.21.15.196:0/1132941877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e00d1410 0x7f73e012c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:15.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.720+0000 7f73dea00640 1 --2- 172.21.15.196:0/1132941877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e00cd420 0x7f73e01588d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f73d002a240 tx=0x7f73d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:15.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.720+0000 7f73d7400640 1 -- 172.21.15.196:0/1132941877 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0041070 con 0x7f73e00cd420 2024-04-28T13:32:15.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.720+0000 7f73d7400640 1 -- 172.21.15.196:0/1132941877 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73d003c870 con 0x7f73e00cd420 2024-04-28T13:32:15.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.720+0000 7f73e5600640 1 -- 172.21.15.196:0/1132941877 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e012d000 con 0x7f73e00cd420 2024-04-28T13:32:15.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.720+0000 7f73d7400640 1 -- 172.21.15.196:0/1132941877 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0037040 con 0x7f73e00cd420 2024-04-28T13:32:15.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.721+0000 7f73e5600640 1 -- 172.21.15.196:0/1132941877 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e012d5c0 con 0x7f73e00cd420 2024-04-28T13:32:15.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.722+0000 7f73e5600640 1 -- 172.21.15.196:0/1132941877 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e006a290 con 0x7f73e00cd420 2024-04-28T13:32:15.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.723+0000 7f73d7400640 1 -- 172.21.15.196:0/1132941877 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73d0044070 con 0x7f73e00cd420 2024-04-28T13:32:15.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.723+0000 7f73d7400640 1 --2- 172.21.15.196:0/1132941877 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73bc06c8f0 0x7f73bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:15.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.724+0000 7f73d7400640 1 -- 172.21.15.196:0/1132941877 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f73d00ac560 con 0x7f73e00cd420 2024-04-28T13:32:15.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.724+0000 7f73d6000640 1 --2- 172.21.15.196:0/1132941877 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73bc06c8f0 0x7f73bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:15.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.725+0000 7f73d6000640 1 --2- 172.21.15.196:0/1132941877 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73bc06c8f0 0x7f73bc06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f73e012f3f0 tx=0x7f73c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:15.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.728+0000 7f73d7400640 1 -- 172.21.15.196:0/1132941877 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f73d007aa30 con 0x7f73e00cd420 2024-04-28T13:32:15.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.930+0000 7f73e5600640 1 -- 172.21.15.196:0/1132941877 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73e006fd90 con 0x7f73e00cd420 2024-04-28T13:32:15.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.933+0000 7f73d7400640 1 -- 172.21.15.196:0/1132941877 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16890 (secure 0 0 0) 0x7f73d007a330 con 0x7f73e00cd420 2024-04-28T13:32:15.934 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:15.934 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:03.882452+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:32:15.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.935+0000 7f73e5600640 1 -- 172.21.15.196:0/1132941877 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73bc06c8f0 msgr2=0x7f73bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:15.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.935+0000 7f73e5600640 1 --2- 172.21.15.196:0/1132941877 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73bc06c8f0 0x7f73bc06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f73e012f3f0 tx=0x7f73c8009040 comp rx=0 tx=0).stop 2024-04-28T13:32:15.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.935+0000 7f73e5600640 1 -- 172.21.15.196:0/1132941877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e00cd420 msgr2=0x7f73e01588d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:15.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.935+0000 7f73e5600640 1 --2- 172.21.15.196:0/1132941877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e00cd420 0x7f73e01588d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f73d002a240 tx=0x7f73d00046c0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.936+0000 7f73e5600640 1 -- 172.21.15.196:0/1132941877 shutdown_connections 2024-04-28T13:32:15.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.936+0000 7f73e5600640 1 --2- 172.21.15.196:0/1132941877 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73bc06c8f0 0x7f73bc06edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.936+0000 7f73e5600640 1 --2- 172.21.15.196:0/1132941877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73e01469f0 0x7f73e00721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.936+0000 7f73e5600640 1 --2- 172.21.15.196:0/1132941877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73e00d1410 0x7f73e012c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.936+0000 7f73e5600640 1 --2- 172.21.15.196:0/1132941877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73e00cd420 0x7f73e01588d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:15.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.936+0000 7f73e5600640 1 -- 172.21.15.196:0/1132941877 >> 172.21.15.196:0/1132941877 conn(0x7f73e00cd830 msgr2=0x7f73e00b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:15.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.936+0000 7f73e5600640 1 -- 172.21.15.196:0/1132941877 shutdown_connections 2024-04-28T13:32:15.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:15.936+0000 7f73e5600640 1 -- 172.21.15.196:0/1132941877 wait complete. 2024-04-28T13:32:15.955 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:31:19.280975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:31:20.037066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:32:15.956 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:32:16.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.155+0000 7f3870a00640 1 Processor -- start 2024-04-28T13:32:16.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.155+0000 7f3870a00640 1 -- start start 2024-04-28T13:32:16.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.157+0000 7f3870a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f386c143840 0x7f386c0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:16.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.157+0000 7f3870a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f386c0b55c0 0x7f386c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:16.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.157+0000 7f3870a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f386c15dca0 con 0x7f386c0b4520 2024-04-28T13:32:16.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.157+0000 7f3870a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f386c143c20 con 0x7f386c143840 2024-04-28T13:32:16.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.157+0000 7f3870a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f386c15df80 con 0x7f386c0b55c0 2024-04-28T13:32:16.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.157+0000 7f3869e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f386c143840 0x7f386c0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:16.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.157+0000 7f3869e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f386c143840 0x7f386c0b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49580/0 (socket says 172.21.15.196:49580) 2024-04-28T13:32:16.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.157+0000 7f3861400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f386c0b4520 0x7f386c0b4eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:44334/0 (socket says 172.21.15.196:44334) 2024-04-28T13:32:16.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.157+0000 7f3869e00640 1 -- 172.21.15.196:0/2569785382 learned_addr learned my addr 172.21.15.196:0/2569785382 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:16.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.157+0000 7f386a800640 1 --2- 172.21.15.196:0/2569785382 >> v2:172.21.15.203:3300/0 conn(0x7f386c0b55c0 0x7f386c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:16.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.158+0000 7f3869e00640 1 -- 172.21.15.196:0/2569785382 >> v2:172.21.15.203:3300/0 conn(0x7f386c0b55c0 msgr2=0x7f386c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:16.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.158+0000 7f3869e00640 1 --2- 172.21.15.196:0/2569785382 >> v2:172.21.15.203:3300/0 conn(0x7f386c0b55c0 0x7f386c0ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.158+0000 7f3869e00640 1 -- 172.21.15.196:0/2569785382 >> v1:172.21.15.196:6789/0 conn(0x7f386c0b4520 legacy=0x7f386c0b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:16.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.158+0000 7f3869e00640 1 -- 172.21.15.196:0/2569785382 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386c15e260 con 0x7f386c143840 2024-04-28T13:32:16.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.158+0000 7f3869400640 1 -- 172.21.15.196:0/2569785382 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3134375670 0 0) 0x7f386c15dca0 con 0x7f386c0b4520 2024-04-28T13:32:16.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.158+0000 7f386a800640 1 --2- 172.21.15.196:0/2569785382 >> v2:172.21.15.203:3300/0 conn(0x7f386c0b55c0 0x7f386c0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:16.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.159+0000 7f3869e00640 1 --2- 172.21.15.196:0/2569785382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386c143840 0x7f386c0b3fe0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f385c009870 tx=0x7f385c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c667202821e575 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:16.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.159+0000 7f3869400640 1 -- 172.21.15.196:0/2569785382 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c037070 con 0x7f386c143840 2024-04-28T13:32:16.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.159+0000 7f3869400640 1 -- 172.21.15.196:0/2569785382 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f385c032440 con 0x7f386c143840 2024-04-28T13:32:16.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.159+0000 7f3869400640 1 -- 172.21.15.196:0/2569785382 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c032810 con 0x7f386c143840 2024-04-28T13:32:16.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.159+0000 7f3870a00640 1 -- 172.21.15.196:0/2569785382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386c143840 msgr2=0x7f386c0b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:16.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.159+0000 7f3870a00640 1 --2- 172.21.15.196:0/2569785382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386c143840 0x7f386c0b3fe0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f385c009870 tx=0x7f385c02a2a0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.160+0000 7f3870a00640 1 -- 172.21.15.196:0/2569785382 shutdown_connections 2024-04-28T13:32:16.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.160+0000 7f3870a00640 1 --2- 172.21.15.196:0/2569785382 >> v2:172.21.15.203:3300/0 conn(0x7f386c0b55c0 0x7f386c0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.160+0000 7f3870a00640 1 --2- 172.21.15.196:0/2569785382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386c143840 0x7f386c0b3fe0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.160+0000 7f3870a00640 1 -- 172.21.15.196:0/2569785382 >> 172.21.15.196:0/2569785382 conn(0x7f386c0cd8a0 msgr2=0x7f386c0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:16.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.160+0000 7f3870a00640 1 -- 172.21.15.196:0/2569785382 shutdown_connections 2024-04-28T13:32:16.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.160+0000 7f3870a00640 1 -- 172.21.15.196:0/2569785382 wait complete. 2024-04-28T13:32:16.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.161+0000 7f3870a00640 1 Processor -- start 2024-04-28T13:32:16.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.161+0000 7f3870a00640 1 -- start start 2024-04-28T13:32:16.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.162+0000 7f3870a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f386c0b4520 0x7f386c1f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:16.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.162+0000 7f3870a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f386c0b55c0 0x7f386c1f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:16.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.162+0000 7f3870a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386c143840 0x7f386c13dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:16.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.162+0000 7f3870a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f386c15da40 con 0x7f386c143840 2024-04-28T13:32:16.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.162+0000 7f3870a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f386c15d8c0 con 0x7f386c0b55c0 2024-04-28T13:32:16.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.162+0000 7f3870a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f386c0cf530 con 0x7f386c0b4520 2024-04-28T13:32:16.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.163+0000 7f386a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386c143840 0x7f386c13dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:16.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.163+0000 7f386a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386c143840 0x7f386c13dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49584/0 (socket says 172.21.15.196:49584) 2024-04-28T13:32:16.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.163+0000 7f386a800640 1 -- 172.21.15.196:0/963170876 learned_addr learned my addr 172.21.15.196:0/963170876 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:16.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.163+0000 7f3869e00640 1 --2- 172.21.15.196:0/963170876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f386c0b4520 0x7f386c1f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:16.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.163+0000 7f3861400640 1 --2- 172.21.15.196:0/963170876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f386c0b55c0 0x7f386c1f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:16.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.163+0000 7f386a800640 1 -- 172.21.15.196:0/963170876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f386c0b4520 msgr2=0x7f386c1f8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:16.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.163+0000 7f386a800640 1 --2- 172.21.15.196:0/963170876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f386c0b4520 0x7f386c1f8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.163+0000 7f386a800640 1 -- 172.21.15.196:0/963170876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f386c0b55c0 msgr2=0x7f386c1f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:16.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.163+0000 7f386a800640 1 --2- 172.21.15.196:0/963170876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f386c0b55c0 0x7f386c1f5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.163+0000 7f386a800640 1 -- 172.21.15.196:0/963170876 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386c1f90e0 con 0x7f386c143840 2024-04-28T13:32:16.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.163+0000 7f386a800640 1 --2- 172.21.15.196:0/963170876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386c143840 0x7f386c13dfe0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f386400ce80 tx=0x7f386400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:16.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.164+0000 7f3862a00640 1 -- 172.21.15.196:0/963170876 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386401a070 con 0x7f386c143840 2024-04-28T13:32:16.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.164+0000 7f3862a00640 1 -- 172.21.15.196:0/963170876 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f386400d880 con 0x7f386c143840 2024-04-28T13:32:16.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.164+0000 7f3862a00640 1 -- 172.21.15.196:0/963170876 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386400dc70 con 0x7f386c143840 2024-04-28T13:32:16.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.164+0000 7f3870a00640 1 -- 172.21.15.196:0/963170876 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f386c1f93c0 con 0x7f386c143840 2024-04-28T13:32:16.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.164+0000 7f3870a00640 1 -- 172.21.15.196:0/963170876 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f386c13e520 con 0x7f386c143840 2024-04-28T13:32:16.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.165+0000 7f3870a00640 1 -- 172.21.15.196:0/963170876 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f386c0709a0 con 0x7f386c143840 2024-04-28T13:32:16.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.166+0000 7f3862a00640 1 -- 172.21.15.196:0/963170876 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f386401d070 con 0x7f386c143840 2024-04-28T13:32:16.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.167+0000 7f3862a00640 1 --2- 172.21.15.196:0/963170876 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f384806c940 0x7f384806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:16.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.167+0000 7f3862a00640 1 -- 172.21.15.196:0/963170876 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f386408c2c0 con 0x7f386c143840 2024-04-28T13:32:16.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.168+0000 7f3869e00640 1 --2- 172.21.15.196:0/963170876 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f384806c940 0x7f384806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:16.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.169+0000 7f3869e00640 1 --2- 172.21.15.196:0/963170876 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f384806c940 0x7f384806ee00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f385c034000 tx=0x7f385c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:16.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.171+0000 7f3862a00640 1 -- 172.21.15.196:0/963170876 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f386c0709a0 con 0x7f386c143840 2024-04-28T13:32:16.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.372+0000 7f3870a00640 1 -- 172.21.15.196:0/963170876 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f386c1f98e0 con 0x7f386c143840 2024-04-28T13:32:16.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.373+0000 7f3862a00640 1 -- 172.21.15.196:0/963170876 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f386405a090 con 0x7f386c143840 2024-04-28T13:32:16.374 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:32:16.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.375+0000 7f3870a00640 1 -- 172.21.15.196:0/963170876 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f384806c940 msgr2=0x7f384806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:16.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.375+0000 7f3870a00640 1 --2- 172.21.15.196:0/963170876 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f384806c940 0x7f384806ee00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f385c034000 tx=0x7f385c02e000 comp rx=0 tx=0).stop 2024-04-28T13:32:16.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.375+0000 7f3870a00640 1 -- 172.21.15.196:0/963170876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386c143840 msgr2=0x7f386c13dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:16.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.375+0000 7f3870a00640 1 --2- 172.21.15.196:0/963170876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386c143840 0x7f386c13dfe0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f386400ce80 tx=0x7f386400a710 comp rx=0 tx=0).stop 2024-04-28T13:32:16.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.376+0000 7f3870a00640 1 -- 172.21.15.196:0/963170876 shutdown_connections 2024-04-28T13:32:16.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.376+0000 7f3870a00640 1 --2- 172.21.15.196:0/963170876 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f384806c940 0x7f384806ee00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.376+0000 7f3870a00640 1 --2- 172.21.15.196:0/963170876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f386c143840 0x7f386c13dfe0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.376+0000 7f3870a00640 1 --2- 172.21.15.196:0/963170876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f386c0b55c0 0x7f386c1f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.376+0000 7f3870a00640 1 --2- 172.21.15.196:0/963170876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f386c0b4520 0x7f386c1f8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.376+0000 7f3870a00640 1 -- 172.21.15.196:0/963170876 >> 172.21.15.196:0/963170876 conn(0x7f386c0cd8a0 msgr2=0x7f386c0cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:16.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.376+0000 7f3870a00640 1 -- 172.21.15.196:0/963170876 shutdown_connections 2024-04-28T13:32:16.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.376+0000 7f3870a00640 1 -- 172.21.15.196:0/963170876 wait complete. 2024-04-28T13:32:16.396 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:32:16.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.592+0000 7faf73200640 1 Processor -- start 2024-04-28T13:32:16.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.593+0000 7faf73200640 1 -- start start 2024-04-28T13:32:16.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.594+0000 7faf73200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf74155da0 0x7faf7415c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:16.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.594+0000 7faf73200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faf740737b0 0x7faf74073c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:16.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.594+0000 7faf73200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf740b4e10 con 0x7faf740b4650 2024-04-28T13:32:16.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.594+0000 7faf73200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7faf74072990 con 0x7faf740737b0 2024-04-28T13:32:16.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.594+0000 7faf73200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7faf74072060 con 0x7faf74155da0 2024-04-28T13:32:16.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.595+0000 7faf72800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faf740737b0 0x7faf74073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:16.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.595+0000 7faf72800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faf740737b0 0x7faf74073c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49588/0 (socket says 172.21.15.196:49588) 2024-04-28T13:32:16.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.595+0000 7faf72800640 1 -- 172.21.15.196:0/2442839170 learned_addr learned my addr 172.21.15.196:0/2442839170 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:16.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.595+0000 7faf71e00640 1 --2- 172.21.15.196:0/2442839170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf74155da0 0x7faf7415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:16.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.595+0000 7faf72800640 1 -- 172.21.15.196:0/2442839170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf74155da0 msgr2=0x7faf7415c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:16.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.595+0000 7faf72800640 1 --2- 172.21.15.196:0/2442839170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf74155da0 0x7faf7415c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.595+0000 7faf72800640 1 -- 172.21.15.196:0/2442839170 >> v1:172.21.15.203:6789/0 conn(0x7faf740b4650 legacy=0x7faf74072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:16.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.595+0000 7faf72800640 1 -- 172.21.15.196:0/2442839170 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf74072340 con 0x7faf740737b0 2024-04-28T13:32:16.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.596+0000 7faf72800640 1 --2- 172.21.15.196:0/2442839170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf740737b0 0x7faf74073c40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00af60 tx=0x7faf6c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b620c0016a104d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:16.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.597+0000 7faf71400640 1 -- 172.21.15.196:0/2442839170 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c038070 con 0x7faf740737b0 2024-04-28T13:32:16.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.597+0000 7faf71400640 1 -- 172.21.15.196:0/2442839170 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf6c00a2e0 con 0x7faf740737b0 2024-04-28T13:32:16.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.597+0000 7faf71400640 1 -- 172.21.15.196:0/2442839170 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c008720 con 0x7faf740737b0 2024-04-28T13:32:16.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.597+0000 7faf73200640 1 -- 172.21.15.196:0/2442839170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf740737b0 msgr2=0x7faf74073c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:16.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.597+0000 7faf73200640 1 --2- 172.21.15.196:0/2442839170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf740737b0 0x7faf74073c40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00af60 tx=0x7faf6c02b860 comp rx=0 tx=0).stop 2024-04-28T13:32:16.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.597+0000 7faf73200640 1 -- 172.21.15.196:0/2442839170 shutdown_connections 2024-04-28T13:32:16.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.597+0000 7faf73200640 1 --2- 172.21.15.196:0/2442839170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf740737b0 0x7faf74073c40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.597+0000 7faf73200640 1 --2- 172.21.15.196:0/2442839170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf74155da0 0x7faf7415c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.597+0000 7faf73200640 1 -- 172.21.15.196:0/2442839170 >> 172.21.15.196:0/2442839170 conn(0x7faf7406c6c0 msgr2=0x7faf7406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:16.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.597+0000 7faf73200640 1 -- 172.21.15.196:0/2442839170 shutdown_connections 2024-04-28T13:32:16.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.597+0000 7faf73200640 1 -- 172.21.15.196:0/2442839170 wait complete. 2024-04-28T13:32:16.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.599+0000 7faf73200640 1 Processor -- start 2024-04-28T13:32:16.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.599+0000 7faf73200640 1 -- start start 2024-04-28T13:32:16.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.600+0000 7faf73200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf740737b0 0x7faf7415b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:16.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.600+0000 7faf73200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf740b4650 0x7faf740ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:16.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.600+0000 7faf73200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf74155da0 0x7faf740aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:16.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.600+0000 7faf73200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faf740d10a0 con 0x7faf740b4650 2024-04-28T13:32:16.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.600+0000 7faf73200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faf740b4ab0 con 0x7faf74155da0 2024-04-28T13:32:16.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.600+0000 7faf73200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7faf740d1220 con 0x7faf740737b0 2024-04-28T13:32:16.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.600+0000 7faf6be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf740b4650 0x7faf740ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:16.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.600+0000 7faf6be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf740b4650 0x7faf740ae9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49596/0 (socket says 172.21.15.196:49596) 2024-04-28T13:32:16.602 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.600+0000 7faf6be00640 1 -- 172.21.15.196:0/3976455915 learned_addr learned my addr 172.21.15.196:0/3976455915 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:16.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.601+0000 7faf72800640 1 --2- 172.21.15.196:0/3976455915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf74155da0 0x7faf740aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:16.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.601+0000 7faf71e00640 1 --2- 172.21.15.196:0/3976455915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf740737b0 0x7faf7415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:16.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.601+0000 7faf6be00640 1 -- 172.21.15.196:0/3976455915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf740737b0 msgr2=0x7faf7415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:16.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.601+0000 7faf6be00640 1 --2- 172.21.15.196:0/3976455915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf740737b0 0x7faf7415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.601+0000 7faf6be00640 1 -- 172.21.15.196:0/3976455915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf74155da0 msgr2=0x7faf740aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:16.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.601+0000 7faf6be00640 1 --2- 172.21.15.196:0/3976455915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf74155da0 0x7faf740aef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.601+0000 7faf6be00640 1 -- 172.21.15.196:0/3976455915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7414f2b0 con 0x7faf740b4650 2024-04-28T13:32:16.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.601+0000 7faf72800640 1 --2- 172.21.15.196:0/3976455915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf74155da0 0x7faf740aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:16.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.601+0000 7faf6be00640 1 --2- 172.21.15.196:0/3976455915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf740b4650 0x7faf740ae9d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00c460 tx=0x7faf5c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:16.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.601+0000 7faf6a000640 1 -- 172.21.15.196:0/3976455915 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c016020 con 0x7faf740b4650 2024-04-28T13:32:16.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.601+0000 7faf6a000640 1 -- 172.21.15.196:0/3976455915 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf5c012870 con 0x7faf740b4650 2024-04-28T13:32:16.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.601+0000 7faf6a000640 1 -- 172.21.15.196:0/3976455915 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c012c60 con 0x7faf740b4650 2024-04-28T13:32:16.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.601+0000 7faf73200640 1 -- 172.21.15.196:0/3976455915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7414f590 con 0x7faf740b4650 2024-04-28T13:32:16.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.602+0000 7faf73200640 1 -- 172.21.15.196:0/3976455915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7414fb50 con 0x7faf740b4650 2024-04-28T13:32:16.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.603+0000 7faf73200640 1 -- 172.21.15.196:0/3976455915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf74050730 con 0x7faf740b4650 2024-04-28T13:32:16.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.604+0000 7faf6a000640 1 -- 172.21.15.196:0/3976455915 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf5c019020 con 0x7faf740b4650 2024-04-28T13:32:16.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.605+0000 7faf6a000640 1 --2- 172.21.15.196:0/3976455915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faf5006c8f0 0x7faf5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:16.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.605+0000 7faf6a000640 1 -- 172.21.15.196:0/3976455915 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7faf5c08b890 con 0x7faf740b4650 2024-04-28T13:32:16.606 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.605+0000 7faf71e00640 1 --2- 172.21.15.196:0/3976455915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faf5006c8f0 0x7faf5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:16.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.607+0000 7faf71e00640 1 --2- 172.21.15.196:0/3976455915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faf5006c8f0 0x7faf5006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf64006fd0 tx=0x7faf64008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:16.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.609+0000 7faf6a000640 1 -- 172.21.15.196:0/3976455915 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf5c020030 con 0x7faf740b4650 2024-04-28T13:32:16.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.810+0000 7faf73200640 1 -- 172.21.15.196:0/3976455915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faf7406ccf0 con 0x7faf740b4650 2024-04-28T13:32:16.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.811+0000 7faf6a000640 1 -- 172.21.15.196:0/3976455915 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7faf5c059c40 con 0x7faf740b4650 2024-04-28T13:32:16.812 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:32:16.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.813+0000 7faf73200640 1 -- 172.21.15.196:0/3976455915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faf5006c8f0 msgr2=0x7faf5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:16.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.813+0000 7faf73200640 1 --2- 172.21.15.196:0/3976455915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faf5006c8f0 0x7faf5006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf64006fd0 tx=0x7faf64008040 comp rx=0 tx=0).stop 2024-04-28T13:32:16.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.813+0000 7faf73200640 1 -- 172.21.15.196:0/3976455915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf740b4650 msgr2=0x7faf740ae9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:16.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.813+0000 7faf73200640 1 --2- 172.21.15.196:0/3976455915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf740b4650 0x7faf740ae9d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00c460 tx=0x7faf5c00c920 comp rx=0 tx=0).stop 2024-04-28T13:32:16.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.814+0000 7faf73200640 1 -- 172.21.15.196:0/3976455915 shutdown_connections 2024-04-28T13:32:16.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.814+0000 7faf73200640 1 --2- 172.21.15.196:0/3976455915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faf5006c8f0 0x7faf5006edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.814+0000 7faf73200640 1 --2- 172.21.15.196:0/3976455915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf74155da0 0x7faf740aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.814+0000 7faf73200640 1 --2- 172.21.15.196:0/3976455915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf740b4650 0x7faf740ae9d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.814+0000 7faf73200640 1 --2- 172.21.15.196:0/3976455915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf740737b0 0x7faf7415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:16.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.814+0000 7faf73200640 1 -- 172.21.15.196:0/3976455915 >> 172.21.15.196:0/3976455915 conn(0x7faf7406c6c0 msgr2=0x7faf7406e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:16.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.814+0000 7faf73200640 1 -- 172.21.15.196:0/3976455915 shutdown_connections 2024-04-28T13:32:16.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:16.814+0000 7faf73200640 1 -- 172.21.15.196:0/3976455915 wait complete. 2024-04-28T13:32:16.833 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:32:17.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.037+0000 7fdff1200640 1 Processor -- start 2024-04-28T13:32:17.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.038+0000 7fdff1200640 1 -- start start 2024-04-28T13:32:17.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.040+0000 7fdff1200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdfec0b65f0 0x7fdfec0b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.040+0000 7fdff1200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdfec06fa50 0x7fdfec0ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.040+0000 7fdff1200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfec073440 con 0x7fdfec163b40 2024-04-28T13:32:17.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.040+0000 7fdff1200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fdfec0b6f10 con 0x7fdfec06fa50 2024-04-28T13:32:17.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.040+0000 7fdff1200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdfec0723a0 con 0x7fdfec0b65f0 2024-04-28T13:32:17.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.040+0000 7fdfea800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdfec06fa50 0x7fdfec0ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.040+0000 7fdfea800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdfec06fa50 0x7fdfec0ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49598/0 (socket says 172.21.15.196:49598) 2024-04-28T13:32:17.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.040+0000 7fdfea800640 1 -- 172.21.15.196:0/2312492011 learned_addr learned my addr 172.21.15.196:0/2312492011 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:17.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.041+0000 7fdfe9e00640 1 --2- 172.21.15.196:0/2312492011 >> v2:172.21.15.203:3300/0 conn(0x7fdfec0b65f0 0x7fdfec0b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.041+0000 7fdfea800640 1 -- 172.21.15.196:0/2312492011 >> v2:172.21.15.203:3300/0 conn(0x7fdfec0b65f0 msgr2=0x7fdfec0b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:17.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.041+0000 7fdfea800640 1 --2- 172.21.15.196:0/2312492011 >> v2:172.21.15.203:3300/0 conn(0x7fdfec0b65f0 0x7fdfec0b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.041+0000 7fdfea800640 1 -- 172.21.15.196:0/2312492011 >> v1:172.21.15.203:6789/0 conn(0x7fdfec163b40 legacy=0x7fdfec06f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:17.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.041+0000 7fdfea800640 1 -- 172.21.15.196:0/2312492011 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfec072680 con 0x7fdfec06fa50 2024-04-28T13:32:17.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.041+0000 7fdfe9e00640 1 --2- 172.21.15.196:0/2312492011 >> v2:172.21.15.203:3300/0 conn(0x7fdfec0b65f0 0x7fdfec0b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:17.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.042+0000 7fdfea800640 1 --2- 172.21.15.196:0/2312492011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfec06fa50 0x7fdfec0ceca0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0d0950 tx=0x7fdfe402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4da5634e13c7ef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:17.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.042+0000 7fdfe8a00640 1 -- 172.21.15.196:0/2312492011 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4036070 con 0x7fdfec06fa50 2024-04-28T13:32:17.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.042+0000 7fdfe8a00640 1 -- 172.21.15.196:0/2312492011 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfe4031600 con 0x7fdfec06fa50 2024-04-28T13:32:17.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.042+0000 7fdfe8a00640 1 -- 172.21.15.196:0/2312492011 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40319d0 con 0x7fdfec06fa50 2024-04-28T13:32:17.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.043+0000 7fdff1200640 1 -- 172.21.15.196:0/2312492011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfec06fa50 msgr2=0x7fdfec0ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:17.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.043+0000 7fdff1200640 1 --2- 172.21.15.196:0/2312492011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfec06fa50 0x7fdfec0ceca0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0d0950 tx=0x7fdfe402a660 comp rx=0 tx=0).stop 2024-04-28T13:32:17.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.043+0000 7fdff1200640 1 -- 172.21.15.196:0/2312492011 shutdown_connections 2024-04-28T13:32:17.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.043+0000 7fdff1200640 1 --2- 172.21.15.196:0/2312492011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfec06fa50 0x7fdfec0ceca0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.043+0000 7fdff1200640 1 --2- 172.21.15.196:0/2312492011 >> v2:172.21.15.203:3300/0 conn(0x7fdfec0b65f0 0x7fdfec0b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.043+0000 7fdff1200640 1 -- 172.21.15.196:0/2312492011 >> 172.21.15.196:0/2312492011 conn(0x7fdfec0cd8a0 msgr2=0x7fdfec0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:17.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.043+0000 7fdff1200640 1 -- 172.21.15.196:0/2312492011 shutdown_connections 2024-04-28T13:32:17.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.043+0000 7fdff1200640 1 -- 172.21.15.196:0/2312492011 wait complete. 2024-04-28T13:32:17.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.045+0000 7fdff1200640 1 Processor -- start 2024-04-28T13:32:17.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.045+0000 7fdff1200640 1 -- start start 2024-04-28T13:32:17.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.046+0000 7fdff1200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfec06fa50 0x7fdfec15cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.046+0000 7fdff1200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfec0b65f0 0x7fdfec15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.046+0000 7fdff1200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfec163b40 0x7fdfec1f90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.046+0000 7fdff1200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec0cf1e0 con 0x7fdfec163b40 2024-04-28T13:32:17.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.046+0000 7fdff1200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec072c40 con 0x7fdfec0b65f0 2024-04-28T13:32:17.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.046+0000 7fdff1200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdfec0cf360 con 0x7fdfec06fa50 2024-04-28T13:32:17.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.046+0000 7fdfea800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfec163b40 0x7fdfec1f90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.046+0000 7fdfea800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfec163b40 0x7fdfec1f90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49602/0 (socket says 172.21.15.196:49602) 2024-04-28T13:32:17.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.046+0000 7fdfea800640 1 -- 172.21.15.196:0/1955303755 learned_addr learned my addr 172.21.15.196:0/1955303755 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:17.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.046+0000 7fdfe9e00640 1 --2- 172.21.15.196:0/1955303755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfec06fa50 0x7fdfec15cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.047+0000 7fdfea800640 1 -- 172.21.15.196:0/1955303755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfec06fa50 msgr2=0x7fdfec15cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:17.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.047+0000 7fdfea800640 1 --2- 172.21.15.196:0/1955303755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfec06fa50 0x7fdfec15cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.047+0000 7fdfea800640 1 -- 172.21.15.196:0/1955303755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfec0b65f0 msgr2=0x7fdfec15b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:17.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.047+0000 7fdfe9400640 1 --2- 172.21.15.196:0/1955303755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfec0b65f0 0x7fdfec15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.047+0000 7fdfea800640 1 --2- 172.21.15.196:0/1955303755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfec0b65f0 0x7fdfec15b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.047+0000 7fdfea800640 1 -- 172.21.15.196:0/1955303755 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfec15b810 con 0x7fdfec163b40 2024-04-28T13:32:17.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.047+0000 7fdfe9e00640 1 --2- 172.21.15.196:0/1955303755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfec06fa50 0x7fdfec15cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:17.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.047+0000 7fdfea800640 1 --2- 172.21.15.196:0/1955303755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfec163b40 0x7fdfec1f90d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4034040 tx=0x7fdfe4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:17.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.047+0000 7fdfe2000640 1 -- 172.21.15.196:0/1955303755 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4041020 con 0x7fdfec163b40 2024-04-28T13:32:17.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.047+0000 7fdfe2000640 1 -- 172.21.15.196:0/1955303755 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfe4007780 con 0x7fdfec163b40 2024-04-28T13:32:17.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.047+0000 7fdfe2000640 1 -- 172.21.15.196:0/1955303755 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4036040 con 0x7fdfec163b40 2024-04-28T13:32:17.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.047+0000 7fdff1200640 1 -- 172.21.15.196:0/1955303755 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfec1640f0 con 0x7fdfec163b40 2024-04-28T13:32:17.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.047+0000 7fdff1200640 1 -- 172.21.15.196:0/1955303755 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfec1646b0 con 0x7fdfec163b40 2024-04-28T13:32:17.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.049+0000 7fdff1200640 1 -- 172.21.15.196:0/1955303755 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfec06a290 con 0x7fdfec163b40 2024-04-28T13:32:17.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.050+0000 7fdfe2000640 1 -- 172.21.15.196:0/1955303755 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfe4044020 con 0x7fdfec163b40 2024-04-28T13:32:17.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.050+0000 7fdfe2000640 1 --2- 172.21.15.196:0/1955303755 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdfc806c8f0 0x7fdfc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.050+0000 7fdfe2000640 1 -- 172.21.15.196:0/1955303755 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fdfe40aca80 con 0x7fdfec163b40 2024-04-28T13:32:17.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.051+0000 7fdfe9e00640 1 --2- 172.21.15.196:0/1955303755 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdfc806c8f0 0x7fdfc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.052+0000 7fdfe9e00640 1 --2- 172.21.15.196:0/1955303755 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdfc806c8f0 0x7fdfc806edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc006fd0 tx=0x7fdfdc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:17.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.055+0000 7fdfe2000640 1 -- 172.21.15.196:0/1955303755 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdfe40aee20 con 0x7fdfec163b40 2024-04-28T13:32:17.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.255+0000 7fdff1200640 1 -- 172.21.15.196:0/1955303755 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdfec161ca0 con 0x7fdfec163b40 2024-04-28T13:32:17.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.255+0000 7fdfe2000640 1 -- 172.21.15.196:0/1955303755 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fdfe404b030 con 0x7fdfec163b40 2024-04-28T13:32:17.256 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:32:17.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.258+0000 7fdff1200640 1 -- 172.21.15.196:0/1955303755 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdfc806c8f0 msgr2=0x7fdfc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:17.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.258+0000 7fdff1200640 1 --2- 172.21.15.196:0/1955303755 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdfc806c8f0 0x7fdfc806edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc006fd0 tx=0x7fdfdc008040 comp rx=0 tx=0).stop 2024-04-28T13:32:17.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.258+0000 7fdff1200640 1 -- 172.21.15.196:0/1955303755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfec163b40 msgr2=0x7fdfec1f90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:17.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.258+0000 7fdff1200640 1 --2- 172.21.15.196:0/1955303755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfec163b40 0x7fdfec1f90d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4034040 tx=0x7fdfe4004a40 comp rx=0 tx=0).stop 2024-04-28T13:32:17.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.258+0000 7fdff1200640 1 -- 172.21.15.196:0/1955303755 shutdown_connections 2024-04-28T13:32:17.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.258+0000 7fdff1200640 1 --2- 172.21.15.196:0/1955303755 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdfc806c8f0 0x7fdfc806edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.258+0000 7fdff1200640 1 --2- 172.21.15.196:0/1955303755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfec163b40 0x7fdfec1f90d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.258+0000 7fdff1200640 1 --2- 172.21.15.196:0/1955303755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfec0b65f0 0x7fdfec15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.258+0000 7fdff1200640 1 --2- 172.21.15.196:0/1955303755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfec06fa50 0x7fdfec15cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.258+0000 7fdff1200640 1 -- 172.21.15.196:0/1955303755 >> 172.21.15.196:0/1955303755 conn(0x7fdfec0cd8a0 msgr2=0x7fdfec0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:17.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.259+0000 7fdff1200640 1 -- 172.21.15.196:0/1955303755 shutdown_connections 2024-04-28T13:32:17.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.259+0000 7fdff1200640 1 -- 172.21.15.196:0/1955303755 wait complete. 2024-04-28T13:32:17.278 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:32:17.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.478+0000 7fe456600640 1 Processor -- start 2024-04-28T13:32:17.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.478+0000 7fe456600640 1 -- start start 2024-04-28T13:32:17.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.480+0000 7fe456600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe4501629c0 0x7fe4500d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.480+0000 7fe456600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe450163bf0 0x7fe450163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.480+0000 7fe456600640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe450162da0 con 0x7fe450163570 2024-04-28T13:32:17.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.480+0000 7fe456600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe4500b6190 con 0x7fe450163bf0 2024-04-28T13:32:17.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.480+0000 7fe456600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fe4500b6470 con 0x7fe4501629c0 2024-04-28T13:32:17.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.480+0000 7fe44fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe450163bf0 0x7fe450163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.480+0000 7fe44fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe450163bf0 0x7fe450163030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49616/0 (socket says 172.21.15.196:49616) 2024-04-28T13:32:17.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.480+0000 7fe44fe00640 1 -- 172.21.15.196:0/2620053231 learned_addr learned my addr 172.21.15.196:0/2620053231 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:17.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.480+0000 7fe455200640 1 --2- 172.21.15.196:0/2620053231 >> v2:172.21.15.203:3300/0 conn(0x7fe4501629c0 0x7fe4500d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.481+0000 7fe44fe00640 1 -- 172.21.15.196:0/2620053231 >> v2:172.21.15.203:3300/0 conn(0x7fe4501629c0 msgr2=0x7fe4500d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:17.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.481+0000 7fe44fe00640 1 --2- 172.21.15.196:0/2620053231 >> v2:172.21.15.203:3300/0 conn(0x7fe4501629c0 0x7fe4500d3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.481+0000 7fe44fe00640 1 -- 172.21.15.196:0/2620053231 >> v1:172.21.15.203:6789/0 conn(0x7fe450163570 legacy=0x7fe4500b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:17.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.481+0000 7fe44fe00640 1 -- 172.21.15.196:0/2620053231 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4500b6750 con 0x7fe450163bf0 2024-04-28T13:32:17.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.482+0000 7fe44fe00640 1 --2- 172.21.15.196:0/2620053231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe450163bf0 0x7fe450163030 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe45008f8a0 tx=0x7fe44002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2a62c9e0d0a821 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:17.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.482+0000 7fe44f400640 1 -- 172.21.15.196:0/2620053231 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440037070 con 0x7fe450163bf0 2024-04-28T13:32:17.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.482+0000 7fe44f400640 1 -- 172.21.15.196:0/2620053231 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe440032440 con 0x7fe450163bf0 2024-04-28T13:32:17.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.482+0000 7fe44f400640 1 -- 172.21.15.196:0/2620053231 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440032810 con 0x7fe450163bf0 2024-04-28T13:32:17.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.483+0000 7fe456600640 1 -- 172.21.15.196:0/2620053231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe450163bf0 msgr2=0x7fe450163030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:17.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.483+0000 7fe456600640 1 --2- 172.21.15.196:0/2620053231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe450163bf0 0x7fe450163030 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe45008f8a0 tx=0x7fe44002a230 comp rx=0 tx=0).stop 2024-04-28T13:32:17.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.483+0000 7fe456600640 1 -- 172.21.15.196:0/2620053231 shutdown_connections 2024-04-28T13:32:17.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.483+0000 7fe456600640 1 --2- 172.21.15.196:0/2620053231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe450163bf0 0x7fe450163030 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.483+0000 7fe456600640 1 --2- 172.21.15.196:0/2620053231 >> v2:172.21.15.203:3300/0 conn(0x7fe4501629c0 0x7fe4500d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.483+0000 7fe456600640 1 -- 172.21.15.196:0/2620053231 >> 172.21.15.196:0/2620053231 conn(0x7fe45009e570 msgr2=0x7fe4500d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:17.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.483+0000 7fe456600640 1 -- 172.21.15.196:0/2620053231 shutdown_connections 2024-04-28T13:32:17.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.483+0000 7fe456600640 1 -- 172.21.15.196:0/2620053231 wait complete. 2024-04-28T13:32:17.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.485+0000 7fe456600640 1 Processor -- start 2024-04-28T13:32:17.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.485+0000 7fe456600640 1 -- start start 2024-04-28T13:32:17.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.486+0000 7fe456600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4501629c0 0x7fe450151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.486+0000 7fe456600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe450163570 0x7fe4500c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.486+0000 7fe456600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe450163bf0 0x7fe4500d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.486+0000 7fe456600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe4500d3c50 con 0x7fe450163570 2024-04-28T13:32:17.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.486+0000 7fe456600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe4500b1e60 con 0x7fe450163bf0 2024-04-28T13:32:17.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.486+0000 7fe456600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe4500d3dd0 con 0x7fe4501629c0 2024-04-28T13:32:17.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.486+0000 7fe44fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe450163570 0x7fe4500c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.486+0000 7fe44fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe450163570 0x7fe4500c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49628/0 (socket says 172.21.15.196:49628) 2024-04-28T13:32:17.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.486+0000 7fe44fe00640 1 -- 172.21.15.196:0/524546272 learned_addr learned my addr 172.21.15.196:0/524546272 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:17.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.486+0000 7fe455200640 1 --2- 172.21.15.196:0/524546272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4501629c0 0x7fe450151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.486+0000 7fe455c00640 1 --2- 172.21.15.196:0/524546272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe450163bf0 0x7fe4500d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.486+0000 7fe44fe00640 1 -- 172.21.15.196:0/524546272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4501629c0 msgr2=0x7fe450151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:17.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.487+0000 7fe44fe00640 1 --2- 172.21.15.196:0/524546272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4501629c0 0x7fe450151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.487+0000 7fe44fe00640 1 -- 172.21.15.196:0/524546272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe450163bf0 msgr2=0x7fe4500d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:17.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.487+0000 7fe44fe00640 1 --2- 172.21.15.196:0/524546272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe450163bf0 0x7fe4500d1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.487+0000 7fe44fe00640 1 -- 172.21.15.196:0/524546272 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe450151bd0 con 0x7fe450163570 2024-04-28T13:32:17.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.487+0000 7fe455c00640 1 --2- 172.21.15.196:0/524546272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe450163bf0 0x7fe4500d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:17.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.487+0000 7fe44fe00640 1 --2- 172.21.15.196:0/524546272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe450163570 0x7fe4500c9700 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe44002a270 tx=0x7fe4400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:17.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.487+0000 7fe44cc00640 1 -- 172.21.15.196:0/524546272 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440041020 con 0x7fe450163570 2024-04-28T13:32:17.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.487+0000 7fe44cc00640 1 -- 172.21.15.196:0/524546272 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4400306c0 con 0x7fe450163570 2024-04-28T13:32:17.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.487+0000 7fe44cc00640 1 -- 172.21.15.196:0/524546272 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440037040 con 0x7fe450163570 2024-04-28T13:32:17.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.487+0000 7fe456600640 1 -- 172.21.15.196:0/524546272 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4500ca0a0 con 0x7fe450163570 2024-04-28T13:32:17.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.487+0000 7fe456600640 1 -- 172.21.15.196:0/524546272 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4500ca660 con 0x7fe450163570 2024-04-28T13:32:17.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.488+0000 7fe456600640 1 -- 172.21.15.196:0/524546272 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45006a290 con 0x7fe450163570 2024-04-28T13:32:17.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.489+0000 7fe44cc00640 1 -- 172.21.15.196:0/524546272 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe440044020 con 0x7fe450163570 2024-04-28T13:32:17.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.491+0000 7fe44cc00640 1 --2- 172.21.15.196:0/524546272 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe43406c8f0 0x7fe43406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.492+0000 7fe44cc00640 1 -- 172.21.15.196:0/524546272 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe4400ac440 con 0x7fe450163570 2024-04-28T13:32:17.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.492+0000 7fe455200640 1 --2- 172.21.15.196:0/524546272 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe43406c8f0 0x7fe43406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.494+0000 7fe455200640 1 --2- 172.21.15.196:0/524546272 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe43406c8f0 0x7fe43406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe444006fd0 tx=0x7fe444008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:17.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.496+0000 7fe44cc00640 1 -- 172.21.15.196:0/524546272 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe44004b030 con 0x7fe450163570 2024-04-28T13:32:17.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.746+0000 7fe456600640 1 -- 172.21.15.196:0/524546272 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4500c89f0 con 0x7fe450163570 2024-04-28T13:32:17.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.747+0000 7fe44cc00640 1 -- 172.21.15.196:0/524546272 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7fe450151bd0 con 0x7fe450163570 2024-04-28T13:32:17.748 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:17.749 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/422653807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":422653807},{"type":"v1","addr":"172.21.15.196:6834","nonce":422653807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2793552376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2793552376},{"type":"v1","addr":"172.21.15.196:6835","nonce":2793552376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1810956193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1810956193},{"type":"v1","addr":"172.21.15.203:6837","nonce":1810956193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:31:21.059586+0000","modified":"2024-04-28T13:31:24.132134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/2114798853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2114798853},{"type":"v1","addr":"172.21.15.203:6836","nonce":2114798853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4253,"qdb_cluster":[4253]},"id":1}]} 2024-04-28T13:32:17.749 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 7 2024-04-28T13:32:17.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.750+0000 7fe456600640 1 -- 172.21.15.196:0/524546272 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe43406c8f0 msgr2=0x7fe43406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:17.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.750+0000 7fe456600640 1 --2- 172.21.15.196:0/524546272 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe43406c8f0 0x7fe43406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe444006fd0 tx=0x7fe444008040 comp rx=0 tx=0).stop 2024-04-28T13:32:17.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.750+0000 7fe456600640 1 -- 172.21.15.196:0/524546272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe450163570 msgr2=0x7fe4500c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:17.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.750+0000 7fe456600640 1 --2- 172.21.15.196:0/524546272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe450163570 0x7fe4500c9700 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe44002a270 tx=0x7fe4400046c0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.750+0000 7fe456600640 1 -- 172.21.15.196:0/524546272 shutdown_connections 2024-04-28T13:32:17.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.750+0000 7fe456600640 1 --2- 172.21.15.196:0/524546272 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe43406c8f0 0x7fe43406edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.750+0000 7fe456600640 1 --2- 172.21.15.196:0/524546272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe450163bf0 0x7fe4500d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.750+0000 7fe456600640 1 --2- 172.21.15.196:0/524546272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe450163570 0x7fe4500c9700 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.750+0000 7fe456600640 1 --2- 172.21.15.196:0/524546272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4501629c0 0x7fe450151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.750+0000 7fe456600640 1 -- 172.21.15.196:0/524546272 >> 172.21.15.196:0/524546272 conn(0x7fe45009e570 msgr2=0x7fe4500b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:17.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.750+0000 7fe456600640 1 -- 172.21.15.196:0/524546272 shutdown_connections 2024-04-28T13:32:17.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.750+0000 7fe456600640 1 -- 172.21.15.196:0/524546272 wait complete. 2024-04-28T13:32:17.769 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:32:17.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.970+0000 7f590d800640 1 Processor -- start 2024-04-28T13:32:17.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.971+0000 7f590d800640 1 -- start start 2024-04-28T13:32:17.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.972+0000 7f590d800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5908163740 0x7f590816a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.972+0000 7f590d800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5908162da0 con 0x7f5908163bf0 2024-04-28T13:32:17.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.972+0000 7f590d800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59080b6190 con 0x7f59081629c0 2024-04-28T13:32:17.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.972+0000 7f590d800640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f59080d3e20 con 0x7f5908163740 2024-04-28T13:32:17.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.973+0000 7f5907400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f5908163bf0 0x7f5908163030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:44348/0 (socket says 172.21.15.196:44348) 2024-04-28T13:32:17.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.973+0000 7f5907400640 1 -- 172.21.15.196:0/3734796226 learned_addr learned my addr 172.21.15.196:0/3734796226 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:17.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.973+0000 7f590ce00640 1 --2- 172.21.15.196:0/3734796226 >> v2:172.21.15.203:3300/0 conn(0x7f5908163740 0x7f590816a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.973+0000 7f5906a00640 1 -- 172.21.15.196:0/3734796226 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4235773509 0 0) 0x7f5908162da0 con 0x7f5908163bf0 2024-04-28T13:32:17.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.973+0000 7f5906a00640 1 -- 172.21.15.196:0/3734796226 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58e8003520 con 0x7f5908163bf0 2024-04-28T13:32:17.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.973+0000 7f5906a00640 1 -- 172.21.15.196:0/3734796226 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 351836889 0 0) 0x7f58e8003520 con 0x7f5908163bf0 2024-04-28T13:32:17.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.973+0000 7f5906a00640 1 -- 172.21.15.196:0/3734796226 >> v2:172.21.15.203:3300/0 conn(0x7f5908163740 msgr2=0x7f590816a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:17.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.973+0000 7f5906a00640 1 --2- 172.21.15.196:0/3734796226 >> v2:172.21.15.203:3300/0 conn(0x7f5908163740 0x7f590816a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.973+0000 7f5906a00640 1 -- 172.21.15.196:0/3734796226 >> v1:172.21.15.203:6789/0 conn(0x7f59081629c0 legacy=0x7f59080d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:17.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.973+0000 7f5906a00640 1 -- 172.21.15.196:0/3734796226 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59080b6470 con 0x7f5908163bf0 2024-04-28T13:32:17.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.974+0000 7f5906a00640 1 -- 172.21.15.196:0/3734796226 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f58f0001670 con 0x7f5908163bf0 2024-04-28T13:32:17.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.974+0000 7f5906a00640 1 -- 172.21.15.196:0/3734796226 >> v1:172.21.15.196:6789/0 conn(0x7f5908163bf0 legacy=0x7f5908163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:17.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.974+0000 7f5906a00640 1 --2- 172.21.15.196:0/3734796226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e80039b0 0x7f58e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.974+0000 7f5906a00640 1 --2- 172.21.15.196:0/3734796226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58e8004800 0x7f58e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.974+0000 7f5906a00640 1 --2- 172.21.15.196:0/3734796226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58e80051c0 0x7f58e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.974+0000 7f5906a00640 1 -- 172.21.15.196:0/3734796226 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59080d3e20 con 0x7f58e80039b0 2024-04-28T13:32:17.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.974+0000 7f5906a00640 1 -- 172.21.15.196:0/3734796226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f58e800c060 con 0x7f58e80051c0 2024-04-28T13:32:17.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.974+0000 7f5906a00640 1 -- 172.21.15.196:0/3734796226 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f58e800c260 con 0x7f58e8004800 2024-04-28T13:32:17.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.974+0000 7f5907e00640 1 --2- 172.21.15.196:0/3734796226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e80039b0 0x7f58e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.974+0000 7f5907e00640 1 -- 172.21.15.196:0/3734796226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58e8004800 msgr2=0x7f58e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:17.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.974+0000 7f5907e00640 1 --2- 172.21.15.196:0/3734796226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58e8004800 0x7f58e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.974+0000 7f5907e00640 1 -- 172.21.15.196:0/3734796226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58e80051c0 msgr2=0x7f58e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:17.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.974+0000 7f5907e00640 1 --2- 172.21.15.196:0/3734796226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58e80051c0 0x7f58e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.974+0000 7f5907e00640 1 -- 172.21.15.196:0/3734796226 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e8003520 con 0x7f58e80039b0 2024-04-28T13:32:17.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.974+0000 7f590ce00640 1 -- 172.21.15.196:0/3734796226 reap_dead start 2024-04-28T13:32:17.978 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.975+0000 7f5907e00640 1 --2- 172.21.15.196:0/3734796226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e80039b0 0x7f58e8003dd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f58f8001750 tx=0x7f58e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeec1be06287b79b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:17.978 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.976+0000 7f5906a00640 1 -- 172.21.15.196:0/3734796226 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8001370 con 0x7f58e80039b0 2024-04-28T13:32:17.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.976+0000 7f5906a00640 1 -- 172.21.15.196:0/3734796226 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f80052d0 con 0x7f58e80039b0 2024-04-28T13:32:17.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.976+0000 7f5906a00640 1 -- 172.21.15.196:0/3734796226 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f802abc0 con 0x7f58e80039b0 2024-04-28T13:32:17.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.976+0000 7f590d800640 1 -- 172.21.15.196:0/3734796226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e80039b0 msgr2=0x7f58e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:17.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.976+0000 7f590d800640 1 --2- 172.21.15.196:0/3734796226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e80039b0 0x7f58e8003dd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f58f8001750 tx=0x7f58e8005830 comp rx=0 tx=0).stop 2024-04-28T13:32:17.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.976+0000 7f590d800640 1 -- 172.21.15.196:0/3734796226 shutdown_connections 2024-04-28T13:32:17.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.976+0000 7f590d800640 1 --2- 172.21.15.196:0/3734796226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e80039b0 0x7f58e8003dd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.976+0000 7f590d800640 1 -- 172.21.15.196:0/3734796226 >> 172.21.15.196:0/3734796226 conn(0x7f590809e570 msgr2=0x7f59080d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:17.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.976+0000 7f590d800640 1 -- 172.21.15.196:0/3734796226 shutdown_connections 2024-04-28T13:32:17.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.976+0000 7f590d800640 1 -- 172.21.15.196:0/3734796226 wait complete. 2024-04-28T13:32:17.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.978+0000 7f590d800640 1 Processor -- start 2024-04-28T13:32:17.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.978+0000 7f590d800640 1 -- start start 2024-04-28T13:32:17.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.979+0000 7f590d800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e80039b0 0x7f590814d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.979+0000 7f590d800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59080b7fa0 0x7f59080d0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.979+0000 7f590d800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59080d2250 0x7f590815c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.979+0000 7f590d800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59080b1e60 con 0x7f58e80039b0 2024-04-28T13:32:17.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.979+0000 7f590d800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f59080b6a30 con 0x7f59080b7fa0 2024-04-28T13:32:17.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.979+0000 7f590d800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f59080d3710 con 0x7f59080d2250 2024-04-28T13:32:17.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.980+0000 7f5907e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e80039b0 0x7f590814d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.980+0000 7f5907e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e80039b0 0x7f590814d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49642/0 (socket says 172.21.15.196:49642) 2024-04-28T13:32:17.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.980+0000 7f5907e00640 1 -- 172.21.15.196:0/741835995 learned_addr learned my addr 172.21.15.196:0/741835995 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:17.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.980+0000 7f590ce00640 1 --2- 172.21.15.196:0/741835995 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59080d2250 0x7f590815c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.980+0000 7f5907400640 1 --2- 172.21.15.196:0/741835995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59080b7fa0 0x7f59080d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.980+0000 7f5907e00640 1 -- 172.21.15.196:0/741835995 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59080d2250 msgr2=0x7f590815c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:17.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.980+0000 7f5907e00640 1 --2- 172.21.15.196:0/741835995 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59080d2250 0x7f590815c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.980+0000 7f5907e00640 1 -- 172.21.15.196:0/741835995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59080b7fa0 msgr2=0x7f59080d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:17.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.980+0000 7f5907e00640 1 --2- 172.21.15.196:0/741835995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59080b7fa0 0x7f59080d0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:17.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.980+0000 7f5907e00640 1 -- 172.21.15.196:0/741835995 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590814db90 con 0x7f58e80039b0 2024-04-28T13:32:17.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.981+0000 7f5907e00640 1 --2- 172.21.15.196:0/741835995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e80039b0 0x7f590814d650 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f58f8000c00 tx=0x7f58f8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:17.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.983+0000 7f58ffe00640 1 -- 172.21.15.196:0/741835995 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f803f030 con 0x7f58e80039b0 2024-04-28T13:32:17.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.983+0000 7f58ffe00640 1 -- 172.21.15.196:0/741835995 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f8005480 con 0x7f58e80039b0 2024-04-28T13:32:17.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.983+0000 7f58ffe00640 1 -- 172.21.15.196:0/741835995 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8037040 con 0x7f58e80039b0 2024-04-28T13:32:17.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.983+0000 7f590d800640 1 -- 172.21.15.196:0/741835995 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59080d11f0 con 0x7f58e80039b0 2024-04-28T13:32:17.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.983+0000 7f590d800640 1 -- 172.21.15.196:0/741835995 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59080d1770 con 0x7f58e80039b0 2024-04-28T13:32:17.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.984+0000 7f590d800640 1 -- 172.21.15.196:0/741835995 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f590806a290 con 0x7f58e80039b0 2024-04-28T13:32:17.985 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.985+0000 7f58ffe00640 1 -- 172.21.15.196:0/741835995 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58f8042070 con 0x7f58e80039b0 2024-04-28T13:32:17.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.986+0000 7f58ffe00640 1 --2- 172.21.15.196:0/741835995 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f58e406c940 0x7f58e406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:17.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.986+0000 7f58ffe00640 1 -- 172.21.15.196:0/741835995 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f58f80ab250 con 0x7f58e80039b0 2024-04-28T13:32:17.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.987+0000 7f5907400640 1 --2- 172.21.15.196:0/741835995 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f58e406c940 0x7f58e406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:17.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.988+0000 7f5907400640 1 --2- 172.21.15.196:0/741835995 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f58e406c940 0x7f58e406ee00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58f0001eb0 tx=0x7f58f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:17.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:17.990+0000 7f58ffe00640 1 -- 172.21.15.196:0/741835995 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f590814db90 con 0x7f58e80039b0 2024-04-28T13:32:18.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.194+0000 7f590d800640 1 -- 172.21.15.196:0/741835995 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f590806d1b0 con 0x7f58e80039b0 2024-04-28T13:32:18.198 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.197+0000 7f58ffe00640 1 -- 172.21.15.196:0/741835995 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16890 (secure 0 0 0) 0x7f58f8079550 con 0x7f58e80039b0 2024-04-28T13:32:18.198 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:18.199 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:03.882452+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:32:18.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.200+0000 7f590d800640 1 -- 172.21.15.196:0/741835995 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f58e406c940 msgr2=0x7f58e406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:18.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.200+0000 7f590d800640 1 --2- 172.21.15.196:0/741835995 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f58e406c940 0x7f58e406ee00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58f0001eb0 tx=0x7f58f0009040 comp rx=0 tx=0).stop 2024-04-28T13:32:18.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.200+0000 7f590d800640 1 -- 172.21.15.196:0/741835995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e80039b0 msgr2=0x7f590814d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:18.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.200+0000 7f590d800640 1 --2- 172.21.15.196:0/741835995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e80039b0 0x7f590814d650 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f58f8000c00 tx=0x7f58f8001120 comp rx=0 tx=0).stop 2024-04-28T13:32:18.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.200+0000 7f590d800640 1 -- 172.21.15.196:0/741835995 shutdown_connections 2024-04-28T13:32:18.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.200+0000 7f590d800640 1 --2- 172.21.15.196:0/741835995 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f58e406c940 0x7f58e406ee00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.200+0000 7f590d800640 1 --2- 172.21.15.196:0/741835995 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59080d2250 0x7f590815c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.200+0000 7f590d800640 1 --2- 172.21.15.196:0/741835995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59080b7fa0 0x7f59080d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.200+0000 7f590d800640 1 --2- 172.21.15.196:0/741835995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e80039b0 0x7f590814d650 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.200+0000 7f590d800640 1 -- 172.21.15.196:0/741835995 >> 172.21.15.196:0/741835995 conn(0x7f590809e570 msgr2=0x7f590806da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:18.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.200+0000 7f590d800640 1 -- 172.21.15.196:0/741835995 shutdown_connections 2024-04-28T13:32:18.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.200+0000 7f590d800640 1 -- 172.21.15.196:0/741835995 wait complete. 2024-04-28T13:32:18.219 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-28T13:32:18.219 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:32:18.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.414+0000 7f5379600640 1 Processor -- start 2024-04-28T13:32:18.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.415+0000 7f5379600640 1 -- start start 2024-04-28T13:32:18.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.417+0000 7f5379600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f53740d1190 0x7f53740b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:18.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.417+0000 7f5379600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5374160f40 con 0x7f53740d0d00 2024-04-28T13:32:18.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.417+0000 7f5379600640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5374161220 con 0x7f53741469f0 2024-04-28T13:32:18.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.417+0000 7f5379600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f53740b5960 con 0x7f53740d1190 2024-04-28T13:32:18.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.417+0000 7f536a000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f53740d0d00 0x7f53740b5dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:44352/0 (socket says 172.21.15.196:44352) 2024-04-28T13:32:18.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.417+0000 7f536a000640 1 -- 172.21.15.196:0/2220293645 learned_addr learned my addr 172.21.15.196:0/2220293645 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:18.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.417+0000 7f5373400640 1 --2- 172.21.15.196:0/2220293645 >> v2:172.21.15.203:3300/0 conn(0x7f53740d1190 0x7f53740b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:18.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372000640 1 -- 172.21.15.196:0/2220293645 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2874449630 0 0) 0x7f5374160f40 con 0x7f53740d0d00 2024-04-28T13:32:18.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372000640 1 -- 172.21.15.196:0/2220293645 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5358003520 con 0x7f53740d0d00 2024-04-28T13:32:18.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372000640 1 -- 172.21.15.196:0/2220293645 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1619733650 0 0) 0x7f5358003520 con 0x7f53740d0d00 2024-04-28T13:32:18.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372000640 1 -- 172.21.15.196:0/2220293645 >> v2:172.21.15.203:3300/0 conn(0x7f53740d1190 msgr2=0x7f53740b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:18.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372000640 1 --2- 172.21.15.196:0/2220293645 >> v2:172.21.15.203:3300/0 conn(0x7f53740d1190 0x7f53740b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372000640 1 -- 172.21.15.196:0/2220293645 >> v1:172.21.15.203:6789/0 conn(0x7f53741469f0 legacy=0x7f53740ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:18.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372000640 1 -- 172.21.15.196:0/2220293645 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5374161500 con 0x7f53740d0d00 2024-04-28T13:32:18.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372000640 1 -- 172.21.15.196:0/2220293645 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f535c001670 con 0x7f53740d0d00 2024-04-28T13:32:18.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372000640 1 -- 172.21.15.196:0/2220293645 >> v1:172.21.15.196:6789/0 conn(0x7f53740d0d00 legacy=0x7f53740b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:18.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372000640 1 --2- 172.21.15.196:0/2220293645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53580039b0 0x7f5358003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:18.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372000640 1 --2- 172.21.15.196:0/2220293645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5358004800 0x7f5358004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:18.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372000640 1 --2- 172.21.15.196:0/2220293645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53580051c0 0x7f535800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:18.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372000640 1 -- 172.21.15.196:0/2220293645 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f535800c030 con 0x7f53580039b0 2024-04-28T13:32:18.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372000640 1 -- 172.21.15.196:0/2220293645 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f535800c1e0 con 0x7f5358004800 2024-04-28T13:32:18.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372000640 1 -- 172.21.15.196:0/2220293645 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f535800c3e0 con 0x7f53580051c0 2024-04-28T13:32:18.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.418+0000 7f5372a00640 1 --2- 172.21.15.196:0/2220293645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53580039b0 0x7f5358003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:18.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.419+0000 7f536a000640 1 --2- 172.21.15.196:0/2220293645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5358004800 0x7f5358004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:18.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.419+0000 7f5373400640 1 --2- 172.21.15.196:0/2220293645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53580051c0 0x7f535800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:18.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.419+0000 7f5372a00640 1 -- 172.21.15.196:0/2220293645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53580051c0 msgr2=0x7f535800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:18.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.419+0000 7f5372a00640 1 --2- 172.21.15.196:0/2220293645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53580051c0 0x7f535800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.419+0000 7f5372a00640 1 -- 172.21.15.196:0/2220293645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5358004800 msgr2=0x7f5358004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:18.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.419+0000 7f5372a00640 1 --2- 172.21.15.196:0/2220293645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5358004800 0x7f5358004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.419+0000 7f5372a00640 1 -- 172.21.15.196:0/2220293645 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5358003520 con 0x7f53580039b0 2024-04-28T13:32:18.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.419+0000 7f5373400640 1 -- 172.21.15.196:0/2220293645 reap_dead start 2024-04-28T13:32:18.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.420+0000 7f5372a00640 1 --2- 172.21.15.196:0/2220293645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53580039b0 0x7f5358003dd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5364001750 tx=0x7f5358005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c4945978c0cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:18.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.420+0000 7f5372000640 1 -- 172.21.15.196:0/2220293645 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364001370 con 0x7f53580039b0 2024-04-28T13:32:18.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.420+0000 7f5372000640 1 -- 172.21.15.196:0/2220293645 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53640052d0 con 0x7f53580039b0 2024-04-28T13:32:18.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.420+0000 7f5372000640 1 -- 172.21.15.196:0/2220293645 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536402abe0 con 0x7f53580039b0 2024-04-28T13:32:18.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.421+0000 7f5379600640 1 -- 172.21.15.196:0/2220293645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53580039b0 msgr2=0x7f5358003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:18.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.421+0000 7f5379600640 1 --2- 172.21.15.196:0/2220293645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53580039b0 0x7f5358003dd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5364001750 tx=0x7f5358005830 comp rx=0 tx=0).stop 2024-04-28T13:32:18.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.421+0000 7f5379600640 1 -- 172.21.15.196:0/2220293645 shutdown_connections 2024-04-28T13:32:18.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.421+0000 7f5379600640 1 --2- 172.21.15.196:0/2220293645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53580039b0 0x7f5358003dd0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.421+0000 7f5379600640 1 -- 172.21.15.196:0/2220293645 >> 172.21.15.196:0/2220293645 conn(0x7f53740cd830 msgr2=0x7f53740cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:18.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.421+0000 7f5379600640 1 -- 172.21.15.196:0/2220293645 shutdown_connections 2024-04-28T13:32:18.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.421+0000 7f5379600640 1 -- 172.21.15.196:0/2220293645 wait complete. 2024-04-28T13:32:18.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.423+0000 7f5379600640 1 Processor -- start 2024-04-28T13:32:18.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.423+0000 7f5379600640 1 -- start start 2024-04-28T13:32:18.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.424+0000 7f5379600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53580039b0 0x7f5374145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:18.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.424+0000 7f5379600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f537415abc0 0x7f5374159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:18.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.424+0000 7f5379600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53740b0af0 0x7f53740b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:18.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.424+0000 7f5379600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5374072020 con 0x7f537415abc0 2024-04-28T13:32:18.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.424+0000 7f5379600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5374161ac0 con 0x7f53580039b0 2024-04-28T13:32:18.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.424+0000 7f5379600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f53740d15c0 con 0x7f53740b0af0 2024-04-28T13:32:18.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.424+0000 7f536a000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f537415abc0 0x7f5374159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:18.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.424+0000 7f536a000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f537415abc0 0x7f5374159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49670/0 (socket says 172.21.15.196:49670) 2024-04-28T13:32:18.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.424+0000 7f536a000640 1 -- 172.21.15.196:0/3669131453 learned_addr learned my addr 172.21.15.196:0/3669131453 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:18.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.425+0000 7f5372a00640 1 --2- 172.21.15.196:0/3669131453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53580039b0 0x7f5374145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:18.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.425+0000 7f5373400640 1 --2- 172.21.15.196:0/3669131453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53740b0af0 0x7f53740b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:18.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.425+0000 7f536a000640 1 -- 172.21.15.196:0/3669131453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53740b0af0 msgr2=0x7f53740b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:18.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.425+0000 7f536a000640 1 --2- 172.21.15.196:0/3669131453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53740b0af0 0x7f53740b0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.425+0000 7f536a000640 1 -- 172.21.15.196:0/3669131453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53580039b0 msgr2=0x7f5374145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:18.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.425+0000 7f536a000640 1 --2- 172.21.15.196:0/3669131453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53580039b0 0x7f5374145770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.425+0000 7f536a000640 1 -- 172.21.15.196:0/3669131453 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53740ceac0 con 0x7f537415abc0 2024-04-28T13:32:18.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.425+0000 7f5372a00640 1 --2- 172.21.15.196:0/3669131453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53580039b0 0x7f5374145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:18.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.425+0000 7f536a000640 1 --2- 172.21.15.196:0/3669131453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f537415abc0 0x7f5374159d30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f535c00e760 tx=0x7f535c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:18.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.425+0000 7f536b400640 1 -- 172.21.15.196:0/3669131453 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c00ee70 con 0x7f537415abc0 2024-04-28T13:32:18.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.425+0000 7f536b400640 1 -- 172.21.15.196:0/3669131453 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535c018070 con 0x7f537415abc0 2024-04-28T13:32:18.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.425+0000 7f536b400640 1 -- 172.21.15.196:0/3669131453 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c00cab0 con 0x7f537415abc0 2024-04-28T13:32:18.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.425+0000 7f5379600640 1 -- 172.21.15.196:0/3669131453 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53740ceda0 con 0x7f537415abc0 2024-04-28T13:32:18.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.426+0000 7f5379600640 1 -- 172.21.15.196:0/3669131453 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53740cf360 con 0x7f537415abc0 2024-04-28T13:32:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.427+0000 7f5379600640 1 -- 172.21.15.196:0/3669131453 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537406a290 con 0x7f537415abc0 2024-04-28T13:32:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.428+0000 7f536b400640 1 -- 172.21.15.196:0/3669131453 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f535c00cc50 con 0x7f537415abc0 2024-04-28T13:32:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.429+0000 7f536b400640 1 --2- 172.21.15.196:0/3669131453 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f535006c920 0x7f535006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:18.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.429+0000 7f536b400640 1 -- 172.21.15.196:0/3669131453 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f535c08bb80 con 0x7f537415abc0 2024-04-28T13:32:18.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.429+0000 7f5372a00640 1 --2- 172.21.15.196:0/3669131453 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f535006c920 0x7f535006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:18.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.430+0000 7f5372a00640 1 --2- 172.21.15.196:0/3669131453 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f535006c920 0x7f535006ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5364032000 tx=0x7f536402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:18.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.433+0000 7f536b400640 1 -- 172.21.15.196:0/3669131453 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f535c059e00 con 0x7f537415abc0 2024-04-28T13:32:18.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.671+0000 7f5379600640 1 -- 172.21.15.196:0/3669131453 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f537406fd90 con 0x7f537415abc0 2024-04-28T13:32:18.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.672+0000 7f536b400640 1 -- 172.21.15.196:0/3669131453 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f535c059700 con 0x7f537415abc0 2024-04-28T13:32:18.674 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:18.674 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/422653807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":422653807},{"type":"v1","addr":"172.21.15.196:6834","nonce":422653807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2793552376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2793552376},{"type":"v1","addr":"172.21.15.196:6835","nonce":2793552376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1810956193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1810956193},{"type":"v1","addr":"172.21.15.203:6837","nonce":1810956193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:31:21.059586+0000","modified":"2024-04-28T13:31:24.132134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/2114798853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2114798853},{"type":"v1","addr":"172.21.15.203:6836","nonce":2114798853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4253,"qdb_cluster":[4253]},"id":1}]} 2024-04-28T13:32:18.674 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 7 2024-04-28T13:32:18.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.675+0000 7f5379600640 1 -- 172.21.15.196:0/3669131453 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f535006c920 msgr2=0x7f535006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:18.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.675+0000 7f5379600640 1 --2- 172.21.15.196:0/3669131453 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f535006c920 0x7f535006ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5364032000 tx=0x7f536402e000 comp rx=0 tx=0).stop 2024-04-28T13:32:18.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.675+0000 7f5379600640 1 -- 172.21.15.196:0/3669131453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f537415abc0 msgr2=0x7f5374159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:18.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.675+0000 7f5379600640 1 --2- 172.21.15.196:0/3669131453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f537415abc0 0x7f5374159d30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f535c00e760 tx=0x7f535c00ec20 comp rx=0 tx=0).stop 2024-04-28T13:32:18.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.675+0000 7f5379600640 1 -- 172.21.15.196:0/3669131453 shutdown_connections 2024-04-28T13:32:18.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.675+0000 7f5379600640 1 --2- 172.21.15.196:0/3669131453 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f535006c920 0x7f535006ede0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.675+0000 7f5379600640 1 --2- 172.21.15.196:0/3669131453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53740b0af0 0x7f53740b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.675+0000 7f5379600640 1 --2- 172.21.15.196:0/3669131453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f537415abc0 0x7f5374159d30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.675+0000 7f5379600640 1 --2- 172.21.15.196:0/3669131453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53580039b0 0x7f5374145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.675+0000 7f5379600640 1 -- 172.21.15.196:0/3669131453 >> 172.21.15.196:0/3669131453 conn(0x7f53740cd830 msgr2=0x7f53740cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:18.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.675+0000 7f5379600640 1 -- 172.21.15.196:0/3669131453 shutdown_connections 2024-04-28T13:32:18.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.675+0000 7f5379600640 1 -- 172.21.15.196:0/3669131453 wait complete. 2024-04-28T13:32:18.696 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:32:18.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.897+0000 7f5473e00640 1 Processor -- start 2024-04-28T13:32:18.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.897+0000 7f5473e00640 1 -- start start 2024-04-28T13:32:18.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.899+0000 7f5473e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f546c1469f0 0x7f546c0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:18.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.899+0000 7f5473e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f546c0cd420 0x7f546c0d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:18.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.899+0000 7f5473e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f546c0d1240 0x7f546c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:18.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.899+0000 7f5473e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f546c0b5960 con 0x7f546c0d1240 2024-04-28T13:32:18.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.899+0000 7f5473e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f546c161ac0 con 0x7f546c0cd420 2024-04-28T13:32:18.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.899+0000 7f5473e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f546c160f40 con 0x7f546c1469f0 2024-04-28T13:32:18.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.899+0000 7f5471e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f546c0d1240 0x7f546c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:18.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.899+0000 7f5471e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f546c0d1240 0x7f546c06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49686/0 (socket says 172.21.15.196:49686) 2024-04-28T13:32:18.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.899+0000 7f5470a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f546c0cd420 0x7f546c0d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:18.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.899+0000 7f5471e00640 1 -- 172.21.15.196:0/720985718 learned_addr learned my addr 172.21.15.196:0/720985718 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:18.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.899+0000 7f5471400640 1 --2- 172.21.15.196:0/720985718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f546c1469f0 0x7f546c0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:18.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.900+0000 7f5471e00640 1 -- 172.21.15.196:0/720985718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f546c1469f0 msgr2=0x7f546c0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:18.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.900+0000 7f5471e00640 1 --2- 172.21.15.196:0/720985718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f546c1469f0 0x7f546c0ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.900+0000 7f5471e00640 1 -- 172.21.15.196:0/720985718 >> v2:172.21.15.203:3300/0 conn(0x7f546c0cd420 msgr2=0x7f546c0d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:18.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.900+0000 7f5471e00640 1 --2- 172.21.15.196:0/720985718 >> v2:172.21.15.203:3300/0 conn(0x7f546c0cd420 0x7f546c0d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.900+0000 7f5471e00640 1 -- 172.21.15.196:0/720985718 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c161220 con 0x7f546c0d1240 2024-04-28T13:32:18.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.901+0000 7f5471e00640 1 --2- 172.21.15.196:0/720985718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c0d1240 0x7f546c06e330 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f546c0d0cd0 tx=0x7f546402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf5859c6e5b25d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:18.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.901+0000 7f546be00640 1 -- 172.21.15.196:0/720985718 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464036070 con 0x7f546c0d1240 2024-04-28T13:32:18.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.901+0000 7f546be00640 1 -- 172.21.15.196:0/720985718 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5464031600 con 0x7f546c0d1240 2024-04-28T13:32:18.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.901+0000 7f546be00640 1 -- 172.21.15.196:0/720985718 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54640319d0 con 0x7f546c0d1240 2024-04-28T13:32:18.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.902+0000 7f5473e00640 1 -- 172.21.15.196:0/720985718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c0d1240 msgr2=0x7f546c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:18.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.902+0000 7f5473e00640 1 --2- 172.21.15.196:0/720985718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c0d1240 0x7f546c06e330 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f546c0d0cd0 tx=0x7f546402a6a0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.902+0000 7f5473e00640 1 -- 172.21.15.196:0/720985718 shutdown_connections 2024-04-28T13:32:18.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.902+0000 7f5473e00640 1 --2- 172.21.15.196:0/720985718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c0d1240 0x7f546c06e330 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.902+0000 7f5473e00640 1 --2- 172.21.15.196:0/720985718 >> v2:172.21.15.203:3300/0 conn(0x7f546c0cd420 0x7f546c0d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.902+0000 7f5473e00640 1 --2- 172.21.15.196:0/720985718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f546c1469f0 0x7f546c0ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.902+0000 7f5473e00640 1 -- 172.21.15.196:0/720985718 >> 172.21.15.196:0/720985718 conn(0x7f546c0cd830 msgr2=0x7f546c0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:18.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.902+0000 7f5473e00640 1 -- 172.21.15.196:0/720985718 shutdown_connections 2024-04-28T13:32:18.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.902+0000 7f5473e00640 1 -- 172.21.15.196:0/720985718 wait complete. 2024-04-28T13:32:18.905 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.904+0000 7f5473e00640 1 Processor -- start 2024-04-28T13:32:18.905 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.904+0000 7f5473e00640 1 -- start start 2024-04-28T13:32:18.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.905+0000 7f5473e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f546c0cd420 0x7f546c1392d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:18.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.905+0000 7f5473e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c0d1240 0x7f546c145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:18.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.905+0000 7f5473e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f546c1469f0 0x7f546c1f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:18.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.905+0000 7f5473e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f546c072020 con 0x7f546c0d1240 2024-04-28T13:32:18.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.905+0000 7f5473e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1617e0 con 0x7f546c0cd420 2024-04-28T13:32:18.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.905+0000 7f5473e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f546c160b80 con 0x7f546c1469f0 2024-04-28T13:32:18.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.905+0000 7f5470a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c0d1240 0x7f546c145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:18.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.905+0000 7f5470a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c0d1240 0x7f546c145590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49688/0 (socket says 172.21.15.196:49688) 2024-04-28T13:32:18.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.905+0000 7f5470a00640 1 -- 172.21.15.196:0/2654712262 learned_addr learned my addr 172.21.15.196:0/2654712262 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:18.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.906+0000 7f5471400640 1 --2- 172.21.15.196:0/2654712262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f546c0cd420 0x7f546c1392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:18.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.906+0000 7f5470a00640 1 -- 172.21.15.196:0/2654712262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f546c1469f0 msgr2=0x7f546c1f2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:18.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.906+0000 7f5470a00640 1 --2- 172.21.15.196:0/2654712262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f546c1469f0 0x7f546c1f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.906+0000 7f5470a00640 1 -- 172.21.15.196:0/2654712262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f546c0cd420 msgr2=0x7f546c1392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:18.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.906+0000 7f5470a00640 1 --2- 172.21.15.196:0/2654712262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f546c0cd420 0x7f546c1392d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:18.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.906+0000 7f5470a00640 1 -- 172.21.15.196:0/2654712262 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c143800 con 0x7f546c0d1240 2024-04-28T13:32:18.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.906+0000 7f5471400640 1 --2- 172.21.15.196:0/2654712262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f546c0cd420 0x7f546c1392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:18.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.906+0000 7f5470a00640 1 --2- 172.21.15.196:0/2654712262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c0d1240 0x7f546c145590 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f545800e870 tx=0x7f545800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:18.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.906+0000 7f5469600640 1 -- 172.21.15.196:0/2654712262 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458017070 con 0x7f546c0d1240 2024-04-28T13:32:18.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.906+0000 7f5469600640 1 -- 172.21.15.196:0/2654712262 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f545800c940 con 0x7f546c0d1240 2024-04-28T13:32:18.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.906+0000 7f5469600640 1 -- 172.21.15.196:0/2654712262 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545800cd10 con 0x7f546c0d1240 2024-04-28T13:32:18.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.907+0000 7f5473e00640 1 -- 172.21.15.196:0/2654712262 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546c143ae0 con 0x7f546c0d1240 2024-04-28T13:32:18.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.907+0000 7f5473e00640 1 -- 172.21.15.196:0/2654712262 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f546c1440a0 con 0x7f546c0d1240 2024-04-28T13:32:18.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.908+0000 7f5473e00640 1 -- 172.21.15.196:0/2654712262 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f546c06a290 con 0x7f546c0d1240 2024-04-28T13:32:18.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.909+0000 7f5469600640 1 -- 172.21.15.196:0/2654712262 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f545801a070 con 0x7f546c0d1240 2024-04-28T13:32:18.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.910+0000 7f5469600640 1 --2- 172.21.15.196:0/2654712262 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f544c06c8f0 0x7f544c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:18.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.910+0000 7f5469600640 1 -- 172.21.15.196:0/2654712262 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f545808be70 con 0x7f546c0d1240 2024-04-28T13:32:18.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.910+0000 7f5471400640 1 --2- 172.21.15.196:0/2654712262 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f544c06c8f0 0x7f544c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:18.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.911+0000 7f5471400640 1 --2- 172.21.15.196:0/2654712262 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f544c06c8f0 0x7f544c06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5460009a40 tx=0x7f5460008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:18.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:18.914+0000 7f5469600640 1 -- 172.21.15.196:0/2654712262 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f545805a180 con 0x7f546c0d1240 2024-04-28T13:32:19.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.142+0000 7f5473e00640 1 -- 172.21.15.196:0/2654712262 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f546c06fd90 con 0x7f546c0d1240 2024-04-28T13:32:19.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.143+0000 7f5469600640 1 -- 172.21.15.196:0/2654712262 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f5458059a80 con 0x7f546c0d1240 2024-04-28T13:32:19.145 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:19.145 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/422653807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":422653807},{"type":"v1","addr":"172.21.15.196:6834","nonce":422653807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2793552376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2793552376},{"type":"v1","addr":"172.21.15.196:6835","nonce":2793552376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1810956193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1810956193},{"type":"v1","addr":"172.21.15.203:6837","nonce":1810956193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:31:21.059586+0000","modified":"2024-04-28T13:31:24.132134+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/2114798853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2114798853},{"type":"v1","addr":"172.21.15.203:6836","nonce":2114798853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4253,"qdb_cluster":[4253]},"id":1}]} 2024-04-28T13:32:19.145 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 7 2024-04-28T13:32:19.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.146+0000 7f5473e00640 1 -- 172.21.15.196:0/2654712262 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f544c06c8f0 msgr2=0x7f544c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:19.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.146+0000 7f5473e00640 1 --2- 172.21.15.196:0/2654712262 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f544c06c8f0 0x7f544c06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5460009a40 tx=0x7f5460008040 comp rx=0 tx=0).stop 2024-04-28T13:32:19.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.146+0000 7f5473e00640 1 -- 172.21.15.196:0/2654712262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c0d1240 msgr2=0x7f546c145590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:19.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.146+0000 7f5473e00640 1 --2- 172.21.15.196:0/2654712262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c0d1240 0x7f546c145590 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f545800e870 tx=0x7f545800ed30 comp rx=0 tx=0).stop 2024-04-28T13:32:19.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.146+0000 7f5473e00640 1 -- 172.21.15.196:0/2654712262 shutdown_connections 2024-04-28T13:32:19.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.146+0000 7f5473e00640 1 --2- 172.21.15.196:0/2654712262 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f544c06c8f0 0x7f544c06edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.146+0000 7f5473e00640 1 --2- 172.21.15.196:0/2654712262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f546c1469f0 0x7f546c1f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.146+0000 7f5473e00640 1 --2- 172.21.15.196:0/2654712262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f546c0d1240 0x7f546c145590 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.146+0000 7f5473e00640 1 --2- 172.21.15.196:0/2654712262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f546c0cd420 0x7f546c1392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.146+0000 7f5473e00640 1 -- 172.21.15.196:0/2654712262 >> 172.21.15.196:0/2654712262 conn(0x7f546c0cd830 msgr2=0x7f546c06e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:19.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.146+0000 7f5473e00640 1 -- 172.21.15.196:0/2654712262 shutdown_connections 2024-04-28T13:32:19.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.146+0000 7f5473e00640 1 -- 172.21.15.196:0/2654712262 wait complete. 2024-04-28T13:32:19.165 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:32:19.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.363+0000 7ff518600640 1 Processor -- start 2024-04-28T13:32:19.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.363+0000 7ff518600640 1 -- start start 2024-04-28T13:32:19.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.365+0000 7ff518600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff510145870 0x7ff51006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:19.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.365+0000 7ff518600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff51006f980 0x7ff510072d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:19.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.365+0000 7ff518600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff510073250 0x7ff51015fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:19.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.365+0000 7ff518600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff51006fde0 con 0x7ff510145870 2024-04-28T13:32:19.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.365+0000 7ff518600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff510073a40 con 0x7ff510073250 2024-04-28T13:32:19.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.365+0000 7ff518600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff51015d8d0 con 0x7ff51006f980 2024-04-28T13:32:19.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.365+0000 7ff515c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff510145870 0x7ff51006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:19.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.365+0000 7ff515c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff510145870 0x7ff51006f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49698/0 (socket says 172.21.15.196:49698) 2024-04-28T13:32:19.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.365+0000 7ff515c00640 1 -- 172.21.15.196:0/1850032430 learned_addr learned my addr 172.21.15.196:0/1850032430 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:19.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.366+0000 7ff515c00640 1 -- 172.21.15.196:0/1850032430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff51006f980 msgr2=0x7ff510072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:19.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.366+0000 7ff515c00640 1 --2- 172.21.15.196:0/1850032430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff51006f980 0x7ff510072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.366+0000 7ff515c00640 1 -- 172.21.15.196:0/1850032430 >> v2:172.21.15.203:3300/0 conn(0x7ff510073250 msgr2=0x7ff51015fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:19.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.366+0000 7ff515c00640 1 --2- 172.21.15.196:0/1850032430 >> v2:172.21.15.203:3300/0 conn(0x7ff510073250 0x7ff51015fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.366+0000 7ff515c00640 1 -- 172.21.15.196:0/1850032430 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51015dbb0 con 0x7ff510145870 2024-04-28T13:32:19.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.367+0000 7ff515c00640 1 --2- 172.21.15.196:0/1850032430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff510145870 0x7ff51006f440 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff5100cf140 tx=0x7ff50002a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44c5e98d27ebb6c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:19.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.367+0000 7ff50fe00640 1 -- 172.21.15.196:0/1850032430 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500037070 con 0x7ff510145870 2024-04-28T13:32:19.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.367+0000 7ff50fe00640 1 -- 172.21.15.196:0/1850032430 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff500032440 con 0x7ff510145870 2024-04-28T13:32:19.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.367+0000 7ff50fe00640 1 -- 172.21.15.196:0/1850032430 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500032810 con 0x7ff510145870 2024-04-28T13:32:19.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.368+0000 7ff518600640 1 -- 172.21.15.196:0/1850032430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff510145870 msgr2=0x7ff51006f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:19.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.368+0000 7ff518600640 1 --2- 172.21.15.196:0/1850032430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff510145870 0x7ff51006f440 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff5100cf140 tx=0x7ff50002a1d0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.368+0000 7ff518600640 1 -- 172.21.15.196:0/1850032430 shutdown_connections 2024-04-28T13:32:19.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.368+0000 7ff518600640 1 --2- 172.21.15.196:0/1850032430 >> v2:172.21.15.203:3300/0 conn(0x7ff510073250 0x7ff51015fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.368+0000 7ff518600640 1 --2- 172.21.15.196:0/1850032430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff51006f980 0x7ff510072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.368+0000 7ff518600640 1 --2- 172.21.15.196:0/1850032430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff510145870 0x7ff51006f440 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.368+0000 7ff518600640 1 -- 172.21.15.196:0/1850032430 >> 172.21.15.196:0/1850032430 conn(0x7ff5100cd830 msgr2=0x7ff5100cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:19.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.368+0000 7ff518600640 1 -- 172.21.15.196:0/1850032430 shutdown_connections 2024-04-28T13:32:19.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.368+0000 7ff518600640 1 -- 172.21.15.196:0/1850032430 wait complete. 2024-04-28T13:32:19.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.370+0000 7ff518600640 1 Processor -- start 2024-04-28T13:32:19.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.370+0000 7ff518600640 1 -- start start 2024-04-28T13:32:19.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.371+0000 7ff518600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff51006f980 0x7ff5101f9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:19.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.371+0000 7ff518600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff510073250 0x7ff510158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:19.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.371+0000 7ff518600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff510145870 0x7ff510144410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:19.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.371+0000 7ff518600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff510145c50 con 0x7ff510145870 2024-04-28T13:32:19.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.371+0000 7ff518600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff51015d5d0 con 0x7ff51006f980 2024-04-28T13:32:19.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.371+0000 7ff518600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff5100736e0 con 0x7ff510073250 2024-04-28T13:32:19.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.371+0000 7ff516600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff510145870 0x7ff510144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:19.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.371+0000 7ff516600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff510145870 0x7ff510144410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49700/0 (socket says 172.21.15.196:49700) 2024-04-28T13:32:19.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.371+0000 7ff516600640 1 -- 172.21.15.196:0/2211374970 learned_addr learned my addr 172.21.15.196:0/2211374970 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:19.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.371+0000 7ff515200640 1 --2- 172.21.15.196:0/2211374970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff510073250 0x7ff510158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:19.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.371+0000 7ff516600640 1 -- 172.21.15.196:0/2211374970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff510073250 msgr2=0x7ff510158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:19.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.371+0000 7ff515c00640 1 --2- 172.21.15.196:0/2211374970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff51006f980 0x7ff5101f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:19.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.371+0000 7ff516600640 1 --2- 172.21.15.196:0/2211374970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff510073250 0x7ff510158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.372+0000 7ff516600640 1 -- 172.21.15.196:0/2211374970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff51006f980 msgr2=0x7ff5101f9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:19.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.372+0000 7ff516600640 1 --2- 172.21.15.196:0/2211374970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff51006f980 0x7ff5101f9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.372+0000 7ff516600640 1 -- 172.21.15.196:0/2211374970 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff510157db0 con 0x7ff510145870 2024-04-28T13:32:19.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.372+0000 7ff515200640 1 --2- 172.21.15.196:0/2211374970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff510073250 0x7ff510158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:19.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.372+0000 7ff516600640 1 --2- 172.21.15.196:0/2211374970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff510145870 0x7ff510144410 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff50800b9e0 tx=0x7ff50800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:19.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.372+0000 7ff50d600640 1 -- 172.21.15.196:0/2211374970 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff508019070 con 0x7ff510145870 2024-04-28T13:32:19.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.372+0000 7ff50d600640 1 -- 172.21.15.196:0/2211374970 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff50800cac0 con 0x7ff510145870 2024-04-28T13:32:19.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.372+0000 7ff50d600640 1 -- 172.21.15.196:0/2211374970 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff508014440 con 0x7ff510145870 2024-04-28T13:32:19.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.372+0000 7ff518600640 1 -- 172.21.15.196:0/2211374970 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff510158090 con 0x7ff510145870 2024-04-28T13:32:19.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.372+0000 7ff518600640 1 -- 172.21.15.196:0/2211374970 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff510158650 con 0x7ff510145870 2024-04-28T13:32:19.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.373+0000 7ff518600640 1 -- 172.21.15.196:0/2211374970 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff510159390 con 0x7ff510145870 2024-04-28T13:32:19.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.374+0000 7ff50d600640 1 -- 172.21.15.196:0/2211374970 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff50801c070 con 0x7ff510145870 2024-04-28T13:32:19.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.375+0000 7ff50d600640 1 --2- 172.21.15.196:0/2211374970 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff4f006c8f0 0x7ff4f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:19.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.375+0000 7ff50d600640 1 -- 172.21.15.196:0/2211374970 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff50808b8b0 con 0x7ff510145870 2024-04-28T13:32:19.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.376+0000 7ff515c00640 1 --2- 172.21.15.196:0/2211374970 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff4f006c8f0 0x7ff4f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:19.378 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.377+0000 7ff515c00640 1 --2- 172.21.15.196:0/2211374970 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff4f006c8f0 0x7ff4f006edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff500034000 tx=0x7ff50002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:19.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.383+0000 7ff50d600640 1 -- 172.21.15.196:0/2211374970 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff508059c50 con 0x7ff510145870 2024-04-28T13:32:19.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.568+0000 7ff518600640 1 -- 172.21.15.196:0/2211374970 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff51006a290 con 0x7ff510145870 2024-04-28T13:32:19.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.570+0000 7ff50d600640 1 -- 172.21.15.196:0/2211374970 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16890 (secure 0 0 0) 0x7ff508059550 con 0x7ff510145870 2024-04-28T13:32:19.572 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:19.572 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:03.882452+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:31:19.280975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T13:31:20.037066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:32:19.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.573+0000 7ff518600640 1 -- 172.21.15.196:0/2211374970 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff4f006c8f0 msgr2=0x7ff4f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:19.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.573+0000 7ff518600640 1 --2- 172.21.15.196:0/2211374970 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff4f006c8f0 0x7ff4f006edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff500034000 tx=0x7ff50002e000 comp rx=0 tx=0).stop 2024-04-28T13:32:19.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.573+0000 7ff518600640 1 -- 172.21.15.196:0/2211374970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff510145870 msgr2=0x7ff510144410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:19.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.573+0000 7ff518600640 1 --2- 172.21.15.196:0/2211374970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff510145870 0x7ff510144410 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff50800b9e0 tx=0x7ff50800bea0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.573+0000 7ff518600640 1 -- 172.21.15.196:0/2211374970 shutdown_connections 2024-04-28T13:32:19.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.573+0000 7ff518600640 1 --2- 172.21.15.196:0/2211374970 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff4f006c8f0 0x7ff4f006edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.573+0000 7ff518600640 1 --2- 172.21.15.196:0/2211374970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff510145870 0x7ff510144410 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.573+0000 7ff518600640 1 --2- 172.21.15.196:0/2211374970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff510073250 0x7ff510158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.573+0000 7ff518600640 1 --2- 172.21.15.196:0/2211374970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff51006f980 0x7ff5101f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.573+0000 7ff518600640 1 -- 172.21.15.196:0/2211374970 >> 172.21.15.196:0/2211374970 conn(0x7ff5100cd830 msgr2=0x7ff5100b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:19.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.573+0000 7ff518600640 1 -- 172.21.15.196:0/2211374970 shutdown_connections 2024-04-28T13:32:19.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.573+0000 7ff518600640 1 -- 172.21.15.196:0/2211374970 wait complete. 2024-04-28T13:32:19.591 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-28T13:32:19.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.788+0000 7fd7cf200640 1 Processor -- start 2024-04-28T13:32:19.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.789+0000 7fd7cf200640 1 -- start start 2024-04-28T13:32:19.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.790+0000 7fd7cf200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd7d01629c0 0x7fd7d00d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:19.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.790+0000 7fd7cf200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7d0162da0 con 0x7fd7d0163740 2024-04-28T13:32:19.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.790+0000 7fd7cf200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7d00b6190 con 0x7fd7d0163bf0 2024-04-28T13:32:19.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.790+0000 7fd7cf200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd7d00b6470 con 0x7fd7d01629c0 2024-04-28T13:32:19.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.790+0000 7fd7ce800640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fd7d0163740 0x7fd7d00b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:44366/0 (socket says 172.21.15.196:44366) 2024-04-28T13:32:19.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.791+0000 7fd7ce800640 1 -- 172.21.15.196:0/2546038300 learned_addr learned my addr 172.21.15.196:0/2546038300 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:19.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.791+0000 7fd7cde00640 1 --2- 172.21.15.196:0/2546038300 >> v2:172.21.15.203:3300/0 conn(0x7fd7d01629c0 0x7fd7d00d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:19.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.791+0000 7fd7cca00640 1 -- 172.21.15.196:0/2546038300 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3646757628 0 0) 0x7fd7d0162da0 con 0x7fd7d0163740 2024-04-28T13:32:19.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.791+0000 7fd7cca00640 1 -- 172.21.15.196:0/2546038300 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7b4003520 con 0x7fd7d0163740 2024-04-28T13:32:19.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.791+0000 7fd7cca00640 1 -- 172.21.15.196:0/2546038300 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2134251109 0 0) 0x7fd7b4003520 con 0x7fd7d0163740 2024-04-28T13:32:19.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.791+0000 7fd7cca00640 1 -- 172.21.15.196:0/2546038300 >> v2:172.21.15.203:3300/0 conn(0x7fd7d01629c0 msgr2=0x7fd7d00d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:19.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.791+0000 7fd7cca00640 1 --2- 172.21.15.196:0/2546038300 >> v2:172.21.15.203:3300/0 conn(0x7fd7d01629c0 0x7fd7d00d3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.791+0000 7fd7cca00640 1 -- 172.21.15.196:0/2546038300 >> v1:172.21.15.203:6789/0 conn(0x7fd7d0163bf0 legacy=0x7fd7d0163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:19.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.791+0000 7fd7cca00640 1 -- 172.21.15.196:0/2546038300 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d00b6750 con 0x7fd7d0163740 2024-04-28T13:32:19.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.791+0000 7fd7cca00640 1 -- 172.21.15.196:0/2546038300 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fd7c80019f0 con 0x7fd7d0163740 2024-04-28T13:32:19.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.791+0000 7fd7cca00640 1 -- 172.21.15.196:0/2546038300 >> v1:172.21.15.196:6789/0 conn(0x7fd7d0163740 legacy=0x7fd7d00b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:19.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.791+0000 7fd7cca00640 1 --2- 172.21.15.196:0/2546038300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b40039b0 0x7fd7b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:19.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.791+0000 7fd7cca00640 1 --2- 172.21.15.196:0/2546038300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7b4004800 0x7fd7b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:19.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.792+0000 7fd7cca00640 1 --2- 172.21.15.196:0/2546038300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7b40051c0 0x7fd7b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:19.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.792+0000 7fd7cca00640 1 -- 172.21.15.196:0/2546038300 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d00b6470 con 0x7fd7b40039b0 2024-04-28T13:32:19.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.792+0000 7fd7cca00640 1 -- 172.21.15.196:0/2546038300 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b400c060 con 0x7fd7b4004800 2024-04-28T13:32:19.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.792+0000 7fd7cca00640 1 -- 172.21.15.196:0/2546038300 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b400c260 con 0x7fd7b40051c0 2024-04-28T13:32:19.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.792+0000 7fd7cde00640 1 --2- 172.21.15.196:0/2546038300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b40039b0 0x7fd7b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:19.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.792+0000 7fd7cde00640 1 -- 172.21.15.196:0/2546038300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7b40051c0 msgr2=0x7fd7b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:19.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.792+0000 7fd7cde00640 1 --2- 172.21.15.196:0/2546038300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7b40051c0 0x7fd7b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.792+0000 7fd7cde00640 1 -- 172.21.15.196:0/2546038300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7b4004800 msgr2=0x7fd7b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:19.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.792+0000 7fd7cde00640 1 --2- 172.21.15.196:0/2546038300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7b4004800 0x7fd7b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.792+0000 7fd7cde00640 1 -- 172.21.15.196:0/2546038300 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b4003520 con 0x7fd7b40039b0 2024-04-28T13:32:19.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.792+0000 7fd7ce800640 1 -- 172.21.15.196:0/2546038300 reap_dead start 2024-04-28T13:32:19.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.793+0000 7fd7cde00640 1 --2- 172.21.15.196:0/2546038300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b40039b0 0x7fd7b4003dd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009ac0 tx=0x7fd7b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74e30afdf892a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:19.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.793+0000 7fd7cca00640 1 -- 172.21.15.196:0/2546038300 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0036070 con 0x7fd7b40039b0 2024-04-28T13:32:19.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.793+0000 7fd7cca00640 1 -- 172.21.15.196:0/2546038300 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c0031400 con 0x7fd7b40039b0 2024-04-28T13:32:19.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.793+0000 7fd7cca00640 1 -- 172.21.15.196:0/2546038300 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0009690 con 0x7fd7b40039b0 2024-04-28T13:32:19.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.794+0000 7fd7cf200640 1 -- 172.21.15.196:0/2546038300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b40039b0 msgr2=0x7fd7b4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:19.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.794+0000 7fd7cf200640 1 --2- 172.21.15.196:0/2546038300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b40039b0 0x7fd7b4003dd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009ac0 tx=0x7fd7b4005830 comp rx=0 tx=0).stop 2024-04-28T13:32:19.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.794+0000 7fd7cf200640 1 -- 172.21.15.196:0/2546038300 shutdown_connections 2024-04-28T13:32:19.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.794+0000 7fd7cf200640 1 --2- 172.21.15.196:0/2546038300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7b40039b0 0x7fd7b4003dd0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.794+0000 7fd7cf200640 1 -- 172.21.15.196:0/2546038300 >> 172.21.15.196:0/2546038300 conn(0x7fd7d009e570 msgr2=0x7fd7d00d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:19.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.794+0000 7fd7cf200640 1 -- 172.21.15.196:0/2546038300 shutdown_connections 2024-04-28T13:32:19.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.794+0000 7fd7cf200640 1 -- 172.21.15.196:0/2546038300 wait complete. 2024-04-28T13:32:19.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.795+0000 7fd7cf200640 1 Processor -- start 2024-04-28T13:32:19.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.796+0000 7fd7cf200640 1 -- start start 2024-04-28T13:32:19.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.796+0000 7fd7cf200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7b40039b0 0x7fd7d00cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:19.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.796+0000 7fd7cf200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7d00cfa20 0x7fd7d00cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:19.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.797+0000 7fd7cf200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d00d2020 0x7fd7d00d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:19.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.797+0000 7fd7cf200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0163030 con 0x7fd7d00d2020 2024-04-28T13:32:19.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.797+0000 7fd7cf200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d00b1e60 con 0x7fd7d00cfa20 2024-04-28T13:32:19.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.797+0000 7fd7cf200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d01631b0 con 0x7fd7b40039b0 2024-04-28T13:32:19.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.797+0000 7fd7ce800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d00d2020 0x7fd7d00d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:19.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.797+0000 7fd7ce800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d00d2020 0x7fd7d00d24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49722/0 (socket says 172.21.15.196:49722) 2024-04-28T13:32:19.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.797+0000 7fd7ce800640 1 -- 172.21.15.196:0/1461869699 learned_addr learned my addr 172.21.15.196:0/1461869699 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:19.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.797+0000 7fd7cde00640 1 --2- 172.21.15.196:0/1461869699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7b40039b0 0x7fd7d00cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:19.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.797+0000 7fd7cd400640 1 --2- 172.21.15.196:0/1461869699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7d00cfa20 0x7fd7d00cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:19.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.797+0000 7fd7ce800640 1 -- 172.21.15.196:0/1461869699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7b40039b0 msgr2=0x7fd7d00cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:19.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.797+0000 7fd7ce800640 1 --2- 172.21.15.196:0/1461869699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7b40039b0 0x7fd7d00cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.797+0000 7fd7ce800640 1 -- 172.21.15.196:0/1461869699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7d00cfa20 msgr2=0x7fd7d00cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:19.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.797+0000 7fd7ce800640 1 --2- 172.21.15.196:0/1461869699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7d00cfa20 0x7fd7d00cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:19.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.797+0000 7fd7ce800640 1 -- 172.21.15.196:0/1461869699 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d00cda30 con 0x7fd7d00d2020 2024-04-28T13:32:19.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.798+0000 7fd7ce800640 1 --2- 172.21.15.196:0/1461869699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d00d2020 0x7fd7d00d24a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00d3120 tx=0x7fd7c800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:19.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.798+0000 7fd7c6000640 1 -- 172.21.15.196:0/1461869699 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8004d60 con 0x7fd7d00d2020 2024-04-28T13:32:19.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.798+0000 7fd7c6000640 1 -- 172.21.15.196:0/1461869699 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c800a5b0 con 0x7fd7d00d2020 2024-04-28T13:32:19.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.798+0000 7fd7c6000640 1 -- 172.21.15.196:0/1461869699 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c800a9a0 con 0x7fd7d00d2020 2024-04-28T13:32:19.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.798+0000 7fd7cf200640 1 -- 172.21.15.196:0/1461869699 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d00cdd10 con 0x7fd7d00d2020 2024-04-28T13:32:19.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.798+0000 7fd7cf200640 1 -- 172.21.15.196:0/1461869699 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d00ce2d0 con 0x7fd7d00d2020 2024-04-28T13:32:19.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.799+0000 7fd7cf200640 1 -- 172.21.15.196:0/1461869699 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d006a290 con 0x7fd7d00d2020 2024-04-28T13:32:19.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.800+0000 7fd7c6000640 1 -- 172.21.15.196:0/1461869699 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd7c8011070 con 0x7fd7d00d2020 2024-04-28T13:32:19.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.801+0000 7fd7c6000640 1 --2- 172.21.15.196:0/1461869699 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7b006c920 0x7fd7b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:19.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.801+0000 7fd7c6000640 1 -- 172.21.15.196:0/1461869699 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd7c808b3e0 con 0x7fd7d00d2020 2024-04-28T13:32:19.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.802+0000 7fd7cde00640 1 --2- 172.21.15.196:0/1461869699 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7b006c920 0x7fd7b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:19.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.803+0000 7fd7cde00640 1 --2- 172.21.15.196:0/1461869699 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7b006c920 0x7fd7b006ede0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0033000 tx=0x7fd7c0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:19.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:19.805+0000 7fd7c6000640 1 -- 172.21.15.196:0/1461869699 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd7c80597b0 con 0x7fd7d00d2020 2024-04-28T13:32:20.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.045+0000 7fd7cf200640 1 -- 172.21.15.196:0/1461869699 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd7d006d390 con 0x7fd7d00d2020 2024-04-28T13:32:20.047 INFO:tasks.ceph.mon.a.smithi196.stderr:2024-04-28T13:32:20.046+0000 7f340ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-28T13:32:20.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.049+0000 7fd7c6000640 1 -- 172.21.15.196:0/1461869699 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7fd7c80590b0 con 0x7fd7d00d2020 2024-04-28T13:32:20.051 INFO:teuthology.orchestra.run.smithi196.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-28T13:32:20.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.053+0000 7fd7cf200640 1 -- 172.21.15.196:0/1461869699 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7b006c920 msgr2=0x7fd7b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:20.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.053+0000 7fd7cf200640 1 --2- 172.21.15.196:0/1461869699 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7b006c920 0x7fd7b006ede0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0033000 tx=0x7fd7c0008030 comp rx=0 tx=0).stop 2024-04-28T13:32:20.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.053+0000 7fd7cf200640 1 -- 172.21.15.196:0/1461869699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d00d2020 msgr2=0x7fd7d00d24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:20.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.053+0000 7fd7cf200640 1 --2- 172.21.15.196:0/1461869699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d00d2020 0x7fd7d00d24a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00d3120 tx=0x7fd7c800cc00 comp rx=0 tx=0).stop 2024-04-28T13:32:20.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.054+0000 7fd7cf200640 1 -- 172.21.15.196:0/1461869699 shutdown_connections 2024-04-28T13:32:20.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.054+0000 7fd7cf200640 1 --2- 172.21.15.196:0/1461869699 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7b006c920 0x7fd7b006ede0 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0033000 tx=0x7fd7c0008030 comp rx=0 tx=0).stop 2024-04-28T13:32:20.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.054+0000 7fd7cf200640 1 --2- 172.21.15.196:0/1461869699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d00d2020 0x7fd7d00d24a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:20.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.054+0000 7fd7cf200640 1 --2- 172.21.15.196:0/1461869699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7d00cfa20 0x7fd7d00cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:20.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.054+0000 7fd7cf200640 1 --2- 172.21.15.196:0/1461869699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7b40039b0 0x7fd7d00cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:20.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.054+0000 7fd7cf200640 1 -- 172.21.15.196:0/1461869699 >> 172.21.15.196:0/1461869699 conn(0x7fd7d009e570 msgr2=0x7fd7d01606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:20.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.054+0000 7fd7cf200640 1 -- 172.21.15.196:0/1461869699 shutdown_connections 2024-04-28T13:32:20.056 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.054+0000 7fd7cf200640 1 -- 172.21.15.196:0/1461869699 wait complete. 2024-04-28T13:32:20.074 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-28T13:32:20.094 INFO:tasks.ceph.mds.b.smithi203.stderr: -8570> 2024-04-28T13:31:22.409+0000 7f16b1200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-28T13:32:20.177 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-28T13:32:20.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.271+0000 7fdbe3e00640 1 Processor -- start 2024-04-28T13:32:20.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.272+0000 7fdbe3e00640 1 -- start start 2024-04-28T13:32:20.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.273+0000 7fdbe3e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdbe40b4ca0 0x7fdbe4166a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:20.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.273+0000 7fdbe3e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbe4072fe0 con 0x7fdbe40af930 2024-04-28T13:32:20.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.273+0000 7fdbe3e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbe40732b0 con 0x7fdbe415be50 2024-04-28T13:32:20.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.273+0000 7fdbe3e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdbe4073ec0 con 0x7fdbe40b4ca0 2024-04-28T13:32:20.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.273+0000 7fdbdbe00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fdbe40af930 0x7fdbe40d10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:44378/0 (socket says 172.21.15.196:44378) 2024-04-28T13:32:20.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.274+0000 7fdbdbe00640 1 -- 172.21.15.196:0/4147454043 learned_addr learned my addr 172.21.15.196:0/4147454043 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:20.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.274+0000 7fdbe2000640 1 -- 172.21.15.196:0/4147454043 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1494073646 0 0) 0x7fdbe4072fe0 con 0x7fdbe40af930 2024-04-28T13:32:20.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.274+0000 7fdbe2000640 1 -- 172.21.15.196:0/4147454043 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbc8005730 con 0x7fdbe40af930 2024-04-28T13:32:20.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.274+0000 7fdbe3400640 1 --2- 172.21.15.196:0/4147454043 >> v2:172.21.15.203:3300/0 conn(0x7fdbe40b4ca0 0x7fdbe4166a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:20.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.274+0000 7fdbe2000640 1 -- 172.21.15.196:0/4147454043 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2432967737 0 0) 0x7fdbc8005730 con 0x7fdbe40af930 2024-04-28T13:32:20.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.274+0000 7fdbe2000640 1 -- 172.21.15.196:0/4147454043 >> v2:172.21.15.203:3300/0 conn(0x7fdbe40b4ca0 msgr2=0x7fdbe4166a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:20.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.274+0000 7fdbe2000640 1 --2- 172.21.15.196:0/4147454043 >> v2:172.21.15.203:3300/0 conn(0x7fdbe40b4ca0 0x7fdbe4166a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:20.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.274+0000 7fdbe2000640 1 -- 172.21.15.196:0/4147454043 >> v1:172.21.15.203:6789/0 conn(0x7fdbe415be50 legacy=0x7fdbe40728d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:20.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.274+0000 7fdbe2000640 1 -- 172.21.15.196:0/4147454043 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe4073590 con 0x7fdbe40af930 2024-04-28T13:32:20.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.274+0000 7fdbe2000640 1 -- 172.21.15.196:0/4147454043 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fdbcc001670 con 0x7fdbe40af930 2024-04-28T13:32:20.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.274+0000 7fdbe2000640 1 -- 172.21.15.196:0/4147454043 >> v1:172.21.15.196:6789/0 conn(0x7fdbe40af930 legacy=0x7fdbe40d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:20.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.274+0000 7fdbe2000640 1 --2- 172.21.15.196:0/4147454043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc8005c10 0x7fdbc8006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:20.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.274+0000 7fdbe2000640 1 --2- 172.21.15.196:0/4147454043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc8006a10 0x7fdbc8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:20.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.275+0000 7fdbe2000640 1 --2- 172.21.15.196:0/4147454043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbc80075d0 0x7fdbc800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:20.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.275+0000 7fdbe2000640 1 -- 172.21.15.196:0/4147454043 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe4073ec0 con 0x7fdbc8006a10 2024-04-28T13:32:20.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.275+0000 7fdbe2000640 1 -- 172.21.15.196:0/4147454043 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc800e440 con 0x7fdbc8005c10 2024-04-28T13:32:20.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.275+0000 7fdbe2000640 1 -- 172.21.15.196:0/4147454043 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc800e640 con 0x7fdbc80075d0 2024-04-28T13:32:20.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.275+0000 7fdbdbe00640 1 --2- 172.21.15.196:0/4147454043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc8006a10 0x7fdbc8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:20.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.275+0000 7fdbe2a00640 1 --2- 172.21.15.196:0/4147454043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc8005c10 0x7fdbc8006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:20.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.275+0000 7fdbe3400640 1 --2- 172.21.15.196:0/4147454043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbc80075d0 0x7fdbc800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:20.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.275+0000 7fdbdbe00640 1 -- 172.21.15.196:0/4147454043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbc80075d0 msgr2=0x7fdbc800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:20.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.275+0000 7fdbdbe00640 1 --2- 172.21.15.196:0/4147454043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbc80075d0 0x7fdbc800dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:20.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.275+0000 7fdbdbe00640 1 -- 172.21.15.196:0/4147454043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc8005c10 msgr2=0x7fdbc8006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:20.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.275+0000 7fdbdbe00640 1 --2- 172.21.15.196:0/4147454043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc8005c10 0x7fdbc8006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:20.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.275+0000 7fdbdbe00640 1 -- 172.21.15.196:0/4147454043 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc8005730 con 0x7fdbc8006a10 2024-04-28T13:32:20.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.275+0000 7fdbe3400640 1 -- 172.21.15.196:0/4147454043 reap_dead start 2024-04-28T13:32:20.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.276+0000 7fdbdbe00640 1 --2- 172.21.15.196:0/4147454043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc8006a10 0x7fdbc8006e90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c0e0 tx=0x7fdbc8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd9634cbea936a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:20.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.277+0000 7fdbe2000640 1 -- 172.21.15.196:0/4147454043 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc007c20 con 0x7fdbc8006a10 2024-04-28T13:32:20.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.277+0000 7fdbe2000640 1 -- 172.21.15.196:0/4147454043 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbcc005af0 con 0x7fdbc8006a10 2024-04-28T13:32:20.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.277+0000 7fdbe2000640 1 -- 172.21.15.196:0/4147454043 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc007070 con 0x7fdbc8006a10 2024-04-28T13:32:20.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.277+0000 7fdbe3e00640 1 -- 172.21.15.196:0/4147454043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc8006a10 msgr2=0x7fdbc8006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:20.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.277+0000 7fdbe3e00640 1 --2- 172.21.15.196:0/4147454043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc8006a10 0x7fdbc8006e90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c0e0 tx=0x7fdbc8007a40 comp rx=0 tx=0).stop 2024-04-28T13:32:20.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.277+0000 7fdbe3e00640 1 -- 172.21.15.196:0/4147454043 shutdown_connections 2024-04-28T13:32:20.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.277+0000 7fdbe3e00640 1 --2- 172.21.15.196:0/4147454043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc8006a10 0x7fdbc8006e90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:20.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.277+0000 7fdbe3e00640 1 -- 172.21.15.196:0/4147454043 >> 172.21.15.196:0/4147454043 conn(0x7fdbe406c6c0 msgr2=0x7fdbe406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:20.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.277+0000 7fdbe3e00640 1 -- 172.21.15.196:0/4147454043 shutdown_connections 2024-04-28T13:32:20.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.277+0000 7fdbe3e00640 1 -- 172.21.15.196:0/4147454043 wait complete. 2024-04-28T13:32:20.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.279+0000 7fdbe3e00640 1 Processor -- start 2024-04-28T13:32:20.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.279+0000 7fdbe3e00640 1 -- start start 2024-04-28T13:32:20.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.280+0000 7fdbe3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc8006a10 0x7fdbe4140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:20.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.280+0000 7fdbe3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe40cf320 0x7fdbe415ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:20.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.280+0000 7fdbe3e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbe406eef0 0x7fdbe406f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:20.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.280+0000 7fdbe3e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe415c230 con 0x7fdbe40cf320 2024-04-28T13:32:20.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.280+0000 7fdbe3e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe4073b50 con 0x7fdbc8006a10 2024-04-28T13:32:20.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.280+0000 7fdbe3e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdbe40728d0 con 0x7fdbe406eef0 2024-04-28T13:32:20.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.280+0000 7fdbdbe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe40cf320 0x7fdbe415ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:20.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.280+0000 7fdbdbe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe40cf320 0x7fdbe415ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49744/0 (socket says 172.21.15.196:49744) 2024-04-28T13:32:20.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.280+0000 7fdbdbe00640 1 -- 172.21.15.196:0/393617060 learned_addr learned my addr 172.21.15.196:0/393617060 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:20.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.281+0000 7fdbe3400640 1 --2- 172.21.15.196:0/393617060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbe406eef0 0x7fdbe406f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:20.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.281+0000 7fdbe2a00640 1 --2- 172.21.15.196:0/393617060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc8006a10 0x7fdbe4140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:20.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.281+0000 7fdbdbe00640 1 -- 172.21.15.196:0/393617060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbe406eef0 msgr2=0x7fdbe406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:20.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.281+0000 7fdbdbe00640 1 --2- 172.21.15.196:0/393617060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbe406eef0 0x7fdbe406f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:20.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.281+0000 7fdbdbe00640 1 -- 172.21.15.196:0/393617060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc8006a10 msgr2=0x7fdbe4140090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:20.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.281+0000 7fdbdbe00640 1 --2- 172.21.15.196:0/393617060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc8006a10 0x7fdbe4140090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:20.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.281+0000 7fdbdbe00640 1 -- 172.21.15.196:0/393617060 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe40cfb20 con 0x7fdbe40cf320 2024-04-28T13:32:20.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.281+0000 7fdbdbe00640 1 --2- 172.21.15.196:0/393617060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe40cf320 0x7fdbe415ebb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c0e0 tx=0x7fdbe4073480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:20.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.281+0000 7fdbdaa00640 1 -- 172.21.15.196:0/393617060 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc037040 con 0x7fdbe40cf320 2024-04-28T13:32:20.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.281+0000 7fdbdaa00640 1 -- 172.21.15.196:0/393617060 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbcc0047e0 con 0x7fdbe40cf320 2024-04-28T13:32:20.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.281+0000 7fdbdaa00640 1 -- 172.21.15.196:0/393617060 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc004b90 con 0x7fdbe40cf320 2024-04-28T13:32:20.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.281+0000 7fdbe3e00640 1 -- 172.21.15.196:0/393617060 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe40cfe00 con 0x7fdbe40cf320 2024-04-28T13:32:20.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.281+0000 7fdbe3e00640 1 -- 172.21.15.196:0/393617060 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe40d03c0 con 0x7fdbe40cf320 2024-04-28T13:32:20.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.283+0000 7fdbe3e00640 1 -- 172.21.15.196:0/393617060 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe4050730 con 0x7fdbe40cf320 2024-04-28T13:32:20.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.283+0000 7fdbdaa00640 1 -- 172.21.15.196:0/393617060 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbcc007070 con 0x7fdbe40cf320 2024-04-28T13:32:20.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.284+0000 7fdbdaa00640 1 --2- 172.21.15.196:0/393617060 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdbc006c920 0x7fdbc006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:20.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.284+0000 7fdbdaa00640 1 -- 172.21.15.196:0/393617060 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fdbcc0ab670 con 0x7fdbe40cf320 2024-04-28T13:32:20.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.285+0000 7fdbe2a00640 1 --2- 172.21.15.196:0/393617060 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdbc006c920 0x7fdbc006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:20.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.286+0000 7fdbe2a00640 1 --2- 172.21.15.196:0/393617060 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdbc006c920 0x7fdbc006ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40d07d0 tx=0x7fdbd4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:20.290 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.289+0000 7fdbdaa00640 1 -- 172.21.15.196:0/393617060 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdbcc079a60 con 0x7fdbe40cf320 2024-04-28T13:32:20.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:20.529+0000 7fdbe3e00640 1 -- 172.21.15.196:0/393617060 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdbe40d0590 con 0x7fdbe40cf320 2024-04-28T13:32:21.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.062+0000 7fdbdaa00640 1 -- 172.21.15.196:0/393617060 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7fdbcc079360 con 0x7fdbe40cf320 2024-04-28T13:32:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.065+0000 7fdbe3e00640 1 -- 172.21.15.196:0/393617060 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdbc006c920 msgr2=0x7fdbc006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.065+0000 7fdbe3e00640 1 --2- 172.21.15.196:0/393617060 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdbc006c920 0x7fdbc006ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40d07d0 tx=0x7fdbd4009040 comp rx=0 tx=0).stop 2024-04-28T13:32:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.065+0000 7fdbe3e00640 1 -- 172.21.15.196:0/393617060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe40cf320 msgr2=0x7fdbe415ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.065+0000 7fdbe3e00640 1 --2- 172.21.15.196:0/393617060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe40cf320 0x7fdbe415ebb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c0e0 tx=0x7fdbe4073480 comp rx=0 tx=0).stop 2024-04-28T13:32:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.065+0000 7fdbe3e00640 1 -- 172.21.15.196:0/393617060 shutdown_connections 2024-04-28T13:32:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.065+0000 7fdbe3e00640 1 --2- 172.21.15.196:0/393617060 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdbc006c920 0x7fdbc006ede0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.065+0000 7fdbe3e00640 1 --2- 172.21.15.196:0/393617060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbe406eef0 0x7fdbe406f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.065+0000 7fdbe3e00640 1 --2- 172.21.15.196:0/393617060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe40cf320 0x7fdbe415ebb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:21.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.065+0000 7fdbe3e00640 1 --2- 172.21.15.196:0/393617060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc8006a10 0x7fdbe4140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:21.068 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.065+0000 7fdbe3e00640 1 -- 172.21.15.196:0/393617060 >> 172.21.15.196:0/393617060 conn(0x7fdbe406c6c0 msgr2=0x7fdbe4157d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:21.068 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.066+0000 7fdbe3e00640 1 -- 172.21.15.196:0/393617060 shutdown_connections 2024-04-28T13:32:21.068 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.066+0000 7fdbe3e00640 1 -- 172.21.15.196:0/393617060 wait complete. 2024-04-28T13:32:21.091 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-28T13:32:21.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.295+0000 7fddc2200640 1 Processor -- start 2024-04-28T13:32:21.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.295+0000 7fddc2200640 1 -- start start 2024-04-28T13:32:21.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.296+0000 7fddc2200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fddbc06e330 0x7fddbc0d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:21.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.296+0000 7fddc2200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fddbc0d16d0 0x7fddbc0cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:21.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.296+0000 7fddc2200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddbc0d1ae0 con 0x7fddbc1454c0 2024-04-28T13:32:21.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.296+0000 7fddc2200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fddbc0721a0 con 0x7fddbc06e330 2024-04-28T13:32:21.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.296+0000 7fddc2200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fddbc0ceca0 con 0x7fddbc0d16d0 2024-04-28T13:32:21.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.297+0000 7fddb3e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fddbc06e330 0x7fddbc0d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:21.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.297+0000 7fddb3e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fddbc06e330 0x7fddbc0d1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49754/0 (socket says 172.21.15.196:49754) 2024-04-28T13:32:21.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.297+0000 7fddb3e00640 1 -- 172.21.15.196:0/3318010726 learned_addr learned my addr 172.21.15.196:0/3318010726 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:21.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.297+0000 7fddb3e00640 1 -- 172.21.15.196:0/3318010726 >> v2:172.21.15.203:3300/0 conn(0x7fddbc0d16d0 msgr2=0x7fddbc0cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:21.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.297+0000 7fddb3e00640 1 --2- 172.21.15.196:0/3318010726 >> v2:172.21.15.203:3300/0 conn(0x7fddbc0d16d0 0x7fddbc0cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:21.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.297+0000 7fddb3e00640 1 -- 172.21.15.196:0/3318010726 >> v1:172.21.15.203:6789/0 conn(0x7fddbc1454c0 legacy=0x7fddbc0d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:21.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.297+0000 7fddb3e00640 1 -- 172.21.15.196:0/3318010726 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc06e830 con 0x7fddbc06e330 2024-04-28T13:32:21.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.299+0000 7fddb3e00640 1 --2- 172.21.15.196:0/3318010726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddbc06e330 0x7fddbc0d1190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0b44a0 tx=0x7fdda402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea342f65675a6636 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:21.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.299+0000 7fddbaa00640 1 -- 172.21.15.196:0/3318010726 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4037070 con 0x7fddbc06e330 2024-04-28T13:32:21.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.299+0000 7fddbaa00640 1 -- 172.21.15.196:0/3318010726 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda4032440 con 0x7fddbc06e330 2024-04-28T13:32:21.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.299+0000 7fddbaa00640 1 -- 172.21.15.196:0/3318010726 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4032810 con 0x7fddbc06e330 2024-04-28T13:32:21.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.300+0000 7fddc2200640 1 -- 172.21.15.196:0/3318010726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddbc06e330 msgr2=0x7fddbc0d1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:21.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.300+0000 7fddc2200640 1 --2- 172.21.15.196:0/3318010726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddbc06e330 0x7fddbc0d1190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0b44a0 tx=0x7fdda402a230 comp rx=0 tx=0).stop 2024-04-28T13:32:21.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.300+0000 7fddc2200640 1 -- 172.21.15.196:0/3318010726 shutdown_connections 2024-04-28T13:32:21.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.300+0000 7fddc2200640 1 --2- 172.21.15.196:0/3318010726 >> v2:172.21.15.203:3300/0 conn(0x7fddbc0d16d0 0x7fddbc0cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:21.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.300+0000 7fddc2200640 1 --2- 172.21.15.196:0/3318010726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddbc06e330 0x7fddbc0d1190 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:21.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.300+0000 7fddc2200640 1 -- 172.21.15.196:0/3318010726 >> 172.21.15.196:0/3318010726 conn(0x7fddbc06c690 msgr2=0x7fddbc06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:21.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.300+0000 7fddc2200640 1 -- 172.21.15.196:0/3318010726 shutdown_connections 2024-04-28T13:32:21.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.300+0000 7fddc2200640 1 -- 172.21.15.196:0/3318010726 wait complete. 2024-04-28T13:32:21.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.302+0000 7fddc2200640 1 Processor -- start 2024-04-28T13:32:21.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.302+0000 7fddc2200640 1 -- start start 2024-04-28T13:32:21.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.303+0000 7fddc2200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddbc06e330 0x7fddbc13c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:21.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.303+0000 7fddc2200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddbc0d16d0 0x7fddbc0cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:21.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.303+0000 7fddc2200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddbc1454c0 0x7fddbc0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:21.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.303+0000 7fddc2200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc06f0d0 con 0x7fddbc1454c0 2024-04-28T13:32:21.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.303+0000 7fddc2200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc06edf0 con 0x7fddbc06e330 2024-04-28T13:32:21.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.303+0000 7fddc2200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fddbc15d8a0 con 0x7fddbc0d16d0 2024-04-28T13:32:21.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.303+0000 7fddbbe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddbc1454c0 0x7fddbc0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:21.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.303+0000 7fddbbe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddbc1454c0 0x7fddbc0ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49768/0 (socket says 172.21.15.196:49768) 2024-04-28T13:32:21.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.303+0000 7fddbbe00640 1 -- 172.21.15.196:0/3004308463 learned_addr learned my addr 172.21.15.196:0/3004308463 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:21.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.303+0000 7fddbb400640 1 --2- 172.21.15.196:0/3004308463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddbc06e330 0x7fddbc13c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:21.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.304+0000 7fddb3e00640 1 --2- 172.21.15.196:0/3004308463 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddbc0d16d0 0x7fddbc0cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:21.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.304+0000 7fddbbe00640 1 -- 172.21.15.196:0/3004308463 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddbc0d16d0 msgr2=0x7fddbc0cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:21.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.304+0000 7fddbbe00640 1 --2- 172.21.15.196:0/3004308463 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddbc0d16d0 0x7fddbc0cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:21.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.304+0000 7fddbbe00640 1 -- 172.21.15.196:0/3004308463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddbc06e330 msgr2=0x7fddbc13c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:21.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.304+0000 7fddbbe00640 1 --2- 172.21.15.196:0/3004308463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddbc06e330 0x7fddbc13c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:21.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.304+0000 7fddbbe00640 1 -- 172.21.15.196:0/3004308463 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc1434e0 con 0x7fddbc1454c0 2024-04-28T13:32:21.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.304+0000 7fddbbe00640 1 --2- 172.21.15.196:0/3004308463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddbc1454c0 0x7fddbc0ce190 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fddb400b970 tx=0x7fddb400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:21.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.304+0000 7fddb3400640 1 -- 172.21.15.196:0/3004308463 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb4019070 con 0x7fddbc1454c0 2024-04-28T13:32:21.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.304+0000 7fddb3400640 1 -- 172.21.15.196:0/3004308463 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb400cb60 con 0x7fddbc1454c0 2024-04-28T13:32:21.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.304+0000 7fddb3400640 1 -- 172.21.15.196:0/3004308463 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb4014440 con 0x7fddbc1454c0 2024-04-28T13:32:21.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.304+0000 7fddc2200640 1 -- 172.21.15.196:0/3004308463 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddbc0ca660 con 0x7fddbc1454c0 2024-04-28T13:32:21.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.304+0000 7fddc2200640 1 -- 172.21.15.196:0/3004308463 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddbc0cac20 con 0x7fddbc1454c0 2024-04-28T13:32:21.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.306+0000 7fddc2200640 1 -- 172.21.15.196:0/3004308463 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddbc06a290 con 0x7fddbc1454c0 2024-04-28T13:32:21.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.306+0000 7fddb3400640 1 -- 172.21.15.196:0/3004308463 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddb401c070 con 0x7fddbc1454c0 2024-04-28T13:32:21.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.307+0000 7fddb3400640 1 --2- 172.21.15.196:0/3004308463 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdd9806c8f0 0x7fdd9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:21.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.308+0000 7fddb3400640 1 -- 172.21.15.196:0/3004308463 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fddb408b9a0 con 0x7fddbc1454c0 2024-04-28T13:32:21.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.308+0000 7fddbb400640 1 --2- 172.21.15.196:0/3004308463 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdd9806c8f0 0x7fdd9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:21.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.309+0000 7fddbb400640 1 --2- 172.21.15.196:0/3004308463 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdd9806c8f0 0x7fdd9806edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fddbc157240 tx=0x7fddac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:21.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.312+0000 7fddb3400640 1 -- 172.21.15.196:0/3004308463 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fddb4059cf0 con 0x7fddbc1454c0 2024-04-28T13:32:21.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:21.516+0000 7fddc2200640 1 -- 172.21.15.196:0/3004308463 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fddbc13cd80 con 0x7fddbc1454c0 2024-04-28T13:32:22.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.066+0000 7fddb3400640 1 -- 172.21.15.196:0/3004308463 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7fddb40595f0 con 0x7fddbc1454c0 2024-04-28T13:32:22.068 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'cephfs_metadata' removed 2024-04-28T13:32:22.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.069+0000 7fddc2200640 1 -- 172.21.15.196:0/3004308463 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdd9806c8f0 msgr2=0x7fdd9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:22.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.069+0000 7fddc2200640 1 --2- 172.21.15.196:0/3004308463 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdd9806c8f0 0x7fdd9806edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fddbc157240 tx=0x7fddac009040 comp rx=0 tx=0).stop 2024-04-28T13:32:22.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.069+0000 7fddc2200640 1 -- 172.21.15.196:0/3004308463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddbc1454c0 msgr2=0x7fddbc0ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:22.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.069+0000 7fddc2200640 1 --2- 172.21.15.196:0/3004308463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddbc1454c0 0x7fddbc0ce190 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fddb400b970 tx=0x7fddb400be30 comp rx=0 tx=0).stop 2024-04-28T13:32:22.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.069+0000 7fddc2200640 1 -- 172.21.15.196:0/3004308463 shutdown_connections 2024-04-28T13:32:22.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.069+0000 7fddc2200640 1 --2- 172.21.15.196:0/3004308463 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdd9806c8f0 0x7fdd9806edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:22.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.069+0000 7fddc2200640 1 --2- 172.21.15.196:0/3004308463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddbc1454c0 0x7fddbc0ce190 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:22.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.069+0000 7fddc2200640 1 --2- 172.21.15.196:0/3004308463 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddbc0d16d0 0x7fddbc0cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:22.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.070+0000 7fddc2200640 1 --2- 172.21.15.196:0/3004308463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddbc06e330 0x7fddbc13c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:22.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.070+0000 7fddc2200640 1 -- 172.21.15.196:0/3004308463 >> 172.21.15.196:0/3004308463 conn(0x7fddbc06c690 msgr2=0x7fddbc15f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:22.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.070+0000 7fddc2200640 1 -- 172.21.15.196:0/3004308463 shutdown_connections 2024-04-28T13:32:22.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.070+0000 7fddc2200640 1 -- 172.21.15.196:0/3004308463 wait complete. 2024-04-28T13:32:22.093 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-28T13:32:22.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.295+0000 7f7d77200640 1 Processor -- start 2024-04-28T13:32:22.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.295+0000 7f7d77200640 1 -- start start 2024-04-28T13:32:22.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.297+0000 7f7d77200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7d780b50c0 0x7f7d780d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:22.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.297+0000 7f7d77200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d780d3900 con 0x7f7d780cfb40 2024-04-28T13:32:22.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.297+0000 7f7d77200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d780d3be0 con 0x7f7d781629c0 2024-04-28T13:32:22.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.297+0000 7f7d77200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f7d780b5580 con 0x7f7d780b50c0 2024-04-28T13:32:22.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.297+0000 7f7d76800640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f7d780cfb40 0x7f7d780b5aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:58034/0 (socket says 172.21.15.196:58034) 2024-04-28T13:32:22.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.297+0000 7f7d76800640 1 -- 172.21.15.196:0/1797670071 learned_addr learned my addr 172.21.15.196:0/1797670071 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:22.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d75400640 1 --2- 172.21.15.196:0/1797670071 >> v2:172.21.15.203:3300/0 conn(0x7f7d780b50c0 0x7f7d780d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:22.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d74a00640 1 -- 172.21.15.196:0/1797670071 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2835508697 0 0) 0x7f7d780d3900 con 0x7f7d780cfb40 2024-04-28T13:32:22.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d74a00640 1 -- 172.21.15.196:0/1797670071 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d5c003520 con 0x7f7d780cfb40 2024-04-28T13:32:22.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d74a00640 1 -- 172.21.15.196:0/1797670071 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2263683907 0 0) 0x7f7d5c003520 con 0x7f7d780cfb40 2024-04-28T13:32:22.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d74a00640 1 -- 172.21.15.196:0/1797670071 >> v2:172.21.15.203:3300/0 conn(0x7f7d780b50c0 msgr2=0x7f7d780d5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:22.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d74a00640 1 --2- 172.21.15.196:0/1797670071 >> v2:172.21.15.203:3300/0 conn(0x7f7d780b50c0 0x7f7d780d5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:22.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d74a00640 1 -- 172.21.15.196:0/1797670071 >> v1:172.21.15.203:6789/0 conn(0x7f7d781629c0 legacy=0x7f7d780b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:22.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d74a00640 1 -- 172.21.15.196:0/1797670071 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d780b4140 con 0x7f7d780cfb40 2024-04-28T13:32:22.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d74a00640 1 -- 172.21.15.196:0/1797670071 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f7d70001670 con 0x7f7d780cfb40 2024-04-28T13:32:22.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d74a00640 1 -- 172.21.15.196:0/1797670071 >> v1:172.21.15.196:6789/0 conn(0x7f7d780cfb40 legacy=0x7f7d780b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:22.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d74a00640 1 --2- 172.21.15.196:0/1797670071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c003a00 0x7f7d5c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:22.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d74a00640 1 --2- 172.21.15.196:0/1797670071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d5c004800 0x7f7d5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:22.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d74a00640 1 --2- 172.21.15.196:0/1797670071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d5c005210 0x7f7d5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:22.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d74a00640 1 -- 172.21.15.196:0/1797670071 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d780b5580 con 0x7f7d5c003a00 2024-04-28T13:32:22.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d74a00640 1 -- 172.21.15.196:0/1797670071 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c00c160 con 0x7f7d5c004800 2024-04-28T13:32:22.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.298+0000 7f7d74a00640 1 -- 172.21.15.196:0/1797670071 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7d5c00c360 con 0x7f7d5c005210 2024-04-28T13:32:22.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.299+0000 7f7d75e00640 1 --2- 172.21.15.196:0/1797670071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c003a00 0x7f7d5c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:22.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.299+0000 7f7d76800640 1 --2- 172.21.15.196:0/1797670071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d5c005210 0x7f7d5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:22.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.299+0000 7f7d75e00640 1 -- 172.21.15.196:0/1797670071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d5c005210 msgr2=0x7f7d5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:22.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.299+0000 7f7d75400640 1 --2- 172.21.15.196:0/1797670071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d5c004800 0x7f7d5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:22.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.299+0000 7f7d75e00640 1 --2- 172.21.15.196:0/1797670071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d5c005210 0x7f7d5c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:22.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.299+0000 7f7d75e00640 1 -- 172.21.15.196:0/1797670071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d5c004800 msgr2=0x7f7d5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:22.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.299+0000 7f7d75e00640 1 --2- 172.21.15.196:0/1797670071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d5c004800 0x7f7d5c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:22.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.299+0000 7f7d75e00640 1 -- 172.21.15.196:0/1797670071 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5c003520 con 0x7f7d5c003a00 2024-04-28T13:32:22.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.299+0000 7f7d76800640 1 -- 172.21.15.196:0/1797670071 reap_dead start 2024-04-28T13:32:22.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.300+0000 7f7d75e00640 1 --2- 172.21.15.196:0/1797670071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c003a00 0x7f7d5c003e20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7d68001e50 tx=0x7f7d5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9e4dd9bb9c738 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:22.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.301+0000 7f7d74a00640 1 -- 172.21.15.196:0/1797670071 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68001a70 con 0x7f7d5c003a00 2024-04-28T13:32:22.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.301+0000 7f7d74a00640 1 -- 172.21.15.196:0/1797670071 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d68003970 con 0x7f7d5c003a00 2024-04-28T13:32:22.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.301+0000 7f7d74a00640 1 -- 172.21.15.196:0/1797670071 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68003d60 con 0x7f7d5c003a00 2024-04-28T13:32:22.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.301+0000 7f7d77200640 1 -- 172.21.15.196:0/1797670071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c003a00 msgr2=0x7f7d5c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:22.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.301+0000 7f7d77200640 1 --2- 172.21.15.196:0/1797670071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c003a00 0x7f7d5c003e20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7d68001e50 tx=0x7f7d5c005830 comp rx=0 tx=0).stop 2024-04-28T13:32:22.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.301+0000 7f7d77200640 1 -- 172.21.15.196:0/1797670071 shutdown_connections 2024-04-28T13:32:22.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.301+0000 7f7d77200640 1 --2- 172.21.15.196:0/1797670071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c003a00 0x7f7d5c003e20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:22.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.301+0000 7f7d77200640 1 -- 172.21.15.196:0/1797670071 >> 172.21.15.196:0/1797670071 conn(0x7f7d7809e570 msgr2=0x7f7d780d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:22.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.301+0000 7f7d77200640 1 -- 172.21.15.196:0/1797670071 shutdown_connections 2024-04-28T13:32:22.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.301+0000 7f7d77200640 1 -- 172.21.15.196:0/1797670071 wait complete. 2024-04-28T13:32:22.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.303+0000 7f7d77200640 1 Processor -- start 2024-04-28T13:32:22.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.303+0000 7f7d77200640 1 -- start start 2024-04-28T13:32:22.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.304+0000 7f7d77200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c003a00 0x7f7d780c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:22.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.304+0000 7f7d77200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d780c97b0 0x7f7d780b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:22.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.304+0000 7f7d77200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d780b9900 0x7f7d780b9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:22.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.304+0000 7f7d77200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d780d4040 con 0x7f7d5c003a00 2024-04-28T13:32:22.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.304+0000 7f7d77200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7d780d3ec0 con 0x7f7d780b9900 2024-04-28T13:32:22.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.304+0000 7f7d77200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7d780b3880 con 0x7f7d780c97b0 2024-04-28T13:32:22.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.305+0000 7f7d75e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c003a00 0x7f7d780c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:22.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.305+0000 7f7d75e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c003a00 0x7f7d780c9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55020/0 (socket says 172.21.15.196:55020) 2024-04-28T13:32:22.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.305+0000 7f7d75e00640 1 -- 172.21.15.196:0/1427413763 learned_addr learned my addr 172.21.15.196:0/1427413763 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:22.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.305+0000 7f7d75400640 1 --2- 172.21.15.196:0/1427413763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d780c97b0 0x7f7d780b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:22.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.305+0000 7f7d75e00640 1 -- 172.21.15.196:0/1427413763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d780c97b0 msgr2=0x7f7d780b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:22.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.305+0000 7f7d75e00640 1 --2- 172.21.15.196:0/1427413763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d780c97b0 0x7f7d780b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:22.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.305+0000 7f7d75e00640 1 -- 172.21.15.196:0/1427413763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d780b9900 msgr2=0x7f7d780b9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:22.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.305+0000 7f7d75e00640 1 --2- 172.21.15.196:0/1427413763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d780b9900 0x7f7d780b9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:22.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.305+0000 7f7d75e00640 1 -- 172.21.15.196:0/1427413763 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d780ba2c0 con 0x7f7d5c003a00 2024-04-28T13:32:22.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.305+0000 7f7d75400640 1 --2- 172.21.15.196:0/1427413763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d780c97b0 0x7f7d780b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:22.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.305+0000 7f7d75e00640 1 --2- 172.21.15.196:0/1427413763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c003a00 0x7f7d780c9270 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7d6802a920 tx=0x7f7d680014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:22.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.306+0000 7f7d6e000640 1 -- 172.21.15.196:0/1427413763 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6803f020 con 0x7f7d5c003a00 2024-04-28T13:32:22.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.306+0000 7f7d6e000640 1 -- 172.21.15.196:0/1427413763 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d680052d0 con 0x7f7d5c003a00 2024-04-28T13:32:22.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.306+0000 7f7d6e000640 1 -- 172.21.15.196:0/1427413763 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68037040 con 0x7f7d5c003a00 2024-04-28T13:32:22.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.306+0000 7f7d77200640 1 -- 172.21.15.196:0/1427413763 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d780c9fe0 con 0x7f7d5c003a00 2024-04-28T13:32:22.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.306+0000 7f7d77200640 1 -- 172.21.15.196:0/1427413763 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d780ca5a0 con 0x7f7d5c003a00 2024-04-28T13:32:22.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.307+0000 7f7d77200640 1 -- 172.21.15.196:0/1427413763 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d78050730 con 0x7f7d5c003a00 2024-04-28T13:32:22.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.308+0000 7f7d6e000640 1 -- 172.21.15.196:0/1427413763 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d68042020 con 0x7f7d5c003a00 2024-04-28T13:32:22.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.309+0000 7f7d6e000640 1 --2- 172.21.15.196:0/1427413763 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7d5806c8f0 0x7f7d5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:22.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.309+0000 7f7d6e000640 1 -- 172.21.15.196:0/1427413763 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5473+0+0 (secure 0 0 0) 0x7f7d680abba0 con 0x7f7d5c003a00 2024-04-28T13:32:22.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.310+0000 7f7d75400640 1 --2- 172.21.15.196:0/1427413763 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7d5806c8f0 0x7f7d5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:22.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.311+0000 7f7d75400640 1 --2- 172.21.15.196:0/1427413763 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7d5806c8f0 0x7f7d5806edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d780bb2a0 tx=0x7f7d64008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:22.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.314+0000 7f7d6e000640 1 -- 172.21.15.196:0/1427413763 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7d68049020 con 0x7f7d5c003a00 2024-04-28T13:32:22.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:22.513+0000 7f7d77200640 1 -- 172.21.15.196:0/1427413763 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7d7815edd0 con 0x7f7d5c003a00 2024-04-28T13:32:23.080 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.079+0000 7f7d6e000640 1 -- 172.21.15.196:0/1427413763 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f7d6807a0b0 con 0x7f7d5c003a00 2024-04-28T13:32:23.080 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'cephfs_data' removed 2024-04-28T13:32:23.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.083+0000 7f7d77200640 1 -- 172.21.15.196:0/1427413763 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7d5806c8f0 msgr2=0x7f7d5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:23.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.083+0000 7f7d77200640 1 --2- 172.21.15.196:0/1427413763 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7d5806c8f0 0x7f7d5806edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d780bb2a0 tx=0x7f7d64008040 comp rx=0 tx=0).stop 2024-04-28T13:32:23.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.083+0000 7f7d77200640 1 -- 172.21.15.196:0/1427413763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c003a00 msgr2=0x7f7d780c9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:23.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.083+0000 7f7d77200640 1 --2- 172.21.15.196:0/1427413763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c003a00 0x7f7d780c9270 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7d6802a920 tx=0x7f7d680014a0 comp rx=0 tx=0).stop 2024-04-28T13:32:23.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.083+0000 7f7d77200640 1 -- 172.21.15.196:0/1427413763 shutdown_connections 2024-04-28T13:32:23.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.083+0000 7f7d77200640 1 --2- 172.21.15.196:0/1427413763 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7d5806c8f0 0x7f7d5806edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:23.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.083+0000 7f7d77200640 1 --2- 172.21.15.196:0/1427413763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d780b9900 0x7f7d780b9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:23.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.083+0000 7f7d77200640 1 --2- 172.21.15.196:0/1427413763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d780c97b0 0x7f7d780b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:23.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.083+0000 7f7d77200640 1 --2- 172.21.15.196:0/1427413763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d5c003a00 0x7f7d780c9270 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:23.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.083+0000 7f7d77200640 1 -- 172.21.15.196:0/1427413763 >> 172.21.15.196:0/1427413763 conn(0x7f7d7809e570 msgr2=0x7f7d7806e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:23.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.087+0000 7f7d77200640 1 -- 172.21.15.196:0/1427413763 shutdown_connections 2024-04-28T13:32:23.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.087+0000 7f7d77200640 1 -- 172.21.15.196:0/1427413763 wait complete. 2024-04-28T13:32:23.112 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-28T13:32:23.112 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-28T13:32:23.112 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-28T13:32:23.112 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:32:23.112 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-28T13:32:23.112 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-28T13:32:23.113 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-28T13:32:23.113 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:32:23.113 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-28T13:32:23.113 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-28T13:32:23.113 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-28T13:32:23.113 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:32:23.113 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-28T13:32:23.113 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-28T13:32:23.114 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-28T13:32:23.114 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:32:23.136 INFO:tasks.ceph.mds.d:Stopped 2024-04-28T13:32:23.136 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-28T13:32:23.137 INFO:tasks.ceph.mds.a:Stopped 2024-04-28T13:32:23.137 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-28T13:32:23.138 INFO:tasks.ceph.mds.b:Stopped 2024-04-28T13:32:23.138 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-28T13:32:23.140 INFO:tasks.ceph.mds.c:Stopped 2024-04-28T13:32:23.140 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-28T13:32:23.185 INFO:tasks.ceph.mds.d:Started 2024-04-28T13:32:23.185 DEBUG:teuthology.parallel:result is None 2024-04-28T13:32:23.186 INFO:tasks.ceph.mds.a:Started 2024-04-28T13:32:23.187 DEBUG:teuthology.parallel:result is None 2024-04-28T13:32:23.190 INFO:tasks.ceph.mds.b:Started 2024-04-28T13:32:23.191 DEBUG:teuthology.parallel:result is None 2024-04-28T13:32:23.192 INFO:tasks.ceph.mds.c:Started 2024-04-28T13:32:23.193 DEBUG:teuthology.parallel:result is None 2024-04-28T13:32:23.193 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-28T13:32:23.338 INFO:tasks.ceph.mds.c.smithi196.stdout:starting mds.c at 2024-04-28T13:32:23.339 INFO:tasks.ceph.mds.a.smithi196.stdout:starting mds.a at 2024-04-28T13:32:23.339 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-28T13:32:23.340 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-28T13:32:23.378 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.377+0000 7f810a800640 1 Processor -- start 2024-04-28T13:32:23.378 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.377+0000 7f810a800640 1 -- start start 2024-04-28T13:32:23.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.378+0000 7f810a800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f81040d0d00 0x7f81040d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:23.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.378+0000 7f810a800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8104160f40 con 0x7f81041469f0 2024-04-28T13:32:23.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.378+0000 7f810a800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8104161220 con 0x7f81040b5dd0 2024-04-28T13:32:23.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.378+0000 7f810a800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f81040b5960 con 0x7f81040d0d00 2024-04-28T13:32:23.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.379+0000 7f8102a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f81040d0d00 0x7f81040d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:23.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.379+0000 7f8103400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f81041469f0 0x7f81040ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:58052/0 (socket says 172.21.15.196:58052) 2024-04-28T13:32:23.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.379+0000 7f8103400640 1 -- 172.21.15.196:0/3709563420 learned_addr learned my addr 172.21.15.196:0/3709563420 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:23.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.379+0000 7f8102a00640 1 -- 172.21.15.196:0/3709563420 >> v1:172.21.15.203:6789/0 conn(0x7f81040b5dd0 legacy=0x7f81040b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:23.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.379+0000 7f8102a00640 1 -- 172.21.15.196:0/3709563420 >> v1:172.21.15.196:6789/0 conn(0x7f81041469f0 legacy=0x7f81040ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:23.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.379+0000 7f8102a00640 1 -- 172.21.15.196:0/3709563420 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8104161500 con 0x7f81040d0d00 2024-04-28T13:32:23.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.380+0000 7f8102a00640 1 --2- 172.21.15.196:0/3709563420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81040d0d00 0x7f81040d1190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f80f00097a0 tx=0x7f80f002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f19c2de4c567bb server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:23.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.380+0000 7f8102000640 1 -- 172.21.15.196:0/3709563420 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0037070 con 0x7f81040d0d00 2024-04-28T13:32:23.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.381+0000 7f8102000640 1 -- 172.21.15.196:0/3709563420 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80f002adc0 con 0x7f81040d0d00 2024-04-28T13:32:23.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.381+0000 7f8102000640 1 -- 172.21.15.196:0/3709563420 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0032660 con 0x7f81040d0d00 2024-04-28T13:32:23.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.381+0000 7f810a800640 1 -- 172.21.15.196:0/3709563420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81040d0d00 msgr2=0x7f81040d1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:23.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.381+0000 7f810a800640 1 --2- 172.21.15.196:0/3709563420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81040d0d00 0x7f81040d1190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f80f00097a0 tx=0x7f80f002a130 comp rx=0 tx=0).stop 2024-04-28T13:32:23.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.381+0000 7f810a800640 1 -- 172.21.15.196:0/3709563420 shutdown_connections 2024-04-28T13:32:23.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.381+0000 7f810a800640 1 --2- 172.21.15.196:0/3709563420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81040d0d00 0x7f81040d1190 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:23.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.381+0000 7f810a800640 1 -- 172.21.15.196:0/3709563420 >> 172.21.15.196:0/3709563420 conn(0x7f81040cd830 msgr2=0x7f81040cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:23.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.381+0000 7f810a800640 1 -- 172.21.15.196:0/3709563420 shutdown_connections 2024-04-28T13:32:23.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.381+0000 7f810a800640 1 -- 172.21.15.196:0/3709563420 wait complete. 2024-04-28T13:32:23.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.383+0000 7f810a800640 1 Processor -- start 2024-04-28T13:32:23.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.383+0000 7f810a800640 1 -- start start 2024-04-28T13:32:23.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.384+0000 7f810a800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81040b5dd0 0x7f81040b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:23.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.384+0000 7f810a800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81040d0d00 0x7f8104143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:23.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.384+0000 7f810a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81041469f0 0x7f81041f9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:23.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.384+0000 7f810a800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81040cd5f0 con 0x7f81041469f0 2024-04-28T13:32:23.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.384+0000 7f810a800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8104161ac0 con 0x7f81040b5dd0 2024-04-28T13:32:23.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.384+0000 7f810a800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8104072020 con 0x7f81040d0d00 2024-04-28T13:32:23.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.384+0000 7f8103e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81041469f0 0x7f81041f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:23.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.384+0000 7f8103e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81041469f0 0x7f81041f9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55084/0 (socket says 172.21.15.196:55084) 2024-04-28T13:32:23.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.384+0000 7f8103e00640 1 -- 172.21.15.196:0/3422090044 learned_addr learned my addr 172.21.15.196:0/3422090044 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:23.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.385+0000 7f8102a00640 1 --2- 172.21.15.196:0/3422090044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81040d0d00 0x7f8104143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:23.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.385+0000 7f8103400640 1 --2- 172.21.15.196:0/3422090044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81040b5dd0 0x7f81040b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:23.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.385+0000 7f8103e00640 1 -- 172.21.15.196:0/3422090044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81040d0d00 msgr2=0x7f8104143250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:23.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.385+0000 7f8103e00640 1 --2- 172.21.15.196:0/3422090044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81040d0d00 0x7f8104143250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:23.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.385+0000 7f8103e00640 1 -- 172.21.15.196:0/3422090044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81040b5dd0 msgr2=0x7f81040b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:23.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.385+0000 7f8103e00640 1 --2- 172.21.15.196:0/3422090044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81040b5dd0 0x7f81040b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:23.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.385+0000 7f8103e00640 1 -- 172.21.15.196:0/3422090044 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8104140a60 con 0x7f81041469f0 2024-04-28T13:32:23.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.385+0000 7f8103e00640 1 --2- 172.21.15.196:0/3422090044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81041469f0 0x7f81041f9160 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00c760 tx=0x7f80fc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:23.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.385+0000 7f80fb400640 1 -- 172.21.15.196:0/3422090044 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc003b70 con 0x7f81041469f0 2024-04-28T13:32:23.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.386+0000 7f80fb400640 1 -- 172.21.15.196:0/3422090044 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80fc003d30 con 0x7f81041469f0 2024-04-28T13:32:23.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.386+0000 7f80fb400640 1 -- 172.21.15.196:0/3422090044 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00aa90 con 0x7f81041469f0 2024-04-28T13:32:23.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.386+0000 7f810a800640 1 -- 172.21.15.196:0/3422090044 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81040b1490 con 0x7f81041469f0 2024-04-28T13:32:23.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.386+0000 7f810a800640 1 -- 172.21.15.196:0/3422090044 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81040b1a50 con 0x7f81041469f0 2024-04-28T13:32:23.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.387+0000 7f810a800640 1 -- 172.21.15.196:0/3422090044 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f810406a290 con 0x7f81041469f0 2024-04-28T13:32:23.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.388+0000 7f80fb400640 1 -- 172.21.15.196:0/3422090044 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80fc00ac30 con 0x7f81041469f0 2024-04-28T13:32:23.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.389+0000 7f80fb400640 1 --2- 172.21.15.196:0/3422090044 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f80e406c8f0 0x7f80e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:23.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.390+0000 7f80fb400640 1 -- 172.21.15.196:0/3422090044 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7f80fc08b570 con 0x7f81041469f0 2024-04-28T13:32:23.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.390+0000 7f8103400640 1 --2- 172.21.15.196:0/3422090044 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f80e406c8f0 0x7f80e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:23.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.391+0000 7f8103400640 1 --2- 172.21.15.196:0/3422090044 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f80e406c8f0 0x7f80e406edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f80f4000f60 tx=0x7f80f4007e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:23.395 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.394+0000 7f80fb400640 1 -- 172.21.15.196:0/3422090044 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f80fc059640 con 0x7f81041469f0 2024-04-28T13:32:23.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:23.608+0000 7f810a800640 1 -- 172.21.15.196:0/3422090044 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f810406fd90 con 0x7f81041469f0 2024-04-28T13:32:24.109 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.108+0000 7f80fb400640 1 -- 172.21.15.196:0/3422090044 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7f80fc05d940 con 0x7f81041469f0 2024-04-28T13:32:24.109 INFO:teuthology.orchestra.run.smithi196.stderr: removed all blocklist entries 2024-04-28T13:32:24.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.111+0000 7f810a800640 1 -- 172.21.15.196:0/3422090044 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f80e406c8f0 msgr2=0x7f80e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:24.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.111+0000 7f810a800640 1 --2- 172.21.15.196:0/3422090044 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f80e406c8f0 0x7f80e406edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f80f4000f60 tx=0x7f80f4007e00 comp rx=0 tx=0).stop 2024-04-28T13:32:24.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.111+0000 7f810a800640 1 -- 172.21.15.196:0/3422090044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81041469f0 msgr2=0x7f81041f9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:24.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.111+0000 7f810a800640 1 --2- 172.21.15.196:0/3422090044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81041469f0 0x7f81041f9160 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00c760 tx=0x7f80fc00cc20 comp rx=0 tx=0).stop 2024-04-28T13:32:24.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.112+0000 7f810a800640 1 -- 172.21.15.196:0/3422090044 shutdown_connections 2024-04-28T13:32:24.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.112+0000 7f810a800640 1 --2- 172.21.15.196:0/3422090044 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f80e406c8f0 0x7f80e406edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.112+0000 7f810a800640 1 --2- 172.21.15.196:0/3422090044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81041469f0 0x7f81041f9160 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.112+0000 7f810a800640 1 --2- 172.21.15.196:0/3422090044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81040d0d00 0x7f8104143250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.112+0000 7f810a800640 1 --2- 172.21.15.196:0/3422090044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81040b5dd0 0x7f81040b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.112+0000 7f810a800640 1 -- 172.21.15.196:0/3422090044 >> 172.21.15.196:0/3422090044 conn(0x7f81040cd830 msgr2=0x7f8104158b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:24.114 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.112+0000 7f810a800640 1 -- 172.21.15.196:0/3422090044 shutdown_connections 2024-04-28T13:32:24.114 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.112+0000 7f810a800640 1 -- 172.21.15.196:0/3422090044 wait complete. 2024-04-28T13:32:24.138 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-28T13:32:24.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.323+0000 7f071e000640 1 Processor -- start 2024-04-28T13:32:24.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.323+0000 7f071e000640 1 -- start start 2024-04-28T13:32:24.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.325+0000 7f071e000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07180b3e70 0x7f0718074020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:24.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.325+0000 7f071e000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f071806ea30 0x7f0718167310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:24.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.325+0000 7f071e000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07180732d0 con 0x7f0718161220 2024-04-28T13:32:24.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.325+0000 7f071e000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f0718073540 con 0x7f071806ea30 2024-04-28T13:32:24.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.325+0000 7f071e000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0718073820 con 0x7f07180b3e70 2024-04-28T13:32:24.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.326+0000 7f0717e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f071806ea30 0x7f0718167310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:24.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.326+0000 7f0717e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f071806ea30 0x7f0718167310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:47462/0 (socket says 172.21.15.196:47462) 2024-04-28T13:32:24.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.326+0000 7f0717e00640 1 -- 172.21.15.196:0/3143477527 learned_addr learned my addr 172.21.15.196:0/3143477527 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:24.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.326+0000 7f070fe00640 1 --2- 172.21.15.196:0/3143477527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07180b3e70 0x7f0718074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:24.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.327+0000 7f0717e00640 1 -- 172.21.15.196:0/3143477527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07180b3e70 msgr2=0x7f0718074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:24.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.327+0000 7f0717e00640 1 --2- 172.21.15.196:0/3143477527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07180b3e70 0x7f0718074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.327+0000 7f0717e00640 1 -- 172.21.15.196:0/3143477527 >> v1:172.21.15.203:6789/0 conn(0x7f0718161220 legacy=0x7f071806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:24.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.327+0000 7f0716a00640 1 -- 172.21.15.196:0/3143477527 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1362257716 0 0) 0x7f07180732d0 con 0x7f0718161220 2024-04-28T13:32:24.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.327+0000 7f0717e00640 1 -- 172.21.15.196:0/3143477527 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0718164c10 con 0x7f071806ea30 2024-04-28T13:32:24.329 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.328+0000 7f0717e00640 1 --2- 172.21.15.196:0/3143477527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f071806ea30 0x7f0718167310 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0710009e50 tx=0x7f071002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b97722ba13380332 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:24.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.329+0000 7f0716a00640 1 -- 172.21.15.196:0/3143477527 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710036070 con 0x7f071806ea30 2024-04-28T13:32:24.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.329+0000 7f0716a00640 1 -- 172.21.15.196:0/3143477527 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0710031600 con 0x7f071806ea30 2024-04-28T13:32:24.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.329+0000 7f0716a00640 1 -- 172.21.15.196:0/3143477527 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07100319d0 con 0x7f071806ea30 2024-04-28T13:32:24.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.330+0000 7f071e000640 1 -- 172.21.15.196:0/3143477527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f071806ea30 msgr2=0x7f0718167310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:24.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.330+0000 7f071e000640 1 --2- 172.21.15.196:0/3143477527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f071806ea30 0x7f0718167310 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0710009e50 tx=0x7f071002a680 comp rx=0 tx=0).stop 2024-04-28T13:32:24.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.330+0000 7f071e000640 1 -- 172.21.15.196:0/3143477527 shutdown_connections 2024-04-28T13:32:24.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.330+0000 7f071e000640 1 --2- 172.21.15.196:0/3143477527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f071806ea30 0x7f0718167310 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.330+0000 7f071e000640 1 --2- 172.21.15.196:0/3143477527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07180b3e70 0x7f0718074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.330+0000 7f071e000640 1 -- 172.21.15.196:0/3143477527 >> 172.21.15.196:0/3143477527 conn(0x7f07180d0d30 msgr2=0x7f07180d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:24.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.330+0000 7f071e000640 1 -- 172.21.15.196:0/3143477527 shutdown_connections 2024-04-28T13:32:24.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.330+0000 7f071e000640 1 -- 172.21.15.196:0/3143477527 wait complete. 2024-04-28T13:32:24.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.332+0000 7f071e000640 1 Processor -- start 2024-04-28T13:32:24.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.332+0000 7f071e000640 1 -- start start 2024-04-28T13:32:24.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.333+0000 7f071e000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f071806ea30 0x7f07180cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:24.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.333+0000 7f071e000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07180b3e70 0x7f071815c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:24.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.333+0000 7f071e000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0718161220 0x7f07181edbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:24.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.333+0000 7f071e000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0718161780 con 0x7f07180b3e70 2024-04-28T13:32:24.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.333+0000 7f071e000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0718161600 con 0x7f0718161220 2024-04-28T13:32:24.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.333+0000 7f071e000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f07181647a0 con 0x7f071806ea30 2024-04-28T13:32:24.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.333+0000 7f070fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07180b3e70 0x7f071815c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:24.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.333+0000 7f070fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07180b3e70 0x7f071815c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55094/0 (socket says 172.21.15.196:55094) 2024-04-28T13:32:24.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.333+0000 7f070fe00640 1 -- 172.21.15.196:0/97985472 learned_addr learned my addr 172.21.15.196:0/97985472 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:24.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.333+0000 7f0717e00640 1 --2- 172.21.15.196:0/97985472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0718161220 0x7f07181edbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:24.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.333+0000 7f0717400640 1 --2- 172.21.15.196:0/97985472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f071806ea30 0x7f07180cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:24.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.334+0000 7f070fe00640 1 -- 172.21.15.196:0/97985472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f071806ea30 msgr2=0x7f07180cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:24.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.334+0000 7f070fe00640 1 --2- 172.21.15.196:0/97985472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f071806ea30 0x7f07180cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.334+0000 7f070fe00640 1 -- 172.21.15.196:0/97985472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0718161220 msgr2=0x7f07181edbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:24.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.334+0000 7f070fe00640 1 --2- 172.21.15.196:0/97985472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0718161220 0x7f07181edbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.334+0000 7f070fe00640 1 -- 172.21.15.196:0/97985472 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071815d930 con 0x7f07180b3e70 2024-04-28T13:32:24.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.334+0000 7f070fe00640 1 --2- 172.21.15.196:0/97985472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07180b3e70 0x7f071815c8c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f070000e8a0 tx=0x7f070000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:24.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.334+0000 7f070f400640 1 -- 172.21.15.196:0/97985472 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700017070 con 0x7f07180b3e70 2024-04-28T13:32:24.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.334+0000 7f070f400640 1 -- 172.21.15.196:0/97985472 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f070000c940 con 0x7f07180b3e70 2024-04-28T13:32:24.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.334+0000 7f071e000640 1 -- 172.21.15.196:0/97985472 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07180d0300 con 0x7f07180b3e70 2024-04-28T13:32:24.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.334+0000 7f070f400640 1 -- 172.21.15.196:0/97985472 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070000cd30 con 0x7f07180b3e70 2024-04-28T13:32:24.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.334+0000 7f071e000640 1 -- 172.21.15.196:0/97985472 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07180cd960 con 0x7f07180b3e70 2024-04-28T13:32:24.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.336+0000 7f071e000640 1 -- 172.21.15.196:0/97985472 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07180756c0 con 0x7f07180b3e70 2024-04-28T13:32:24.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.337+0000 7f070f400640 1 -- 172.21.15.196:0/97985472 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f070001a070 con 0x7f07180b3e70 2024-04-28T13:32:24.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.338+0000 7f070f400640 1 --2- 172.21.15.196:0/97985472 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f06f406c8f0 0x7f06f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:24.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.338+0000 7f070f400640 1 -- 172.21.15.196:0/97985472 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f070008b690 con 0x7f07180b3e70 2024-04-28T13:32:24.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.339+0000 7f0717400640 1 --2- 172.21.15.196:0/97985472 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f06f406c8f0 0x7f06f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:24.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.340+0000 7f0717400640 1 --2- 172.21.15.196:0/97985472 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f06f406c8f0 0x7f06f406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0708000f60 tx=0x7f0708009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:24.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.343+0000 7f070f400640 1 -- 172.21.15.196:0/97985472 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f07180756c0 con 0x7f07180b3e70 2024-04-28T13:32:24.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.610+0000 7f071e000640 1 -- 172.21.15.196:0/97985472 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f071806a290 con 0x7f07180b3e70 2024-04-28T13:32:24.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.611+0000 7f070f400640 1 -- 172.21.15.196:0/97985472 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f0700059e10 con 0x7f07180b3e70 2024-04-28T13:32:24.612 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:24.612 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:32:24.612 INFO:teuthology.orchestra.run.smithi196.stdout: "auth_dump": [ 2024-04-28T13:32:24.612 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.612 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "mds.a", 2024-04-28T13:32:24.612 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBkTy5mGDY2OhAAO3Cl3Q176Jk7wky/cM1g7A==", 2024-04-28T13:32:24.612 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.612 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:32:24.612 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow *", 2024-04-28T13:32:24.612 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow *", 2024-04-28T13:32:24.612 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:32:24.612 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.612 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.613 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.613 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "mds.b", 2024-04-28T13:32:24.613 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBlTy5m5wRGFhAAzFebm/R1yHYctJDIlWndyQ==", 2024-04-28T13:32:24.613 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.613 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:32:24.613 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow *", 2024-04-28T13:32:24.613 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow *", 2024-04-28T13:32:24.613 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:32:24.613 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.613 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.613 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.613 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "mds.c", 2024-04-28T13:32:24.613 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBlTy5mJqNwCRAA9EAW936ooX1fIhI8D5Y71w==", 2024-04-28T13:32:24.613 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.614 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:32:24.614 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow *", 2024-04-28T13:32:24.614 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow *", 2024-04-28T13:32:24.614 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:32:24.614 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.614 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.614 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.614 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "mds.d", 2024-04-28T13:32:24.614 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBlTy5m0Bo4JBAASWXwIPgangccm1ErTZtFxg==", 2024-04-28T13:32:24.614 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.614 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:32:24.614 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow *", 2024-04-28T13:32:24.614 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow *", 2024-04-28T13:32:24.614 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:32:24.618 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.618 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.618 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.618 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.0", 2024-04-28T13:32:24.618 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBpTy5mOG48JhAAMO70X2PHMWLcrTFLBuDe1Q==", 2024-04-28T13:32:24.618 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.618 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:32:24.618 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:32:24.618 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:32:24.618 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.618 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.618 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.618 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.1", 2024-04-28T13:32:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBtTy5mADYZBhAARTLL+BqFjPyzcEKny+CBYg==", 2024-04-28T13:32:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:32:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:32:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:32:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.2", 2024-04-28T13:32:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBwTy5m26gJEBAAuBn3BMPn/EZzOLzMeaaX2w==", 2024-04-28T13:32:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:32:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:32:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:32:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.3", 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBzTy5mrx1KKRAAUG56DD6uIyHP7oD0wOpRkQ==", 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.4", 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQB6Ty5mPoFDAhAA/sprBELVpeGanXkX7jCEHw==", 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:32:24.620 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:32:24.621 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:32:24.621 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.621 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.621 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.621 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.5", 2024-04-28T13:32:24.621 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQB9Ty5m77ATCxAA0Fxy75nfDOZjBxd5an+1Gg==", 2024-04-28T13:32:24.621 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.621 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:32:24.621 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:32:24.621 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:32:24.621 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.621 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.621 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.621 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.6", 2024-04-28T13:32:24.622 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQCATy5mrnQ4FhAAmKIZXHzagKbIkyof69CBTA==", 2024-04-28T13:32:24.622 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.622 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:32:24.622 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:32:24.622 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:32:24.622 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.622 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.622 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.622 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.7", 2024-04-28T13:32:24.622 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQCDTy5mSQbQIRAAUHxlY004z6danZMCClG+BQ==", 2024-04-28T13:32:24.622 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.622 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:32:24.622 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:32:24.622 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:32:24.623 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.623 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.623 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.623 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.0", 2024-04-28T13:32:24.623 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBlTy5m8movMBAAnKfzHF0pAkX1F/dZiB0qBg==", 2024-04-28T13:32:24.623 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.623 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:32:24.623 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow r", 2024-04-28T13:32:24.623 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow rw", 2024-04-28T13:32:24.623 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow rwx" 2024-04-28T13:32:24.623 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.623 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.623 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.623 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.1", 2024-04-28T13:32:24.623 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBmTy5m9BmQBhAAQh8TiIkSt7WqdfovcCbPBg==", 2024-04-28T13:32:24.624 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.624 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:32:24.624 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow r", 2024-04-28T13:32:24.624 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow rw", 2024-04-28T13:32:24.624 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow rwx" 2024-04-28T13:32:24.624 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.624 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.624 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.624 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.2", 2024-04-28T13:32:24.624 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBlTy5mpPCYORAAPt0CzLT0pHKQTOTtE3L1xw==", 2024-04-28T13:32:24.624 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.624 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:32:24.624 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow r", 2024-04-28T13:32:24.624 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow rw", 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow rwx" 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.3", 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBmTy5m0oq7DxAAm765AI1ow5e0KlQoBRCr/Q==", 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow r", 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow rw", 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow rwx" 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.admin", 2024-04-28T13:32:24.625 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBjTy5mYyTCORAAp9GesFfEOekDdOFZC8mrhw==", 2024-04-28T13:32:24.626 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.626 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow *", 2024-04-28T13:32:24.626 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow *", 2024-04-28T13:32:24.626 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow *", 2024-04-28T13:32:24.626 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:32:24.626 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.626 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.626 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.626 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.bootstrap-mds", 2024-04-28T13:32:24.626 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQCRTy5maKC4BhAA+sQeuoNUhwu2YVvJAthflA==", 2024-04-28T13:32:24.626 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.626 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile bootstrap-mds" 2024-04-28T13:32:24.626 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.626 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.626 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.627 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.bootstrap-mgr", 2024-04-28T13:32:24.627 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQCRTy5m7re4BhAAWg7TFGHR081mTcvn3l0y8Q==", 2024-04-28T13:32:24.627 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.627 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-28T13:32:24.627 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.627 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.627 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.627 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.bootstrap-osd", 2024-04-28T13:32:24.627 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQCRTy5m0NC4BhAAiA1IAt99QsNE6ewe0AXogQ==", 2024-04-28T13:32:24.627 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.627 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile bootstrap-osd" 2024-04-28T13:32:24.627 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.627 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.627 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.627 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.bootstrap-rbd", 2024-04-28T13:32:24.628 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQCRTy5mnOO4BhAAsWTVdKJFyRJk7YIwHp068g==", 2024-04-28T13:32:24.628 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.628 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-28T13:32:24.628 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.628 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.628 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.628 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-28T13:32:24.628 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQCRTy5mQfa4BhAAywV10KnYfRHXsliKYWv8+g==", 2024-04-28T13:32:24.628 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.628 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-28T13:32:24.628 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.628 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.628 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.628 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.bootstrap-rgw", 2024-04-28T13:32:24.628 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQCRTy5mJAm5BhAAxHdzq+E+PeuA7p/Qm9UDJg==", 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "mgr.x", 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBkTy5mV7E7MhAAtess+f/dkv11zc9Ujl2jqg==", 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow *", 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile mgr", 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "mgr.y", 2024-04-28T13:32:24.629 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBkTy5mh2AXJxAA+BPc+Xi43waIQJ2YjGmw9Q==", 2024-04-28T13:32:24.630 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:32:24.630 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow *", 2024-04-28T13:32:24.630 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile mgr", 2024-04-28T13:32:24.630 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:32:24.630 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.630 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:24.630 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-28T13:32:24.630 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:32:24.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.613+0000 7f071e000640 1 -- 172.21.15.196:0/97985472 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f06f406c8f0 msgr2=0x7f06f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:24.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.613+0000 7f071e000640 1 --2- 172.21.15.196:0/97985472 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f06f406c8f0 0x7f06f406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0708000f60 tx=0x7f0708009040 comp rx=0 tx=0).stop 2024-04-28T13:32:24.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.613+0000 7f071e000640 1 -- 172.21.15.196:0/97985472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07180b3e70 msgr2=0x7f071815c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:24.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.613+0000 7f071e000640 1 --2- 172.21.15.196:0/97985472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07180b3e70 0x7f071815c8c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f070000e8a0 tx=0x7f070000ed60 comp rx=0 tx=0).stop 2024-04-28T13:32:24.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.613+0000 7f071e000640 1 -- 172.21.15.196:0/97985472 shutdown_connections 2024-04-28T13:32:24.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.613+0000 7f071e000640 1 --2- 172.21.15.196:0/97985472 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f06f406c8f0 0x7f06f406edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.613+0000 7f071e000640 1 --2- 172.21.15.196:0/97985472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0718161220 0x7f07181edbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.613+0000 7f071e000640 1 --2- 172.21.15.196:0/97985472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07180b3e70 0x7f071815c8c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.613+0000 7f071e000640 1 --2- 172.21.15.196:0/97985472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f071806ea30 0x7f07180cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.613+0000 7f071e000640 1 -- 172.21.15.196:0/97985472 >> 172.21.15.196:0/97985472 conn(0x7f07180d0d30 msgr2=0x7f0718163910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:24.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.614+0000 7f071e000640 1 -- 172.21.15.196:0/97985472 shutdown_connections 2024-04-28T13:32:24.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.614+0000 7f071e000640 1 -- 172.21.15.196:0/97985472 wait complete. 2024-04-28T13:32:24.635 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-28T13:32:24.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.827+0000 7fb8e3600640 1 Processor -- start 2024-04-28T13:32:24.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.827+0000 7fb8e3600640 1 -- start start 2024-04-28T13:32:24.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.829+0000 7fb8e3600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb8dc06fa50 0x7fb8dc06fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:24.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.829+0000 7fb8e3600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb8dc0731f0 0x7fb8dc073620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:24.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.829+0000 7fb8e3600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8dc0ae8b0 con 0x7fb8dc1485b0 2024-04-28T13:32:24.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.829+0000 7fb8e3600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb8dc073b60 con 0x7fb8dc06fa50 2024-04-28T13:32:24.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.829+0000 7fb8e3600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb8dc0aeb90 con 0x7fb8dc0731f0 2024-04-28T13:32:24.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.829+0000 7fb8d3e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb8dc06fa50 0x7fb8dc06fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:24.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.829+0000 7fb8e0c00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fb8dc1485b0 0x7fb8dc0afc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:58062/0 (socket says 172.21.15.196:58062) 2024-04-28T13:32:24.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.829+0000 7fb8e0c00640 1 -- 172.21.15.196:0/3982641368 learned_addr learned my addr 172.21.15.196:0/3982641368 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:24.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.830+0000 7fb8e1600640 1 --2- 172.21.15.196:0/3982641368 >> v2:172.21.15.203:3300/0 conn(0x7fb8dc0731f0 0x7fb8dc073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:24.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.830+0000 7fb8d3e00640 1 -- 172.21.15.196:0/3982641368 >> v2:172.21.15.203:3300/0 conn(0x7fb8dc0731f0 msgr2=0x7fb8dc073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:24.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.830+0000 7fb8d3e00640 1 --2- 172.21.15.196:0/3982641368 >> v2:172.21.15.203:3300/0 conn(0x7fb8dc0731f0 0x7fb8dc073620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.830+0000 7fb8d3e00640 1 -- 172.21.15.196:0/3982641368 >> v1:172.21.15.196:6789/0 conn(0x7fb8dc1485b0 legacy=0x7fb8dc0afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:24.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.830+0000 7fb8d3e00640 1 -- 172.21.15.196:0/3982641368 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc0aee70 con 0x7fb8dc06fa50 2024-04-28T13:32:24.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.830+0000 7fb8e1600640 1 --2- 172.21.15.196:0/3982641368 >> v2:172.21.15.203:3300/0 conn(0x7fb8dc0731f0 0x7fb8dc073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:24.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.831+0000 7fb8d3e00640 1 --2- 172.21.15.196:0/3982641368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8dc06fa50 0x7fb8dc06fee0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc149020 tx=0x7fb8c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265c350c45ceae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:24.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.831+0000 7fb8dbe00640 1 -- 172.21.15.196:0/3982641368 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4037070 con 0x7fb8dc06fa50 2024-04-28T13:32:24.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.831+0000 7fb8dbe00640 1 -- 172.21.15.196:0/3982641368 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c4032440 con 0x7fb8dc06fa50 2024-04-28T13:32:24.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.831+0000 7fb8dbe00640 1 -- 172.21.15.196:0/3982641368 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4032810 con 0x7fb8dc06fa50 2024-04-28T13:32:24.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.832+0000 7fb8e3600640 1 -- 172.21.15.196:0/3982641368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8dc06fa50 msgr2=0x7fb8dc06fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:24.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.832+0000 7fb8e3600640 1 --2- 172.21.15.196:0/3982641368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8dc06fa50 0x7fb8dc06fee0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc149020 tx=0x7fb8c402a1a0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.832+0000 7fb8e3600640 1 -- 172.21.15.196:0/3982641368 shutdown_connections 2024-04-28T13:32:24.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.832+0000 7fb8e3600640 1 --2- 172.21.15.196:0/3982641368 >> v2:172.21.15.203:3300/0 conn(0x7fb8dc0731f0 0x7fb8dc073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.832+0000 7fb8e3600640 1 --2- 172.21.15.196:0/3982641368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8dc06fa50 0x7fb8dc06fee0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.832+0000 7fb8e3600640 1 -- 172.21.15.196:0/3982641368 >> 172.21.15.196:0/3982641368 conn(0x7fb8dc0d0d30 msgr2=0x7fb8dc0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:24.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.832+0000 7fb8e3600640 1 -- 172.21.15.196:0/3982641368 shutdown_connections 2024-04-28T13:32:24.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.832+0000 7fb8e3600640 1 -- 172.21.15.196:0/3982641368 wait complete. 2024-04-28T13:32:24.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.834+0000 7fb8e3600640 1 Processor -- start 2024-04-28T13:32:24.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.834+0000 7fb8e3600640 1 -- start start 2024-04-28T13:32:24.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.835+0000 7fb8e3600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8dc06fa50 0x7fb8dc132f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:24.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.835+0000 7fb8e3600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8dc0731f0 0x7fb8dc142f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:24.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.835+0000 7fb8e3600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8dc1485b0 0x7fb8dc0b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:24.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.835+0000 7fb8e3600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc0af710 con 0x7fb8dc06fa50 2024-04-28T13:32:24.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.835+0000 7fb8e3600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc0af430 con 0x7fb8dc0731f0 2024-04-28T13:32:24.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.835+0000 7fb8e3600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb8dc0b4cd0 con 0x7fb8dc1485b0 2024-04-28T13:32:24.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.835+0000 7fb8e0c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8dc06fa50 0x7fb8dc132f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:24.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.836+0000 7fb8e0c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8dc06fa50 0x7fb8dc132f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55112/0 (socket says 172.21.15.196:55112) 2024-04-28T13:32:24.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.836+0000 7fb8e0c00640 1 -- 172.21.15.196:0/1596115828 learned_addr learned my addr 172.21.15.196:0/1596115828 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:24.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.836+0000 7fb8d3e00640 1 --2- 172.21.15.196:0/1596115828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8dc0731f0 0x7fb8dc142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:24.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.836+0000 7fb8e0c00640 1 -- 172.21.15.196:0/1596115828 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8dc1485b0 msgr2=0x7fb8dc0b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:24.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.836+0000 7fb8e1600640 1 --2- 172.21.15.196:0/1596115828 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8dc1485b0 0x7fb8dc0b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:24.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.836+0000 7fb8e0c00640 1 --2- 172.21.15.196:0/1596115828 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8dc1485b0 0x7fb8dc0b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.836+0000 7fb8e0c00640 1 -- 172.21.15.196:0/1596115828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8dc0731f0 msgr2=0x7fb8dc142f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:24.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.836+0000 7fb8e0c00640 1 --2- 172.21.15.196:0/1596115828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8dc0731f0 0x7fb8dc142f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:24.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.836+0000 7fb8e0c00640 1 -- 172.21.15.196:0/1596115828 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc15f3b0 con 0x7fb8dc06fa50 2024-04-28T13:32:24.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.836+0000 7fb8e0c00640 1 --2- 172.21.15.196:0/1596115828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8dc06fa50 0x7fb8dc132f50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00c3f0 tx=0x7fb8cc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:24.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.837+0000 7fb8d9600640 1 -- 172.21.15.196:0/1596115828 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc017070 con 0x7fb8dc06fa50 2024-04-28T13:32:24.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.837+0000 7fb8d9600640 1 -- 172.21.15.196:0/1596115828 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8cc005ce0 con 0x7fb8dc06fa50 2024-04-28T13:32:24.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.837+0000 7fb8e3600640 1 -- 172.21.15.196:0/1596115828 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8dc15f670 con 0x7fb8dc06fa50 2024-04-28T13:32:24.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.837+0000 7fb8d9600640 1 -- 172.21.15.196:0/1596115828 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc0072f0 con 0x7fb8dc06fa50 2024-04-28T13:32:24.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.837+0000 7fb8e3600640 1 -- 172.21.15.196:0/1596115828 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8dc0b2440 con 0x7fb8dc06fa50 2024-04-28T13:32:24.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.838+0000 7fb8e3600640 1 -- 172.21.15.196:0/1596115828 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8dc06a290 con 0x7fb8dc06fa50 2024-04-28T13:32:24.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.839+0000 7fb8d9600640 1 -- 172.21.15.196:0/1596115828 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8cc01a070 con 0x7fb8dc06fa50 2024-04-28T13:32:24.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.840+0000 7fb8d9600640 1 --2- 172.21.15.196:0/1596115828 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb8b806c920 0x7fb8b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:24.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.840+0000 7fb8d9600640 1 -- 172.21.15.196:0/1596115828 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fb8cc08ac20 con 0x7fb8dc06fa50 2024-04-28T13:32:24.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.840+0000 7fb8d3e00640 1 --2- 172.21.15.196:0/1596115828 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb8b806c920 0x7fb8b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:24.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.842+0000 7fb8d3e00640 1 --2- 172.21.15.196:0/1596115828 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb8b806c920 0x7fb8b806ede0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4034000 tx=0x7fb8c402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:24.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:24.844+0000 7fb8d9600640 1 -- 172.21.15.196:0/1596115828 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8cc059300 con 0x7fb8dc06fa50 2024-04-28T13:32:25.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.099+0000 7fb8e3600640 1 -- 172.21.15.196:0/1596115828 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fb8dc147af0 con 0x7fb8dc06fa50 2024-04-28T13:32:25.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.118+0000 7fb8d9600640 1 -- 172.21.15.196:0/1596115828 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fb8cc058c00 con 0x7fb8dc06fa50 2024-04-28T13:32:25.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.120+0000 7fb8e3600640 1 -- 172.21.15.196:0/1596115828 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb8b806c920 msgr2=0x7fb8b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:25.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.120+0000 7fb8e3600640 1 --2- 172.21.15.196:0/1596115828 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb8b806c920 0x7fb8b806ede0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4034000 tx=0x7fb8c402e000 comp rx=0 tx=0).stop 2024-04-28T13:32:25.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.120+0000 7fb8e3600640 1 -- 172.21.15.196:0/1596115828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8dc06fa50 msgr2=0x7fb8dc132f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:25.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.120+0000 7fb8e3600640 1 --2- 172.21.15.196:0/1596115828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8dc06fa50 0x7fb8dc132f50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00c3f0 tx=0x7fb8cc00c8b0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.120+0000 7fb8e3600640 1 -- 172.21.15.196:0/1596115828 shutdown_connections 2024-04-28T13:32:25.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.120+0000 7fb8e3600640 1 --2- 172.21.15.196:0/1596115828 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb8b806c920 0x7fb8b806ede0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.120+0000 7fb8e3600640 1 --2- 172.21.15.196:0/1596115828 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8dc1485b0 0x7fb8dc0b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.120+0000 7fb8e3600640 1 --2- 172.21.15.196:0/1596115828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8dc0731f0 0x7fb8dc142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.120+0000 7fb8e3600640 1 --2- 172.21.15.196:0/1596115828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8dc06fa50 0x7fb8dc132f50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.120+0000 7fb8e3600640 1 -- 172.21.15.196:0/1596115828 >> 172.21.15.196:0/1596115828 conn(0x7fb8dc0d0d30 msgr2=0x7fb8dc146bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:25.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.121+0000 7fb8e3600640 1 -- 172.21.15.196:0/1596115828 shutdown_connections 2024-04-28T13:32:25.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.121+0000 7fb8e3600640 1 -- 172.21.15.196:0/1596115828 wait complete. 2024-04-28T13:32:25.141 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-28T13:32:25.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.330+0000 7f2263200640 1 Processor -- start 2024-04-28T13:32:25.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.330+0000 7f2263200640 1 -- start start 2024-04-28T13:32:25.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.332+0000 7f2263200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f226415bc40 0x7f22640d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:25.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.332+0000 7f2263200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f226406f800 0x7f22640d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:25.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.332+0000 7f2263200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2264074240 con 0x7f226415c020 2024-04-28T13:32:25.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.332+0000 7f2263200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f226415fc10 con 0x7f226415bc40 2024-04-28T13:32:25.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.332+0000 7f2263200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f226415fef0 con 0x7f226406f800 2024-04-28T13:32:25.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.332+0000 7f2261400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f226415c020 0x7f22640b4a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:58078/0 (socket says 172.21.15.196:58078) 2024-04-28T13:32:25.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.332+0000 7f2261400640 1 -- 172.21.15.196:0/1300368500 learned_addr learned my addr 172.21.15.196:0/1300368500 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:25.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.332+0000 7f2261e00640 1 --2- 172.21.15.196:0/1300368500 >> v2:172.21.15.203:3300/0 conn(0x7f226415bc40 0x7f22640d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:25.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2262800640 1 --2- 172.21.15.196:0/1300368500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f226406f800 0x7f22640d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:25.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 -- 172.21.15.196:0/1300368500 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 204192231 0 0) 0x7f2264074240 con 0x7f226415c020 2024-04-28T13:32:25.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 -- 172.21.15.196:0/1300368500 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2248003520 con 0x7f226415c020 2024-04-28T13:32:25.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 -- 172.21.15.196:0/1300368500 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 478623662 0 0) 0x7f2248003520 con 0x7f226415c020 2024-04-28T13:32:25.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 -- 172.21.15.196:0/1300368500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f226406f800 msgr2=0x7f22640d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:25.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 --2- 172.21.15.196:0/1300368500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f226406f800 0x7f22640d15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 -- 172.21.15.196:0/1300368500 >> v2:172.21.15.203:3300/0 conn(0x7f226415bc40 msgr2=0x7f22640d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:25.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 --2- 172.21.15.196:0/1300368500 >> v2:172.21.15.203:3300/0 conn(0x7f226415bc40 0x7f22640d1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 -- 172.21.15.196:0/1300368500 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22641601d0 con 0x7f226415c020 2024-04-28T13:32:25.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 -- 172.21.15.196:0/1300368500 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f2250001670 con 0x7f226415c020 2024-04-28T13:32:25.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 -- 172.21.15.196:0/1300368500 >> v1:172.21.15.196:6789/0 conn(0x7f226415c020 legacy=0x7f22640b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:25.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 --2- 172.21.15.196:0/1300368500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22480039b0 0x7f2248003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:25.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 --2- 172.21.15.196:0/1300368500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2248004800 0x7f2248004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:25.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 --2- 172.21.15.196:0/1300368500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22480051c0 0x7f224800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:25.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 -- 172.21.15.196:0/1300368500 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f226415fc10 con 0x7f22480051c0 2024-04-28T13:32:25.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 -- 172.21.15.196:0/1300368500 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f226415fef0 con 0x7f22480039b0 2024-04-28T13:32:25.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2260a00640 1 -- 172.21.15.196:0/1300368500 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f224800e190 con 0x7f2248004800 2024-04-28T13:32:25.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.333+0000 7f2262800640 1 --2- 172.21.15.196:0/1300368500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f226406f800 0x7f22640d15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:32:25.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.334+0000 7f2262800640 1 --2- 172.21.15.196:0/1300368500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22480051c0 0x7f224800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:25.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.334+0000 7f2261e00640 1 --2- 172.21.15.196:0/1300368500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22480039b0 0x7f2248003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:25.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.334+0000 7f2262800640 1 -- 172.21.15.196:0/1300368500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2248004800 msgr2=0x7f2248004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:25.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.334+0000 7f2261400640 1 --2- 172.21.15.196:0/1300368500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2248004800 0x7f2248004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:25.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.334+0000 7f2262800640 1 --2- 172.21.15.196:0/1300368500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2248004800 0x7f2248004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.334+0000 7f2262800640 1 -- 172.21.15.196:0/1300368500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22480039b0 msgr2=0x7f2248003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:25.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.334+0000 7f2262800640 1 --2- 172.21.15.196:0/1300368500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22480039b0 0x7f2248003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.334+0000 7f2262800640 1 -- 172.21.15.196:0/1300368500 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2248003520 con 0x7f22480051c0 2024-04-28T13:32:25.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.334+0000 7f2261400640 1 --2- 172.21.15.196:0/1300368500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2248004800 0x7f2248004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:25.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.334+0000 7f2261e00640 1 --2- 172.21.15.196:0/1300368500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22480039b0 0x7f2248003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:32:25.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.335+0000 7f2262800640 1 --2- 172.21.15.196:0/1300368500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22480051c0 0x7f224800b9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f226406fbe0 tx=0x7f2248005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e399043917794d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:25.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.336+0000 7f2260a00640 1 -- 172.21.15.196:0/1300368500 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c036070 con 0x7f22480051c0 2024-04-28T13:32:25.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.336+0000 7f2260a00640 1 -- 172.21.15.196:0/1300368500 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f225c003b90 con 0x7f22480051c0 2024-04-28T13:32:25.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.336+0000 7f2260a00640 1 -- 172.21.15.196:0/1300368500 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c009b60 con 0x7f22480051c0 2024-04-28T13:32:25.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.336+0000 7f2263200640 1 -- 172.21.15.196:0/1300368500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22480051c0 msgr2=0x7f224800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:25.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.336+0000 7f2263200640 1 --2- 172.21.15.196:0/1300368500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22480051c0 0x7f224800b9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f226406fbe0 tx=0x7f2248005830 comp rx=0 tx=0).stop 2024-04-28T13:32:25.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.336+0000 7f2262800640 1 -- 172.21.15.196:0/1300368500 reap_dead start 2024-04-28T13:32:25.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.336+0000 7f2263200640 1 -- 172.21.15.196:0/1300368500 shutdown_connections 2024-04-28T13:32:25.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.336+0000 7f2263200640 1 -- 172.21.15.196:0/1300368500 >> 172.21.15.196:0/1300368500 conn(0x7f226406c6c0 msgr2=0x7f226406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:25.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.336+0000 7f2263200640 1 -- 172.21.15.196:0/1300368500 shutdown_connections 2024-04-28T13:32:25.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.336+0000 7f2263200640 1 -- 172.21.15.196:0/1300368500 wait complete. 2024-04-28T13:32:25.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.338+0000 7f2263200640 1 Processor -- start 2024-04-28T13:32:25.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.338+0000 7f2263200640 1 -- start start 2024-04-28T13:32:25.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.339+0000 7f2263200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f226414fe80 0x7f2264150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:25.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.339+0000 7f2263200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2264158f00 0x7f22641517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:25.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.339+0000 7f2263200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2264151ce0 0x7f226409e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:25.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.339+0000 7f2263200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22640b0580 con 0x7f2264158f00 2024-04-28T13:32:25.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.339+0000 7f2263200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f22640ae730 con 0x7f2264151ce0 2024-04-28T13:32:25.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.339+0000 7f2263200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f22640b0700 con 0x7f226414fe80 2024-04-28T13:32:25.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.339+0000 7f2261400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2264158f00 0x7f22641517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:25.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.339+0000 7f2261400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2264158f00 0x7f22641517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55130/0 (socket says 172.21.15.196:55130) 2024-04-28T13:32:25.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.339+0000 7f2261400640 1 -- 172.21.15.196:0/1782495476 learned_addr learned my addr 172.21.15.196:0/1782495476 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:25.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.340+0000 7f2261e00640 1 --2- 172.21.15.196:0/1782495476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f226414fe80 0x7f2264150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:25.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.340+0000 7f2261400640 1 -- 172.21.15.196:0/1782495476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f226414fe80 msgr2=0x7f2264150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:25.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.340+0000 7f2261400640 1 --2- 172.21.15.196:0/1782495476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f226414fe80 0x7f2264150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.340+0000 7f2261400640 1 -- 172.21.15.196:0/1782495476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2264151ce0 msgr2=0x7f226409e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:25.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.340+0000 7f2261400640 1 --2- 172.21.15.196:0/1782495476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2264151ce0 0x7f226409e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.340+0000 7f2261400640 1 -- 172.21.15.196:0/1782495476 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2264159340 con 0x7f2264158f00 2024-04-28T13:32:25.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.340+0000 7f2261400640 1 --2- 172.21.15.196:0/1782495476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2264158f00 0x7f22641517a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f225000db40 tx=0x7f2250001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:25.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.340+0000 7f225a000640 1 -- 172.21.15.196:0/1782495476 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250019070 con 0x7f2264158f00 2024-04-28T13:32:25.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.340+0000 7f225a000640 1 -- 172.21.15.196:0/1782495476 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2250004cb0 con 0x7f2264158f00 2024-04-28T13:32:25.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.340+0000 7f225a000640 1 -- 172.21.15.196:0/1782495476 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250004e60 con 0x7f2264158f00 2024-04-28T13:32:25.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.340+0000 7f2263200640 1 -- 172.21.15.196:0/1782495476 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2264154c70 con 0x7f2264158f00 2024-04-28T13:32:25.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.341+0000 7f2263200640 1 -- 172.21.15.196:0/1782495476 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2264155230 con 0x7f2264158f00 2024-04-28T13:32:25.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.342+0000 7f2263200640 1 -- 172.21.15.196:0/1782495476 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f226409f0f0 con 0x7f2264158f00 2024-04-28T13:32:25.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.343+0000 7f225a000640 1 -- 172.21.15.196:0/1782495476 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22500072d0 con 0x7f2264158f00 2024-04-28T13:32:25.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.344+0000 7f225a000640 1 --2- 172.21.15.196:0/1782495476 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f224406c920 0x7f224406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:25.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.344+0000 7f225a000640 1 -- 172.21.15.196:0/1782495476 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f225008b5d0 con 0x7f2264158f00 2024-04-28T13:32:25.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.344+0000 7f2261e00640 1 --2- 172.21.15.196:0/1782495476 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f224406c920 0x7f224406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:25.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.346+0000 7f2261e00640 1 --2- 172.21.15.196:0/1782495476 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f224406c920 0x7f224406ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2254009930 tx=0x7f2254008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:25.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.348+0000 7f225a000640 1 -- 172.21.15.196:0/1782495476 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2264159340 con 0x7f2264158f00 2024-04-28T13:32:25.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.628+0000 7f2263200640 1 -- 172.21.15.196:0/1782495476 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f22640507a0 con 0x7f2264158f00 2024-04-28T13:32:25.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.632+0000 7f225a000640 1 -- 172.21.15.196:0/1782495476 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f2250059660 con 0x7f2264158f00 2024-04-28T13:32:25.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.634+0000 7f2263200640 1 -- 172.21.15.196:0/1782495476 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f224406c920 msgr2=0x7f224406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:25.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.634+0000 7f2263200640 1 --2- 172.21.15.196:0/1782495476 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f224406c920 0x7f224406ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2254009930 tx=0x7f2254008040 comp rx=0 tx=0).stop 2024-04-28T13:32:25.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.634+0000 7f2263200640 1 -- 172.21.15.196:0/1782495476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2264158f00 msgr2=0x7f22641517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:25.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.634+0000 7f2263200640 1 --2- 172.21.15.196:0/1782495476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2264158f00 0x7f22641517a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f225000db40 tx=0x7f2250001da0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.634+0000 7f2263200640 1 -- 172.21.15.196:0/1782495476 shutdown_connections 2024-04-28T13:32:25.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.634+0000 7f2263200640 1 --2- 172.21.15.196:0/1782495476 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f224406c920 0x7f224406ede0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.634+0000 7f2263200640 1 --2- 172.21.15.196:0/1782495476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2264151ce0 0x7f226409e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.634+0000 7f2263200640 1 --2- 172.21.15.196:0/1782495476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2264158f00 0x7f22641517a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.634+0000 7f2263200640 1 --2- 172.21.15.196:0/1782495476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f226414fe80 0x7f2264150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.634+0000 7f2263200640 1 -- 172.21.15.196:0/1782495476 >> 172.21.15.196:0/1782495476 conn(0x7f226406c6c0 msgr2=0x7f226415d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:25.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.635+0000 7f2263200640 1 -- 172.21.15.196:0/1782495476 shutdown_connections 2024-04-28T13:32:25.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.635+0000 7f2263200640 1 -- 172.21.15.196:0/1782495476 wait complete. 2024-04-28T13:32:25.655 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-28T13:32:25.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.851+0000 7fa51b800640 1 Processor -- start 2024-04-28T13:32:25.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.851+0000 7fa51b800640 1 -- start start 2024-04-28T13:32:25.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.853+0000 7fa51b800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa51413ff70 0x7fa514146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:25.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.853+0000 7fa51b800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa5140cff20 0x7fa5141465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:25.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.853+0000 7fa51b800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5140d05b0 con 0x7fa514140350 2024-04-28T13:32:25.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.853+0000 7fa51b800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa5140d0890 con 0x7fa51413ff70 2024-04-28T13:32:25.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.853+0000 7fa51b800640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa51406e450 con 0x7fa5140cff20 2024-04-28T13:32:25.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.853+0000 7fa518e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa51413ff70 0x7fa514146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:25.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.853+0000 7fa513e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa514140350 0x7fa5140cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:58086/0 (socket says 172.21.15.196:58086) 2024-04-28T13:32:25.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.854+0000 7fa513e00640 1 -- 172.21.15.196:0/540284248 learned_addr learned my addr 172.21.15.196:0/540284248 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:25.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.854+0000 7fa519800640 1 --2- 172.21.15.196:0/540284248 >> v2:172.21.15.203:3300/0 conn(0x7fa5140cff20 0x7fa5141465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:25.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.854+0000 7fa518e00640 1 -- 172.21.15.196:0/540284248 >> v2:172.21.15.203:3300/0 conn(0x7fa5140cff20 msgr2=0x7fa5141465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:25.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.854+0000 7fa518e00640 1 --2- 172.21.15.196:0/540284248 >> v2:172.21.15.203:3300/0 conn(0x7fa5140cff20 0x7fa5141465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.854+0000 7fa513400640 1 -- 172.21.15.196:0/540284248 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1570623075 0 0) 0x7fa5140d05b0 con 0x7fa514140350 2024-04-28T13:32:25.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.854+0000 7fa518e00640 1 -- 172.21.15.196:0/540284248 >> v1:172.21.15.196:6789/0 conn(0x7fa514140350 legacy=0x7fa5140cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:25.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.854+0000 7fa518e00640 1 -- 172.21.15.196:0/540284248 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51406e730 con 0x7fa51413ff70 2024-04-28T13:32:25.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.854+0000 7fa519800640 1 --2- 172.21.15.196:0/540284248 >> v2:172.21.15.203:3300/0 conn(0x7fa5140cff20 0x7fa5141465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:25.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.855+0000 7fa518e00640 1 --2- 172.21.15.196:0/540284248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa51413ff70 0x7fa514146080 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa5140cec30 tx=0x7fa50402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1927eaaab69d9b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:25.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.856+0000 7fa513400640 1 -- 172.21.15.196:0/540284248 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504037070 con 0x7fa51413ff70 2024-04-28T13:32:25.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.856+0000 7fa513400640 1 -- 172.21.15.196:0/540284248 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa504032440 con 0x7fa51413ff70 2024-04-28T13:32:25.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.856+0000 7fa513400640 1 -- 172.21.15.196:0/540284248 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50402ae60 con 0x7fa51413ff70 2024-04-28T13:32:25.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.856+0000 7fa51b800640 1 -- 172.21.15.196:0/540284248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa51413ff70 msgr2=0x7fa514146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:25.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.856+0000 7fa51b800640 1 --2- 172.21.15.196:0/540284248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa51413ff70 0x7fa514146080 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa5140cec30 tx=0x7fa50402a110 comp rx=0 tx=0).stop 2024-04-28T13:32:25.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.856+0000 7fa51b800640 1 -- 172.21.15.196:0/540284248 shutdown_connections 2024-04-28T13:32:25.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.856+0000 7fa51b800640 1 --2- 172.21.15.196:0/540284248 >> v2:172.21.15.203:3300/0 conn(0x7fa5140cff20 0x7fa5141465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.856+0000 7fa51b800640 1 --2- 172.21.15.196:0/540284248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa51413ff70 0x7fa514146080 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.856+0000 7fa51b800640 1 -- 172.21.15.196:0/540284248 >> 172.21.15.196:0/540284248 conn(0x7fa5140d0d30 msgr2=0x7fa5140d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:25.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.856+0000 7fa51b800640 1 -- 172.21.15.196:0/540284248 shutdown_connections 2024-04-28T13:32:25.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.856+0000 7fa51b800640 1 -- 172.21.15.196:0/540284248 wait complete. 2024-04-28T13:32:25.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.858+0000 7fa51b800640 1 Processor -- start 2024-04-28T13:32:25.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.858+0000 7fa51b800640 1 -- start start 2024-04-28T13:32:25.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.859+0000 7fa51b800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5140cff20 0x7fa514145010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:25.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.859+0000 7fa51b800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa51413ff70 0x7fa514142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:25.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.859+0000 7fa51b800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa514140350 0x7fa514159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:25.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.859+0000 7fa51b800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa51406efe0 con 0x7fa514140350 2024-04-28T13:32:25.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.859+0000 7fa51b800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa51406ee60 con 0x7fa5140cff20 2024-04-28T13:32:25.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.859+0000 7fa51b800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa51406f160 con 0x7fa51413ff70 2024-04-28T13:32:25.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.860+0000 7fa519800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa514140350 0x7fa514159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:25.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.860+0000 7fa519800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa514140350 0x7fa514159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55140/0 (socket says 172.21.15.196:55140) 2024-04-28T13:32:25.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.860+0000 7fa519800640 1 -- 172.21.15.196:0/3717053718 learned_addr learned my addr 172.21.15.196:0/3717053718 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:25.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.860+0000 7fa518e00640 1 --2- 172.21.15.196:0/3717053718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5140cff20 0x7fa514145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:25.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.860+0000 7fa513e00640 1 --2- 172.21.15.196:0/3717053718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa51413ff70 0x7fa514142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:25.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.860+0000 7fa519800640 1 -- 172.21.15.196:0/3717053718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa51413ff70 msgr2=0x7fa514142c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:25.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.860+0000 7fa519800640 1 --2- 172.21.15.196:0/3717053718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa51413ff70 0x7fa514142c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.860+0000 7fa519800640 1 -- 172.21.15.196:0/3717053718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5140cff20 msgr2=0x7fa514145010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:25.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.860+0000 7fa519800640 1 --2- 172.21.15.196:0/3717053718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5140cff20 0x7fa514145010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:25.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.860+0000 7fa519800640 1 -- 172.21.15.196:0/3717053718 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5141f4ff0 con 0x7fa514140350 2024-04-28T13:32:25.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.861+0000 7fa519800640 1 --2- 172.21.15.196:0/3717053718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa514140350 0x7fa514159590 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00c820 tx=0x7fa50c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:25.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.861+0000 7fa510c00640 1 -- 172.21.15.196:0/3717053718 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c015070 con 0x7fa514140350 2024-04-28T13:32:25.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.861+0000 7fa510c00640 1 -- 172.21.15.196:0/3717053718 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa50c007a30 con 0x7fa514140350 2024-04-28T13:32:25.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.861+0000 7fa510c00640 1 -- 172.21.15.196:0/3717053718 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c007e20 con 0x7fa514140350 2024-04-28T13:32:25.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.861+0000 7fa51b800640 1 -- 172.21.15.196:0/3717053718 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5141f5dd0 con 0x7fa514140350 2024-04-28T13:32:25.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.861+0000 7fa51b800640 1 -- 172.21.15.196:0/3717053718 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5141f5850 con 0x7fa514140350 2024-04-28T13:32:25.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.862+0000 7fa51b800640 1 -- 172.21.15.196:0/3717053718 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa51406a290 con 0x7fa514140350 2024-04-28T13:32:25.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.863+0000 7fa510c00640 1 -- 172.21.15.196:0/3717053718 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa50c018070 con 0x7fa514140350 2024-04-28T13:32:25.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.864+0000 7fa510c00640 1 --2- 172.21.15.196:0/3717053718 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa4f006c8f0 0x7fa4f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:25.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.864+0000 7fa510c00640 1 -- 172.21.15.196:0/3717053718 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fa50c08b7d0 con 0x7fa514140350 2024-04-28T13:32:25.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.865+0000 7fa518e00640 1 --2- 172.21.15.196:0/3717053718 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa4f006c8f0 0x7fa4f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:25.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.866+0000 7fa518e00640 1 --2- 172.21.15.196:0/3717053718 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa4f006c8f0 0x7fa4f006edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa504034000 tx=0x7fa50402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:25.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:25.869+0000 7fa510c00640 1 -- 172.21.15.196:0/3717053718 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa50c059eb0 con 0x7fa514140350 2024-04-28T13:32:26.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.140+0000 7fa51b800640 1 -- 172.21.15.196:0/3717053718 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fa5140b45e0 con 0x7fa514140350 2024-04-28T13:32:26.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.145+0000 7fa510c00640 1 -- 172.21.15.196:0/3717053718 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fa50c0597b0 con 0x7fa514140350 2024-04-28T13:32:26.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.148+0000 7fa51b800640 1 -- 172.21.15.196:0/3717053718 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa4f006c8f0 msgr2=0x7fa4f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:26.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.148+0000 7fa51b800640 1 --2- 172.21.15.196:0/3717053718 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa4f006c8f0 0x7fa4f006edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa504034000 tx=0x7fa50402e000 comp rx=0 tx=0).stop 2024-04-28T13:32:26.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.148+0000 7fa51b800640 1 -- 172.21.15.196:0/3717053718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa514140350 msgr2=0x7fa514159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:26.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.148+0000 7fa51b800640 1 --2- 172.21.15.196:0/3717053718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa514140350 0x7fa514159590 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00c820 tx=0x7fa50c00cce0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.148+0000 7fa51b800640 1 -- 172.21.15.196:0/3717053718 shutdown_connections 2024-04-28T13:32:26.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.148+0000 7fa51b800640 1 --2- 172.21.15.196:0/3717053718 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa4f006c8f0 0x7fa4f006edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.148+0000 7fa51b800640 1 --2- 172.21.15.196:0/3717053718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa514140350 0x7fa514159590 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.148+0000 7fa51b800640 1 --2- 172.21.15.196:0/3717053718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa51413ff70 0x7fa514142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.148+0000 7fa51b800640 1 --2- 172.21.15.196:0/3717053718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5140cff20 0x7fa514145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.148+0000 7fa51b800640 1 -- 172.21.15.196:0/3717053718 >> 172.21.15.196:0/3717053718 conn(0x7fa5140d0d30 msgr2=0x7fa5140b52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:26.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.148+0000 7fa51b800640 1 -- 172.21.15.196:0/3717053718 shutdown_connections 2024-04-28T13:32:26.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.149+0000 7fa51b800640 1 -- 172.21.15.196:0/3717053718 wait complete. 2024-04-28T13:32:26.168 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-28T13:32:26.362 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.361+0000 7fdea3000640 1 Processor -- start 2024-04-28T13:32:26.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.362+0000 7fdea3000640 1 -- start start 2024-04-28T13:32:26.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.363+0000 7fdea3000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde9c0d1a90 0x7fde9c15f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:26.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.363+0000 7fdea3000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fde9c0d0df0 0x7fde9c15f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:26.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.363+0000 7fdea3000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde9c0d5e60 con 0x7fde9c0d1e70 2024-04-28T13:32:26.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.363+0000 7fdea3000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fde9c0d5c10 con 0x7fde9c0d0df0 2024-04-28T13:32:26.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.363+0000 7fdea3000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fde9c0d38e0 con 0x7fde9c0d1a90 2024-04-28T13:32:26.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.363+0000 7fdea1200640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fde9c0d1e70 0x7fde9c1632c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:58100/0 (socket says 172.21.15.196:58100) 2024-04-28T13:32:26.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.363+0000 7fdea1200640 1 -- 172.21.15.196:0/1144910140 learned_addr learned my addr 172.21.15.196:0/1144910140 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:26.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.364+0000 7fdea2600640 1 --2- 172.21.15.196:0/1144910140 >> v2:172.21.15.203:3300/0 conn(0x7fde9c0d0df0 0x7fde9c15f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:26.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.364+0000 7fdea1c00640 1 --2- 172.21.15.196:0/1144910140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde9c0d1a90 0x7fde9c15f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:26.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.364+0000 7fde9be00640 1 -- 172.21.15.196:0/1144910140 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2981509906 0 0) 0x7fde9c0d5e60 con 0x7fde9c0d1e70 2024-04-28T13:32:26.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.364+0000 7fde9be00640 1 -- 172.21.15.196:0/1144910140 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde80003520 con 0x7fde9c0d1e70 2024-04-28T13:32:26.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.364+0000 7fde9be00640 1 -- 172.21.15.196:0/1144910140 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 577731245 0 0) 0x7fde80003520 con 0x7fde9c0d1e70 2024-04-28T13:32:26.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.364+0000 7fde9be00640 1 -- 172.21.15.196:0/1144910140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde9c0d1a90 msgr2=0x7fde9c15f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:26.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.364+0000 7fde9be00640 1 --2- 172.21.15.196:0/1144910140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde9c0d1a90 0x7fde9c15f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.364+0000 7fde9be00640 1 -- 172.21.15.196:0/1144910140 >> v2:172.21.15.203:3300/0 conn(0x7fde9c0d0df0 msgr2=0x7fde9c15f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:26.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.364+0000 7fde9be00640 1 --2- 172.21.15.196:0/1144910140 >> v2:172.21.15.203:3300/0 conn(0x7fde9c0d0df0 0x7fde9c15f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.364+0000 7fde9be00640 1 -- 172.21.15.196:0/1144910140 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c0d3b30 con 0x7fde9c0d1e70 2024-04-28T13:32:26.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.364+0000 7fde9be00640 1 -- 172.21.15.196:0/1144910140 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fde880019f0 con 0x7fde9c0d1e70 2024-04-28T13:32:26.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.364+0000 7fde9be00640 1 -- 172.21.15.196:0/1144910140 >> v1:172.21.15.196:6789/0 conn(0x7fde9c0d1e70 legacy=0x7fde9c1632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:26.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fde9be00640 1 --2- 172.21.15.196:0/1144910140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde800039b0 0x7fde80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:26.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fde9be00640 1 --2- 172.21.15.196:0/1144910140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde9c0d1a90 0x7fde80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:26.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fde9be00640 1 --2- 172.21.15.196:0/1144910140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde80004d40 0x7fde800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:26.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fde9be00640 1 -- 172.21.15.196:0/1144910140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c0d5c10 con 0x7fde800039b0 2024-04-28T13:32:26.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fde9be00640 1 -- 172.21.15.196:0/1144910140 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c0d38e0 con 0x7fde80004d40 2024-04-28T13:32:26.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fde9be00640 1 -- 172.21.15.196:0/1144910140 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fde8000b9c0 con 0x7fde9c0d1a90 2024-04-28T13:32:26.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fdea1c00640 1 --2- 172.21.15.196:0/1144910140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde800039b0 0x7fde80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:26.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fdea2600640 1 --2- 172.21.15.196:0/1144910140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde80004d40 0x7fde800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:26.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fdea1200640 1 --2- 172.21.15.196:0/1144910140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde9c0d1a90 0x7fde80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:26.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fdea1c00640 1 -- 172.21.15.196:0/1144910140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde9c0d1a90 msgr2=0x7fde80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:26.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fdea1c00640 1 --2- 172.21.15.196:0/1144910140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde9c0d1a90 0x7fde80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fdea1c00640 1 -- 172.21.15.196:0/1144910140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde80004d40 msgr2=0x7fde800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:26.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fdea1c00640 1 --2- 172.21.15.196:0/1144910140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde80004d40 0x7fde800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fdea1c00640 1 -- 172.21.15.196:0/1144910140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde80003520 con 0x7fde800039b0 2024-04-28T13:32:26.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fdea2600640 1 --2- 172.21.15.196:0/1144910140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde80004d40 0x7fde800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:26.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.365+0000 7fdea1200640 1 --2- 172.21.15.196:0/1144910140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde9c0d1a90 0x7fde80004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:26.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.366+0000 7fdea1c00640 1 --2- 172.21.15.196:0/1144910140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde800039b0 0x7fde80003dd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009ac0 tx=0x7fde80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7e1df8d1cc88cf server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:26.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.367+0000 7fde9be00640 1 -- 172.21.15.196:0/1144910140 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c036070 con 0x7fde800039b0 2024-04-28T13:32:26.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.367+0000 7fde9be00640 1 -- 172.21.15.196:0/1144910140 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde8c031400 con 0x7fde800039b0 2024-04-28T13:32:26.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.367+0000 7fde9be00640 1 -- 172.21.15.196:0/1144910140 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c009690 con 0x7fde800039b0 2024-04-28T13:32:26.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.367+0000 7fdea3000640 1 -- 172.21.15.196:0/1144910140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde800039b0 msgr2=0x7fde80003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:26.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.367+0000 7fdea3000640 1 --2- 172.21.15.196:0/1144910140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde800039b0 0x7fde80003dd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009ac0 tx=0x7fde80005830 comp rx=0 tx=0).stop 2024-04-28T13:32:26.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.367+0000 7fdea2600640 1 -- 172.21.15.196:0/1144910140 reap_dead start 2024-04-28T13:32:26.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.367+0000 7fdea3000640 1 -- 172.21.15.196:0/1144910140 shutdown_connections 2024-04-28T13:32:26.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.367+0000 7fdea3000640 1 -- 172.21.15.196:0/1144910140 >> 172.21.15.196:0/1144910140 conn(0x7fde9c0d50f0 msgr2=0x7fde9c0d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:26.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.367+0000 7fdea3000640 1 -- 172.21.15.196:0/1144910140 shutdown_connections 2024-04-28T13:32:26.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.368+0000 7fdea3000640 1 -- 172.21.15.196:0/1144910140 wait complete. 2024-04-28T13:32:26.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.369+0000 7fdea3000640 1 Processor -- start 2024-04-28T13:32:26.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.369+0000 7fdea3000640 1 -- start start 2024-04-28T13:32:26.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.370+0000 7fdea3000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde800039b0 0x7fde9c155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:26.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.370+0000 7fdea3000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c0b3590 0x7fde9c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:26.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.370+0000 7fdea3000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde9c0b3a10 0x7fde9c06d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:26.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.370+0000 7fdea3000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c0d3ea0 con 0x7fde9c0b3590 2024-04-28T13:32:26.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.370+0000 7fdea3000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c0b6910 con 0x7fde800039b0 2024-04-28T13:32:26.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.370+0000 7fdea3000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fde9c0b5c60 con 0x7fde9c0b3a10 2024-04-28T13:32:26.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.370+0000 7fdea1200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c0b3590 0x7fde9c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:26.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.371+0000 7fdea1200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c0b3590 0x7fde9c06d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55148/0 (socket says 172.21.15.196:55148) 2024-04-28T13:32:26.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.371+0000 7fdea1200640 1 -- 172.21.15.196:0/1375648163 learned_addr learned my addr 172.21.15.196:0/1375648163 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:26.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.371+0000 7fdea1c00640 1 --2- 172.21.15.196:0/1375648163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde800039b0 0x7fde9c155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:26.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.371+0000 7fdea1200640 1 -- 172.21.15.196:0/1375648163 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde9c0b3a10 msgr2=0x7fde9c06d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:26.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.371+0000 7fdea1200640 1 --2- 172.21.15.196:0/1375648163 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde9c0b3a10 0x7fde9c06d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.371+0000 7fdea1200640 1 -- 172.21.15.196:0/1375648163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde800039b0 msgr2=0x7fde9c155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:26.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.371+0000 7fdea1200640 1 --2- 172.21.15.196:0/1375648163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde800039b0 0x7fde9c155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.371+0000 7fdea1200640 1 -- 172.21.15.196:0/1375648163 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c0b2b50 con 0x7fde9c0b3590 2024-04-28T13:32:26.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.371+0000 7fdea1c00640 1 --2- 172.21.15.196:0/1375648163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde800039b0 0x7fde9c155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:26.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.371+0000 7fdea1200640 1 --2- 172.21.15.196:0/1375648163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c0b3590 0x7fde9c06d260 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fde880012e0 tx=0x7fde8800b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:26.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.372+0000 7fde99600640 1 -- 172.21.15.196:0/1375648163 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88019070 con 0x7fde9c0b3590 2024-04-28T13:32:26.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.372+0000 7fde99600640 1 -- 172.21.15.196:0/1375648163 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde88004d60 con 0x7fde9c0b3590 2024-04-28T13:32:26.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.372+0000 7fde99600640 1 -- 172.21.15.196:0/1375648163 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde880072d0 con 0x7fde9c0b3590 2024-04-28T13:32:26.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.372+0000 7fdea3000640 1 -- 172.21.15.196:0/1375648163 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c0a1890 con 0x7fde9c0b3590 2024-04-28T13:32:26.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.372+0000 7fdea3000640 1 -- 172.21.15.196:0/1375648163 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c0a14b0 con 0x7fde9c0b3590 2024-04-28T13:32:26.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.373+0000 7fdea3000640 1 -- 172.21.15.196:0/1375648163 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde9c0a3030 con 0x7fde9c0b3590 2024-04-28T13:32:26.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.374+0000 7fde99600640 1 -- 172.21.15.196:0/1375648163 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde8801c070 con 0x7fde9c0b3590 2024-04-28T13:32:26.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.375+0000 7fde99600640 1 --2- 172.21.15.196:0/1375648163 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fde7c06c8f0 0x7fde7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:26.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.375+0000 7fde99600640 1 -- 172.21.15.196:0/1375648163 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fde8808ba40 con 0x7fde9c0b3590 2024-04-28T13:32:26.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.375+0000 7fdea1c00640 1 --2- 172.21.15.196:0/1375648163 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fde7c06c8f0 0x7fde7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:26.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.377+0000 7fdea1c00640 1 --2- 172.21.15.196:0/1375648163 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fde7c06c8f0 0x7fde7c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fde8c033000 tx=0x7fde8c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:26.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.379+0000 7fde99600640 1 -- 172.21.15.196:0/1375648163 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fde8805a120 con 0x7fde9c0b3590 2024-04-28T13:32:26.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.652+0000 7fdea3000640 1 -- 172.21.15.196:0/1375648163 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fde9c0507a0 con 0x7fde9c0b3590 2024-04-28T13:32:26.658 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.657+0000 7fde99600640 1 -- 172.21.15.196:0/1375648163 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fde88059a20 con 0x7fde9c0b3590 2024-04-28T13:32:26.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.659+0000 7fdea3000640 1 -- 172.21.15.196:0/1375648163 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fde7c06c8f0 msgr2=0x7fde7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:26.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.659+0000 7fdea3000640 1 --2- 172.21.15.196:0/1375648163 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fde7c06c8f0 0x7fde7c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fde8c033000 tx=0x7fde8c008030 comp rx=0 tx=0).stop 2024-04-28T13:32:26.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.659+0000 7fdea3000640 1 -- 172.21.15.196:0/1375648163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c0b3590 msgr2=0x7fde9c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:26.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.659+0000 7fdea3000640 1 --2- 172.21.15.196:0/1375648163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c0b3590 0x7fde9c06d260 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fde880012e0 tx=0x7fde8800b560 comp rx=0 tx=0).stop 2024-04-28T13:32:26.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.660+0000 7fdea3000640 1 -- 172.21.15.196:0/1375648163 shutdown_connections 2024-04-28T13:32:26.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.660+0000 7fdea3000640 1 --2- 172.21.15.196:0/1375648163 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fde7c06c8f0 0x7fde7c06edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.660+0000 7fdea3000640 1 --2- 172.21.15.196:0/1375648163 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde9c0b3a10 0x7fde9c06d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.660+0000 7fdea3000640 1 --2- 172.21.15.196:0/1375648163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c0b3590 0x7fde9c06d260 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.662 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.660+0000 7fdea3000640 1 --2- 172.21.15.196:0/1375648163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde800039b0 0x7fde9c155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.662 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.660+0000 7fdea3000640 1 -- 172.21.15.196:0/1375648163 >> 172.21.15.196:0/1375648163 conn(0x7fde9c0d50f0 msgr2=0x7fde9c15cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:26.662 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.660+0000 7fdea3000640 1 -- 172.21.15.196:0/1375648163 shutdown_connections 2024-04-28T13:32:26.662 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.660+0000 7fdea3000640 1 -- 172.21.15.196:0/1375648163 wait complete. 2024-04-28T13:32:26.679 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-28T13:32:26.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.877+0000 7f963be00640 1 Processor -- start 2024-04-28T13:32:26.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.878+0000 7f963be00640 1 -- start start 2024-04-28T13:32:26.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.879+0000 7f963be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9634145980 0x7f9634160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:26.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.879+0000 7f963be00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96340b3de0 0x7f96340b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:26.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.879+0000 7f963be00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96340b4260 con 0x7f96340b5fd0 2024-04-28T13:32:26.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.879+0000 7f963be00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f96340b6440 con 0x7f9634145980 2024-04-28T13:32:26.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.879+0000 7f963be00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f96340b5140 con 0x7f96340b3de0 2024-04-28T13:32:26.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.880+0000 7f9639400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9634145980 0x7f9634160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:26.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.880+0000 7f9639400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9634145980 0x7f9634160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55160/0 (socket says 172.21.15.196:55160) 2024-04-28T13:32:26.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.880+0000 7f9639e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f96340b5fd0 0x7f963415c510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:58116/0 (socket says 172.21.15.196:58116) 2024-04-28T13:32:26.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.880+0000 7f9639400640 1 -- 172.21.15.196:0/3994902133 learned_addr learned my addr 172.21.15.196:0/3994902133 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:26.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.880+0000 7f9638a00640 1 --2- 172.21.15.196:0/3994902133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96340b3de0 0x7f96340b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:26.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.880+0000 7f9639400640 1 -- 172.21.15.196:0/3994902133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96340b3de0 msgr2=0x7f96340b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:26.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.880+0000 7f9639400640 1 --2- 172.21.15.196:0/3994902133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96340b3de0 0x7f96340b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.880+0000 7f9639400640 1 -- 172.21.15.196:0/3994902133 >> v1:172.21.15.196:6789/0 conn(0x7f96340b5fd0 legacy=0x7f963415c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:26.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.880+0000 7f9639400640 1 -- 172.21.15.196:0/3994902133 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96340b4530 con 0x7f9634145980 2024-04-28T13:32:26.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.880+0000 7f9638a00640 1 --2- 172.21.15.196:0/3994902133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96340b3de0 0x7f96340b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:26.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.880+0000 7f9633e00640 1 -- 172.21.15.196:0/3994902133 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2486309881 0 0) 0x7f96340b4260 con 0x7f96340b5fd0 2024-04-28T13:32:26.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.881+0000 7f9639400640 1 --2- 172.21.15.196:0/3994902133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9634145980 0x7f9634160630 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9628009870 tx=0x7f962802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b487613176c219 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:26.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.881+0000 7f9633e00640 1 -- 172.21.15.196:0/3994902133 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628037070 con 0x7f9634145980 2024-04-28T13:32:26.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.882+0000 7f9633e00640 1 -- 172.21.15.196:0/3994902133 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9628032440 con 0x7f9634145980 2024-04-28T13:32:26.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.882+0000 7f9633e00640 1 -- 172.21.15.196:0/3994902133 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628032810 con 0x7f9634145980 2024-04-28T13:32:26.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.882+0000 7f963be00640 1 -- 172.21.15.196:0/3994902133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9634145980 msgr2=0x7f9634160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:26.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.882+0000 7f963be00640 1 --2- 172.21.15.196:0/3994902133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9634145980 0x7f9634160630 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9628009870 tx=0x7f962802a1c0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.882+0000 7f963be00640 1 -- 172.21.15.196:0/3994902133 shutdown_connections 2024-04-28T13:32:26.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.882+0000 7f963be00640 1 --2- 172.21.15.196:0/3994902133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96340b3de0 0x7f96340b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.882+0000 7f963be00640 1 --2- 172.21.15.196:0/3994902133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9634145980 0x7f9634160630 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.882+0000 7f963be00640 1 -- 172.21.15.196:0/3994902133 >> 172.21.15.196:0/3994902133 conn(0x7f96340d0d30 msgr2=0x7f96340d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:26.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.882+0000 7f963be00640 1 -- 172.21.15.196:0/3994902133 shutdown_connections 2024-04-28T13:32:26.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.882+0000 7f963be00640 1 -- 172.21.15.196:0/3994902133 wait complete. 2024-04-28T13:32:26.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.884+0000 7f963be00640 1 Processor -- start 2024-04-28T13:32:26.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.884+0000 7f963be00640 1 -- start start 2024-04-28T13:32:26.887 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.885+0000 7f963be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96340b3de0 0x7f963415b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:26.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.885+0000 7f963be00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96340b5fd0 0x7f963413c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:26.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.885+0000 7f963be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9634145980 0x7f963413a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:26.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.885+0000 7f963be00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f96340b4dd0 con 0x7f96340b3de0 2024-04-28T13:32:26.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.885+0000 7f963be00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96340b4af0 con 0x7f9634145980 2024-04-28T13:32:26.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.885+0000 7f963be00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f96340730e0 con 0x7f96340b5fd0 2024-04-28T13:32:26.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.885+0000 7f9639400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96340b3de0 0x7f963415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:26.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.885+0000 7f9639400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96340b3de0 0x7f963415b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55170/0 (socket says 172.21.15.196:55170) 2024-04-28T13:32:26.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.885+0000 7f9639400640 1 -- 172.21.15.196:0/1583093539 learned_addr learned my addr 172.21.15.196:0/1583093539 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:26.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.886+0000 7f9639400640 1 -- 172.21.15.196:0/1583093539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96340b5fd0 msgr2=0x7f963413c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:26.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.886+0000 7f9639400640 1 --2- 172.21.15.196:0/1583093539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96340b5fd0 0x7f963413c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.886+0000 7f9639400640 1 -- 172.21.15.196:0/1583093539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9634145980 msgr2=0x7f963413a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:26.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.886+0000 7f9639e00640 1 --2- 172.21.15.196:0/1583093539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9634145980 0x7f963413a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:26.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.886+0000 7f9639400640 1 --2- 172.21.15.196:0/1583093539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9634145980 0x7f963413a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:26.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.886+0000 7f9639400640 1 -- 172.21.15.196:0/1583093539 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963415b640 con 0x7f96340b3de0 2024-04-28T13:32:26.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.886+0000 7f9639400640 1 --2- 172.21.15.196:0/1583093539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96340b3de0 0x7f963415b100 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9628000c00 tx=0x7f96280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:26.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.887+0000 7f9631600640 1 -- 172.21.15.196:0/1583093539 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628037040 con 0x7f96340b3de0 2024-04-28T13:32:26.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.887+0000 7f9631600640 1 -- 172.21.15.196:0/1583093539 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9628042070 con 0x7f96340b3de0 2024-04-28T13:32:26.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.887+0000 7f9631600640 1 -- 172.21.15.196:0/1583093539 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628030bd0 con 0x7f96340b3de0 2024-04-28T13:32:26.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.887+0000 7f963be00640 1 -- 172.21.15.196:0/1583093539 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963413b4d0 con 0x7f96340b3de0 2024-04-28T13:32:26.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.887+0000 7f963be00640 1 -- 172.21.15.196:0/1583093539 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963413ba90 con 0x7f96340b3de0 2024-04-28T13:32:26.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.888+0000 7f963be00640 1 -- 172.21.15.196:0/1583093539 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f963406a290 con 0x7f96340b3de0 2024-04-28T13:32:26.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.889+0000 7f9631600640 1 -- 172.21.15.196:0/1583093539 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f962802ad10 con 0x7f96340b3de0 2024-04-28T13:32:26.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.890+0000 7f9631600640 1 --2- 172.21.15.196:0/1583093539 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f961406c8f0 0x7f961406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:26.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.890+0000 7f9631600640 1 -- 172.21.15.196:0/1583093539 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f96280ac2a0 con 0x7f96340b3de0 2024-04-28T13:32:26.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.890+0000 7f9638a00640 1 --2- 172.21.15.196:0/1583093539 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f961406c8f0 0x7f961406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:26.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.891+0000 7f9638a00640 1 --2- 172.21.15.196:0/1583093539 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f961406c8f0 0x7f961406edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9620006fd0 tx=0x7f9620008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:26.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:26.894+0000 7f9631600640 1 -- 172.21.15.196:0/1583093539 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f962807a8f0 con 0x7f96340b3de0 2024-04-28T13:32:27.171 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.170+0000 7f963be00640 1 -- 172.21.15.196:0/1583093539 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f9634073cc0 con 0x7f96340b3de0 2024-04-28T13:32:27.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.174+0000 7f9631600640 1 -- 172.21.15.196:0/1583093539 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f962807a1f0 con 0x7f96340b3de0 2024-04-28T13:32:27.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.177+0000 7f963be00640 1 -- 172.21.15.196:0/1583093539 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f961406c8f0 msgr2=0x7f961406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:27.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.177+0000 7f963be00640 1 --2- 172.21.15.196:0/1583093539 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f961406c8f0 0x7f961406edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9620006fd0 tx=0x7f9620008040 comp rx=0 tx=0).stop 2024-04-28T13:32:27.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.177+0000 7f963be00640 1 -- 172.21.15.196:0/1583093539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96340b3de0 msgr2=0x7f963415b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:27.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.177+0000 7f963be00640 1 --2- 172.21.15.196:0/1583093539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96340b3de0 0x7f963415b100 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9628000c00 tx=0x7f96280046c0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.177+0000 7f963be00640 1 -- 172.21.15.196:0/1583093539 shutdown_connections 2024-04-28T13:32:27.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.177+0000 7f963be00640 1 --2- 172.21.15.196:0/1583093539 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f961406c8f0 0x7f961406edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.177+0000 7f963be00640 1 --2- 172.21.15.196:0/1583093539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9634145980 0x7f963413a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.177+0000 7f963be00640 1 --2- 172.21.15.196:0/1583093539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96340b5fd0 0x7f963413c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.177+0000 7f963be00640 1 --2- 172.21.15.196:0/1583093539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96340b3de0 0x7f963415b100 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.177+0000 7f963be00640 1 -- 172.21.15.196:0/1583093539 >> 172.21.15.196:0/1583093539 conn(0x7f96340d0d30 msgr2=0x7f9634143bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:27.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.177+0000 7f963be00640 1 -- 172.21.15.196:0/1583093539 shutdown_connections 2024-04-28T13:32:27.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.177+0000 7f963be00640 1 -- 172.21.15.196:0/1583093539 wait complete. 2024-04-28T13:32:27.196 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-28T13:32:27.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.385+0000 7f6b35c00640 1 Processor -- start 2024-04-28T13:32:27.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.385+0000 7f6b35c00640 1 -- start start 2024-04-28T13:32:27.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.386+0000 7f6b35c00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6b300d1790 0x7f6b30073230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:27.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.386+0000 7f6b35c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b3015c020 0x7f6b30073770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:27.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.386+0000 7f6b35c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b3015fc10 con 0x7f6b3015bc40 2024-04-28T13:32:27.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.386+0000 7f6b35c00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f6b3015fef0 con 0x7f6b300d1790 2024-04-28T13:32:27.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.386+0000 7f6b35c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6b301601d0 con 0x7f6b3015c020 2024-04-28T13:32:27.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.387+0000 7f6b2fe00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f6b3015bc40 0x7f6b300d1080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:58122/0 (socket says 172.21.15.196:58122) 2024-04-28T13:32:27.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.387+0000 7f6b2fe00640 1 -- 172.21.15.196:0/1063869626 learned_addr learned my addr 172.21.15.196:0/1063869626 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:27.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.387+0000 7f6b2ea00640 1 -- 172.21.15.196:0/1063869626 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1311918884 0 0) 0x7f6b3015fc10 con 0x7f6b3015bc40 2024-04-28T13:32:27.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.387+0000 7f6b35200640 1 --2- 172.21.15.196:0/1063869626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b3015c020 0x7f6b30073770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:27.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.387+0000 7f6b2f400640 1 --2- 172.21.15.196:0/1063869626 >> v2:172.21.15.203:3300/0 conn(0x7f6b300d1790 0x7f6b30073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:27.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.387+0000 7f6b2ea00640 1 -- 172.21.15.196:0/1063869626 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b14005730 con 0x7f6b3015bc40 2024-04-28T13:32:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.387+0000 7f6b2ea00640 1 -- 172.21.15.196:0/1063869626 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3701253302 0 0) 0x7f6b14005730 con 0x7f6b3015bc40 2024-04-28T13:32:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2ea00640 1 -- 172.21.15.196:0/1063869626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b3015c020 msgr2=0x7f6b30073770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2ea00640 1 --2- 172.21.15.196:0/1063869626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b3015c020 0x7f6b30073770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2ea00640 1 -- 172.21.15.196:0/1063869626 >> v2:172.21.15.203:3300/0 conn(0x7f6b300d1790 msgr2=0x7f6b30073230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:27.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2ea00640 1 --2- 172.21.15.196:0/1063869626 >> v2:172.21.15.203:3300/0 conn(0x7f6b300d1790 0x7f6b30073230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2ea00640 1 -- 172.21.15.196:0/1063869626 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b301604b0 con 0x7f6b3015bc40 2024-04-28T13:32:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b35200640 1 --2- 172.21.15.196:0/1063869626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b3015c020 0x7f6b30073770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:32:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2ea00640 1 -- 172.21.15.196:0/1063869626 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f6b20001690 con 0x7f6b3015bc40 2024-04-28T13:32:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2ea00640 1 -- 172.21.15.196:0/1063869626 >> v1:172.21.15.196:6789/0 conn(0x7f6b3015bc40 legacy=0x7f6b300d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2ea00640 1 --2- 172.21.15.196:0/1063869626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b14005bc0 0x7f6b14005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2ea00640 1 --2- 172.21.15.196:0/1063869626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14006a10 0x7f6b14006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2ea00640 1 --2- 172.21.15.196:0/1063869626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b3015c020 0x7f6b140073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2ea00640 1 -- 172.21.15.196:0/1063869626 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3015fef0 con 0x7f6b14006a10 2024-04-28T13:32:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2ea00640 1 -- 172.21.15.196:0/1063869626 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6b301601d0 con 0x7f6b14005bc0 2024-04-28T13:32:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2ea00640 1 -- 172.21.15.196:0/1063869626 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1400dc80 con 0x7f6b3015c020 2024-04-28T13:32:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2f400640 1 --2- 172.21.15.196:0/1063869626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14006a10 0x7f6b14006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2fe00640 1 --2- 172.21.15.196:0/1063869626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b14005bc0 0x7f6b14005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2f400640 1 -- 172.21.15.196:0/1063869626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b3015c020 msgr2=0x7f6b140073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:27.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2f400640 1 --2- 172.21.15.196:0/1063869626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b3015c020 0x7f6b140073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2f400640 1 -- 172.21.15.196:0/1063869626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b14005bc0 msgr2=0x7f6b14005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2f400640 1 --2- 172.21.15.196:0/1063869626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b14005bc0 0x7f6b14005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.388+0000 7f6b2f400640 1 -- 172.21.15.196:0/1063869626 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b14005730 con 0x7f6b14006a10 2024-04-28T13:32:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.389+0000 7f6b2f400640 1 --2- 172.21.15.196:0/1063869626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14006a10 0x7f6b14006e90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00ba10 tx=0x7f6b14007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddfd8df8eb06595 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.390+0000 7f6b2ea00640 1 -- 172.21.15.196:0/1063869626 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c030070 con 0x7f6b14006a10 2024-04-28T13:32:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.390+0000 7f6b2ea00640 1 -- 172.21.15.196:0/1063869626 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1c02ae20 con 0x7f6b14006a10 2024-04-28T13:32:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.390+0000 7f6b2ea00640 1 -- 172.21.15.196:0/1063869626 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c00b5e0 con 0x7f6b14006a10 2024-04-28T13:32:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.390+0000 7f6b35c00640 1 -- 172.21.15.196:0/1063869626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14006a10 msgr2=0x7f6b14006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.390+0000 7f6b35c00640 1 --2- 172.21.15.196:0/1063869626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14006a10 0x7f6b14006e90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00ba10 tx=0x7f6b14007a40 comp rx=0 tx=0).stop 2024-04-28T13:32:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.390+0000 7f6b35200640 1 -- 172.21.15.196:0/1063869626 reap_dead start 2024-04-28T13:32:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.390+0000 7f6b35c00640 1 -- 172.21.15.196:0/1063869626 shutdown_connections 2024-04-28T13:32:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.390+0000 7f6b35c00640 1 -- 172.21.15.196:0/1063869626 >> 172.21.15.196:0/1063869626 conn(0x7f6b3006c6c0 msgr2=0x7f6b3006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.390+0000 7f6b35c00640 1 -- 172.21.15.196:0/1063869626 shutdown_connections 2024-04-28T13:32:27.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.390+0000 7f6b35c00640 1 -- 172.21.15.196:0/1063869626 wait complete. 2024-04-28T13:32:27.395 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.392+0000 7f6b35c00640 1 Processor -- start 2024-04-28T13:32:27.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.392+0000 7f6b35c00640 1 -- start start 2024-04-28T13:32:27.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.393+0000 7f6b35c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14006a10 0x7f6b3009ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:27.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.393+0000 7f6b35c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b30145cc0 0x7f6b301f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:27.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.393+0000 7f6b35c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b301f4c80 0x7f6b30144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:27.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.393+0000 7f6b35c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b300b0700 con 0x7f6b14006a10 2024-04-28T13:32:27.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.393+0000 7f6b35c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6b300b0580 con 0x7f6b30145cc0 2024-04-28T13:32:27.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.393+0000 7f6b35c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6b3006f800 con 0x7f6b301f4c80 2024-04-28T13:32:27.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.393+0000 7f6b2fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14006a10 0x7f6b3009ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:27.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.393+0000 7f6b2fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14006a10 0x7f6b3009ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55192/0 (socket says 172.21.15.196:55192) 2024-04-28T13:32:27.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.393+0000 7f6b2fe00640 1 -- 172.21.15.196:0/26706035 learned_addr learned my addr 172.21.15.196:0/26706035 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:27.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.394+0000 7f6b35200640 1 --2- 172.21.15.196:0/26706035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b301f4c80 0x7f6b30144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:27.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.394+0000 7f6b2f400640 1 --2- 172.21.15.196:0/26706035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b30145cc0 0x7f6b301f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:27.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.394+0000 7f6b2fe00640 1 -- 172.21.15.196:0/26706035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b301f4c80 msgr2=0x7f6b30144dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:27.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.394+0000 7f6b2fe00640 1 --2- 172.21.15.196:0/26706035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b301f4c80 0x7f6b30144dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.394+0000 7f6b2fe00640 1 -- 172.21.15.196:0/26706035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b30145cc0 msgr2=0x7f6b301f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:27.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.394+0000 7f6b2fe00640 1 --2- 172.21.15.196:0/26706035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b30145cc0 0x7f6b301f4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.394+0000 7f6b2fe00640 1 -- 172.21.15.196:0/26706035 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b300b0ac0 con 0x7f6b14006a10 2024-04-28T13:32:27.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.394+0000 7f6b35200640 1 --2- 172.21.15.196:0/26706035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b301f4c80 0x7f6b30144dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:27.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.394+0000 7f6b2fe00640 1 --2- 172.21.15.196:0/26706035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14006a10 0x7f6b3009ba40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000e8a0 tx=0x7f6b2000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:27.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.394+0000 7f6b27e00640 1 -- 172.21.15.196:0/26706035 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000c850 con 0x7f6b14006a10 2024-04-28T13:32:27.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.394+0000 7f6b27e00640 1 -- 172.21.15.196:0/26706035 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b20018070 con 0x7f6b14006a10 2024-04-28T13:32:27.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.394+0000 7f6b35c00640 1 -- 172.21.15.196:0/26706035 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3009c020 con 0x7f6b14006a10 2024-04-28T13:32:27.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.394+0000 7f6b27e00640 1 -- 172.21.15.196:0/26706035 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000cc50 con 0x7f6b14006a10 2024-04-28T13:32:27.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.394+0000 7f6b35c00640 1 -- 172.21.15.196:0/26706035 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b300b1060 con 0x7f6b14006a10 2024-04-28T13:32:27.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.395+0000 7f6b35c00640 1 -- 172.21.15.196:0/26706035 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b3009f0f0 con 0x7f6b14006a10 2024-04-28T13:32:27.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.396+0000 7f6b27e00640 1 -- 172.21.15.196:0/26706035 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b2001d460 con 0x7f6b14006a10 2024-04-28T13:32:27.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.397+0000 7f6b27e00640 1 --2- 172.21.15.196:0/26706035 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6b1006c8f0 0x7f6b1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:27.399 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.397+0000 7f6b27e00640 1 -- 172.21.15.196:0/26706035 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f6b2008b820 con 0x7f6b14006a10 2024-04-28T13:32:27.399 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.398+0000 7f6b2f400640 1 --2- 172.21.15.196:0/26706035 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6b1006c8f0 0x7f6b1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:27.399 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.399+0000 7f6b2f400640 1 --2- 172.21.15.196:0/26706035 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6b1006c8f0 0x7f6b1006edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00b9e0 tx=0x7f6b1c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:27.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.402+0000 7f6b27e00640 1 -- 172.21.15.196:0/26706035 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6b20059f00 con 0x7f6b14006a10 2024-04-28T13:32:27.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.652+0000 7f6b35c00640 1 -- 172.21.15.196:0/26706035 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f6b300507a0 con 0x7f6b14006a10 2024-04-28T13:32:27.658 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.657+0000 7f6b27e00640 1 -- 172.21.15.196:0/26706035 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f6b20059800 con 0x7f6b14006a10 2024-04-28T13:32:27.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.660+0000 7f6b35c00640 1 -- 172.21.15.196:0/26706035 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6b1006c8f0 msgr2=0x7f6b1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:27.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.660+0000 7f6b35c00640 1 --2- 172.21.15.196:0/26706035 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6b1006c8f0 0x7f6b1006edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00b9e0 tx=0x7f6b1c030000 comp rx=0 tx=0).stop 2024-04-28T13:32:27.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.660+0000 7f6b35c00640 1 -- 172.21.15.196:0/26706035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14006a10 msgr2=0x7f6b3009ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:27.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.660+0000 7f6b35c00640 1 --2- 172.21.15.196:0/26706035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14006a10 0x7f6b3009ba40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000e8a0 tx=0x7f6b2000ed60 comp rx=0 tx=0).stop 2024-04-28T13:32:27.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.660+0000 7f6b35c00640 1 -- 172.21.15.196:0/26706035 shutdown_connections 2024-04-28T13:32:27.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.660+0000 7f6b35c00640 1 --2- 172.21.15.196:0/26706035 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6b1006c8f0 0x7f6b1006edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.660+0000 7f6b35c00640 1 --2- 172.21.15.196:0/26706035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b301f4c80 0x7f6b30144dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.660+0000 7f6b35c00640 1 --2- 172.21.15.196:0/26706035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b30145cc0 0x7f6b301f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.660+0000 7f6b35c00640 1 --2- 172.21.15.196:0/26706035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b14006a10 0x7f6b3009ba40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.660+0000 7f6b35c00640 1 -- 172.21.15.196:0/26706035 >> 172.21.15.196:0/26706035 conn(0x7f6b3006c6c0 msgr2=0x7f6b300d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:27.662 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.660+0000 7f6b35c00640 1 -- 172.21.15.196:0/26706035 shutdown_connections 2024-04-28T13:32:27.662 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.660+0000 7f6b35c00640 1 -- 172.21.15.196:0/26706035 wait complete. 2024-04-28T13:32:27.678 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:32:27.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.870+0000 7f62a7e00640 1 Processor -- start 2024-04-28T13:32:27.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.870+0000 7f62a7e00640 1 -- start start 2024-04-28T13:32:27.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.872+0000 7f62a7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f62a8163bf0 0x7f62a8163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:27.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.872+0000 7f62a7e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a8163570 0x7f62a80b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:27.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.872+0000 7f62a7e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62a8162da0 con 0x7f62a81629c0 2024-04-28T13:32:27.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.872+0000 7f62a7e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f62a80d3e20 con 0x7f62a8163bf0 2024-04-28T13:32:27.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.872+0000 7f62a7e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f62a80b6190 con 0x7f62a8163570 2024-04-28T13:32:27.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.872+0000 7f62a6000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f62a8163bf0 0x7f62a8163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:27.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.872+0000 7f62a6000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f62a8163bf0 0x7f62a8163030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55204/0 (socket says 172.21.15.196:55204) 2024-04-28T13:32:27.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.872+0000 7f62a6000640 1 -- 172.21.15.196:0/3305845455 learned_addr learned my addr 172.21.15.196:0/3305845455 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:27.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.873+0000 7f62a7400640 1 --2- 172.21.15.196:0/3305845455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a8163570 0x7f62a80b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:27.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.873+0000 7f62a6000640 1 -- 172.21.15.196:0/3305845455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a8163570 msgr2=0x7f62a80b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:27.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.873+0000 7f62a6000640 1 --2- 172.21.15.196:0/3305845455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a8163570 0x7f62a80b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.873+0000 7f62a6000640 1 -- 172.21.15.196:0/3305845455 >> v1:172.21.15.203:6789/0 conn(0x7f62a81629c0 legacy=0x7f62a80d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:27.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.873+0000 7f62a6000640 1 -- 172.21.15.196:0/3305845455 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a80b6470 con 0x7f62a8163bf0 2024-04-28T13:32:27.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.873+0000 7f62a7400640 1 --2- 172.21.15.196:0/3305845455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a8163570 0x7f62a80b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:27.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.874+0000 7f62a6000640 1 --2- 172.21.15.196:0/3305845455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a8163bf0 0x7f62a8163030 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6290009e50 tx=0x7f629002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2593da03054d37b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:27.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.874+0000 7f62a5600640 1 -- 172.21.15.196:0/3305845455 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290036070 con 0x7f62a8163bf0 2024-04-28T13:32:27.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.874+0000 7f62a5600640 1 -- 172.21.15.196:0/3305845455 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6290031600 con 0x7f62a8163bf0 2024-04-28T13:32:27.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.874+0000 7f62a5600640 1 -- 172.21.15.196:0/3305845455 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62900319d0 con 0x7f62a8163bf0 2024-04-28T13:32:27.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.875+0000 7f62a7e00640 1 -- 172.21.15.196:0/3305845455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a8163bf0 msgr2=0x7f62a8163030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:27.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.875+0000 7f62a7e00640 1 --2- 172.21.15.196:0/3305845455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a8163bf0 0x7f62a8163030 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6290009e50 tx=0x7f629002a6c0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.875+0000 7f62a7e00640 1 -- 172.21.15.196:0/3305845455 shutdown_connections 2024-04-28T13:32:27.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.875+0000 7f62a7e00640 1 --2- 172.21.15.196:0/3305845455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a8163570 0x7f62a80b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.875+0000 7f62a7e00640 1 --2- 172.21.15.196:0/3305845455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a8163bf0 0x7f62a8163030 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.875+0000 7f62a7e00640 1 -- 172.21.15.196:0/3305845455 >> 172.21.15.196:0/3305845455 conn(0x7f62a809e570 msgr2=0x7f62a80d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:27.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.875+0000 7f62a7e00640 1 -- 172.21.15.196:0/3305845455 shutdown_connections 2024-04-28T13:32:27.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.875+0000 7f62a7e00640 1 -- 172.21.15.196:0/3305845455 wait complete. 2024-04-28T13:32:27.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.877+0000 7f62a7e00640 1 Processor -- start 2024-04-28T13:32:27.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.877+0000 7f62a7e00640 1 -- start start 2024-04-28T13:32:27.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.878+0000 7f62a7e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62a81629c0 0x7f62a80cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:27.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.878+0000 7f62a7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a8163570 0x7f62a815e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:27.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.878+0000 7f62a7e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a8163bf0 0x7f62a80cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:27.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.878+0000 7f62a7e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62a80b1e60 con 0x7f62a8163570 2024-04-28T13:32:27.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.878+0000 7f62a7e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f62a80b6a30 con 0x7f62a81629c0 2024-04-28T13:32:27.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.878+0000 7f62a7e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f62a80b28c0 con 0x7f62a8163bf0 2024-04-28T13:32:27.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.878+0000 7f62a6000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a8163570 0x7f62a815e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:27.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.878+0000 7f62a6000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a8163570 0x7f62a815e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55216/0 (socket says 172.21.15.196:55216) 2024-04-28T13:32:27.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.878+0000 7f62a6000640 1 -- 172.21.15.196:0/2225337744 learned_addr learned my addr 172.21.15.196:0/2225337744 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:27.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.878+0000 7f62a6a00640 1 --2- 172.21.15.196:0/2225337744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62a81629c0 0x7f62a80cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:27.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.878+0000 7f62a6000640 1 -- 172.21.15.196:0/2225337744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a8163bf0 msgr2=0x7f62a80cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:27.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.878+0000 7f62a6000640 1 --2- 172.21.15.196:0/2225337744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a8163bf0 0x7f62a80cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.878+0000 7f62a6000640 1 -- 172.21.15.196:0/2225337744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62a81629c0 msgr2=0x7f62a80cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:27.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.878+0000 7f62a6000640 1 --2- 172.21.15.196:0/2225337744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62a81629c0 0x7f62a80cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:27.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.878+0000 7f62a6000640 1 -- 172.21.15.196:0/2225337744 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a80cd4a0 con 0x7f62a8163570 2024-04-28T13:32:27.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.879+0000 7f62a6000640 1 --2- 172.21.15.196:0/2225337744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a8163570 0x7f62a815e8f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6290034040 tx=0x7f6290004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:27.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.879+0000 7f629ea00640 1 -- 172.21.15.196:0/2225337744 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290041020 con 0x7f62a8163570 2024-04-28T13:32:27.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.879+0000 7f629ea00640 1 -- 172.21.15.196:0/2225337744 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6290007820 con 0x7f62a8163570 2024-04-28T13:32:27.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.879+0000 7f629ea00640 1 -- 172.21.15.196:0/2225337744 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290036040 con 0x7f62a8163570 2024-04-28T13:32:27.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.879+0000 7f62a7e00640 1 -- 172.21.15.196:0/2225337744 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a80b9bf0 con 0x7f62a8163570 2024-04-28T13:32:27.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.879+0000 7f62a7e00640 1 -- 172.21.15.196:0/2225337744 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a80cf140 con 0x7f62a8163570 2024-04-28T13:32:27.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.880+0000 7f62a7e00640 1 -- 172.21.15.196:0/2225337744 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a8161ab0 con 0x7f62a8163570 2024-04-28T13:32:27.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.881+0000 7f629ea00640 1 -- 172.21.15.196:0/2225337744 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6290044020 con 0x7f62a8163570 2024-04-28T13:32:27.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.882+0000 7f629ea00640 1 --2- 172.21.15.196:0/2225337744 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f628406c920 0x7f628406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:27.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.883+0000 7f629ea00640 1 -- 172.21.15.196:0/2225337744 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f62900ac720 con 0x7f62a8163570 2024-04-28T13:32:27.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.883+0000 7f62a6a00640 1 --2- 172.21.15.196:0/2225337744 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f628406c920 0x7f628406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:27.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.884+0000 7f62a6a00640 1 --2- 172.21.15.196:0/2225337744 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f628406c920 0x7f628406ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6298003d90 tx=0x7f6298009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:27.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:27.887+0000 7f629ea00640 1 -- 172.21.15.196:0/2225337744 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f62a8161ab0 con 0x7f62a8163570 2024-04-28T13:32:28.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.086+0000 7f62a7e00640 1 -- 172.21.15.196:0/2225337744 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62a806a290 con 0x7f62a8163570 2024-04-28T13:32:28.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.088+0000 7f629ea00640 1 -- 172.21.15.196:0/2225337744 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12927 (secure 0 0 0) 0x7f629004b030 con 0x7f62a8163570 2024-04-28T13:32:28.090 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:28.090 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":29,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:24.124979+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:32:28.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.091+0000 7f62a7e00640 1 -- 172.21.15.196:0/2225337744 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f628406c920 msgr2=0x7f628406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:28.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.091+0000 7f62a7e00640 1 --2- 172.21.15.196:0/2225337744 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f628406c920 0x7f628406ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6298003d90 tx=0x7f6298009040 comp rx=0 tx=0).stop 2024-04-28T13:32:28.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.091+0000 7f62a7e00640 1 -- 172.21.15.196:0/2225337744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a8163570 msgr2=0x7f62a815e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:28.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.091+0000 7f62a7e00640 1 --2- 172.21.15.196:0/2225337744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a8163570 0x7f62a815e8f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6290034040 tx=0x7f6290004a40 comp rx=0 tx=0).stop 2024-04-28T13:32:28.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.091+0000 7f62a7e00640 1 -- 172.21.15.196:0/2225337744 shutdown_connections 2024-04-28T13:32:28.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.091+0000 7f62a7e00640 1 --2- 172.21.15.196:0/2225337744 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f628406c920 0x7f628406ede0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.091+0000 7f62a7e00640 1 --2- 172.21.15.196:0/2225337744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a8163bf0 0x7f62a80cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.091+0000 7f62a7e00640 1 --2- 172.21.15.196:0/2225337744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a8163570 0x7f62a815e8f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.091+0000 7f62a7e00640 1 --2- 172.21.15.196:0/2225337744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62a81629c0 0x7f62a80cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.091+0000 7f62a7e00640 1 -- 172.21.15.196:0/2225337744 >> 172.21.15.196:0/2225337744 conn(0x7f62a809e570 msgr2=0x7f62a80b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:28.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.092+0000 7f62a7e00640 1 -- 172.21.15.196:0/2225337744 shutdown_connections 2024-04-28T13:32:28.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.092+0000 7f62a7e00640 1 -- 172.21.15.196:0/2225337744 wait complete. 2024-04-28T13:32:28.110 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:32:28.110 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:32:28.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.295+0000 7fdf85e00640 1 Processor -- start 2024-04-28T13:32:28.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.295+0000 7fdf85e00640 1 -- start start 2024-04-28T13:32:28.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.296+0000 7fdf85e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdf8015f150 0x7fdf8006f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:28.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.297+0000 7fdf85e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8006df10 con 0x7fdf800b5df0 2024-04-28T13:32:28.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.297+0000 7fdf85e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8006e1f0 con 0x7fdf800d17b0 2024-04-28T13:32:28.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.297+0000 7fdf85e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdf8006fda0 con 0x7fdf8015f150 2024-04-28T13:32:28.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.297+0000 7fdf7fe00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fdf800b5df0 0x7fdf800d10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:58134/0 (socket says 172.21.15.196:58134) 2024-04-28T13:32:28.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.297+0000 7fdf7fe00640 1 -- 172.21.15.196:0/4060993765 learned_addr learned my addr 172.21.15.196:0/4060993765 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:28.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.297+0000 7fdf84a00640 1 --2- 172.21.15.196:0/4060993765 >> v2:172.21.15.203:3300/0 conn(0x7fdf8015f150 0x7fdf8006f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.297+0000 7fdf7f400640 1 -- 172.21.15.196:0/4060993765 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4283417199 0 0) 0x7fdf8006df10 con 0x7fdf800b5df0 2024-04-28T13:32:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.298+0000 7fdf7f400640 1 -- 172.21.15.196:0/4060993765 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf68003520 con 0x7fdf800b5df0 2024-04-28T13:32:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.298+0000 7fdf7f400640 1 -- 172.21.15.196:0/4060993765 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 144735332 0 0) 0x7fdf68003520 con 0x7fdf800b5df0 2024-04-28T13:32:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.298+0000 7fdf7f400640 1 -- 172.21.15.196:0/4060993765 >> v2:172.21.15.203:3300/0 conn(0x7fdf8015f150 msgr2=0x7fdf8006f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.298+0000 7fdf7f400640 1 --2- 172.21.15.196:0/4060993765 >> v2:172.21.15.203:3300/0 conn(0x7fdf8015f150 0x7fdf8006f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.298+0000 7fdf7f400640 1 -- 172.21.15.196:0/4060993765 >> v1:172.21.15.203:6789/0 conn(0x7fdf800d17b0 legacy=0x7fdf80072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.298+0000 7fdf7f400640 1 -- 172.21.15.196:0/4060993765 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8006e4d0 con 0x7fdf800b5df0 2024-04-28T13:32:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.298+0000 7fdf7f400640 1 -- 172.21.15.196:0/4060993765 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fdf6c001670 con 0x7fdf800b5df0 2024-04-28T13:32:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.298+0000 7fdf7f400640 1 -- 172.21.15.196:0/4060993765 >> v1:172.21.15.196:6789/0 conn(0x7fdf800b5df0 legacy=0x7fdf800d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.298+0000 7fdf7f400640 1 --2- 172.21.15.196:0/4060993765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf68003a00 0x7fdf68003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.298+0000 7fdf7f400640 1 --2- 172.21.15.196:0/4060993765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf68004800 0x7fdf68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.298+0000 7fdf7f400640 1 --2- 172.21.15.196:0/4060993765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf680051c0 0x7fdf6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.298+0000 7fdf7f400640 1 -- 172.21.15.196:0/4060993765 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8006fda0 con 0x7fdf680051c0 2024-04-28T13:32:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.298+0000 7fdf7f400640 1 -- 172.21.15.196:0/4060993765 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6800c0c0 con 0x7fdf68003a00 2024-04-28T13:32:28.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.298+0000 7fdf7f400640 1 -- 172.21.15.196:0/4060993765 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6800c2c0 con 0x7fdf68004800 2024-04-28T13:32:28.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.298+0000 7fdf85400640 1 --2- 172.21.15.196:0/4060993765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf680051c0 0x7fdf6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:28.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.299+0000 7fdf84a00640 1 --2- 172.21.15.196:0/4060993765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf68003a00 0x7fdf68003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:28.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.299+0000 7fdf7fe00640 1 --2- 172.21.15.196:0/4060993765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf68004800 0x7fdf68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:28.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.299+0000 7fdf85400640 1 -- 172.21.15.196:0/4060993765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf68004800 msgr2=0x7fdf68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:28.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.299+0000 7fdf85400640 1 --2- 172.21.15.196:0/4060993765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf68004800 0x7fdf68004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.299+0000 7fdf85400640 1 -- 172.21.15.196:0/4060993765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf68003a00 msgr2=0x7fdf68003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:28.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.299+0000 7fdf85400640 1 --2- 172.21.15.196:0/4060993765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf68003a00 0x7fdf68003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.299+0000 7fdf85400640 1 -- 172.21.15.196:0/4060993765 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf68003520 con 0x7fdf680051c0 2024-04-28T13:32:28.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.300+0000 7fdf85400640 1 -- 172.21.15.196:0/4060993765 reap_dead start 2024-04-28T13:32:28.302 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.300+0000 7fdf85400640 1 --2- 172.21.15.196:0/4060993765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf680051c0 0x7fdf6800b9c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdf80071e80 tx=0x7fdf68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14bf7915fa86bbc server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:28.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.300+0000 7fdf7f400640 1 -- 172.21.15.196:0/4060993765 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78001a70 con 0x7fdf680051c0 2024-04-28T13:32:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.300+0000 7fdf7f400640 1 -- 172.21.15.196:0/4060993765 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7802ae10 con 0x7fdf680051c0 2024-04-28T13:32:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.300+0000 7fdf7f400640 1 -- 172.21.15.196:0/4060993765 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78003bb0 con 0x7fdf680051c0 2024-04-28T13:32:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.301+0000 7fdf85e00640 1 -- 172.21.15.196:0/4060993765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf680051c0 msgr2=0x7fdf6800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.301+0000 7fdf85e00640 1 --2- 172.21.15.196:0/4060993765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf680051c0 0x7fdf6800b9c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdf80071e80 tx=0x7fdf68005830 comp rx=0 tx=0).stop 2024-04-28T13:32:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.301+0000 7fdf85e00640 1 -- 172.21.15.196:0/4060993765 shutdown_connections 2024-04-28T13:32:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.301+0000 7fdf85e00640 1 --2- 172.21.15.196:0/4060993765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf680051c0 0x7fdf6800b9c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.301+0000 7fdf85e00640 1 -- 172.21.15.196:0/4060993765 >> 172.21.15.196:0/4060993765 conn(0x7fdf8006c6c0 msgr2=0x7fdf8006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.301+0000 7fdf85e00640 1 -- 172.21.15.196:0/4060993765 shutdown_connections 2024-04-28T13:32:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.301+0000 7fdf85e00640 1 -- 172.21.15.196:0/4060993765 wait complete. 2024-04-28T13:32:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.303+0000 7fdf85e00640 1 Processor -- start 2024-04-28T13:32:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.303+0000 7fdf85e00640 1 -- start start 2024-04-28T13:32:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.304+0000 7fdf85e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf680051c0 0x7fdf8014fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:28.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.305+0000 7fdf85e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf800b23a0 0x7fdf80155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.305+0000 7fdf85e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf8015be20 0x7fdf8015c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.305+0000 7fdf85e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8015f530 con 0x7fdf680051c0 2024-04-28T13:32:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.305+0000 7fdf85e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8006ea90 con 0x7fdf8015be20 2024-04-28T13:32:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.305+0000 7fdf85e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdf8006f860 con 0x7fdf800b23a0 2024-04-28T13:32:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.305+0000 7fdf84a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf680051c0 0x7fdf8014fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.305+0000 7fdf84a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf680051c0 0x7fdf8014fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55228/0 (socket says 172.21.15.196:55228) 2024-04-28T13:32:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.305+0000 7fdf84a00640 1 -- 172.21.15.196:0/1982530073 learned_addr learned my addr 172.21.15.196:0/1982530073 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.305+0000 7fdf84a00640 1 -- 172.21.15.196:0/1982530073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf800b23a0 msgr2=0x7fdf80155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.305+0000 7fdf84a00640 1 --2- 172.21.15.196:0/1982530073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf800b23a0 0x7fdf80155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.305+0000 7fdf84a00640 1 -- 172.21.15.196:0/1982530073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf8015be20 msgr2=0x7fdf8015c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.305+0000 7fdf84a00640 1 --2- 172.21.15.196:0/1982530073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf8015be20 0x7fdf8015c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.305+0000 7fdf84a00640 1 -- 172.21.15.196:0/1982530073 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8014ff70 con 0x7fdf680051c0 2024-04-28T13:32:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.306+0000 7fdf84a00640 1 --2- 172.21.15.196:0/1982530073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf680051c0 0x7fdf8014fa30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdf74011570 tx=0x7fdf74011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:28.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.306+0000 7fdf7cc00640 1 -- 172.21.15.196:0/1982530073 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74015930 con 0x7fdf680051c0 2024-04-28T13:32:28.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.306+0000 7fdf7cc00640 1 -- 172.21.15.196:0/1982530073 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf74015ad0 con 0x7fdf680051c0 2024-04-28T13:32:28.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.306+0000 7fdf85e00640 1 -- 172.21.15.196:0/1982530073 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf80155b50 con 0x7fdf680051c0 2024-04-28T13:32:28.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.306+0000 7fdf7cc00640 1 -- 172.21.15.196:0/1982530073 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7400f460 con 0x7fdf680051c0 2024-04-28T13:32:28.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.306+0000 7fdf85e00640 1 -- 172.21.15.196:0/1982530073 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf80156110 con 0x7fdf680051c0 2024-04-28T13:32:28.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.307+0000 7fdf85e00640 1 -- 172.21.15.196:0/1982530073 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf80050730 con 0x7fdf680051c0 2024-04-28T13:32:28.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.308+0000 7fdf7cc00640 1 -- 172.21.15.196:0/1982530073 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf7400f600 con 0x7fdf680051c0 2024-04-28T13:32:28.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.309+0000 7fdf7cc00640 1 --2- 172.21.15.196:0/1982530073 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdf6006c8f0 0x7fdf6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:28.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.309+0000 7fdf7cc00640 1 -- 172.21.15.196:0/1982530073 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fdf7408c0b0 con 0x7fdf680051c0 2024-04-28T13:32:28.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.310+0000 7fdf7fe00640 1 --2- 172.21.15.196:0/1982530073 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdf6006c8f0 0x7fdf6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:28.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.311+0000 7fdf7fe00640 1 --2- 172.21.15.196:0/1982530073 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdf6006c8f0 0x7fdf6006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdf800b3330 tx=0x7fdf6c000ef0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:28.314 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.314+0000 7fdf7cc00640 1 -- 172.21.15.196:0/1982530073 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf74019030 con 0x7fdf680051c0 2024-04-28T13:32:28.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.498+0000 7fdf85e00640 1 -- 172.21.15.196:0/1982530073 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdf800d0560 con 0x7fdf680051c0 2024-04-28T13:32:28.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.498+0000 7fdf7cc00640 1 -- 172.21.15.196:0/1982530073 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7fdf7405a820 con 0x7fdf680051c0 2024-04-28T13:32:28.499 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:32:28.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.501+0000 7fdf85e00640 1 -- 172.21.15.196:0/1982530073 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdf6006c8f0 msgr2=0x7fdf6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:28.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.501+0000 7fdf85e00640 1 --2- 172.21.15.196:0/1982530073 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdf6006c8f0 0x7fdf6006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdf800b3330 tx=0x7fdf6c000ef0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.501+0000 7fdf85e00640 1 -- 172.21.15.196:0/1982530073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf680051c0 msgr2=0x7fdf8014fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:28.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.501+0000 7fdf85e00640 1 --2- 172.21.15.196:0/1982530073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf680051c0 0x7fdf8014fa30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdf74011570 tx=0x7fdf74011a30 comp rx=0 tx=0).stop 2024-04-28T13:32:28.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.501+0000 7fdf85e00640 1 -- 172.21.15.196:0/1982530073 shutdown_connections 2024-04-28T13:32:28.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.501+0000 7fdf85e00640 1 --2- 172.21.15.196:0/1982530073 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdf6006c8f0 0x7fdf6006edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.501+0000 7fdf85e00640 1 --2- 172.21.15.196:0/1982530073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf8015be20 0x7fdf8015c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.501+0000 7fdf85e00640 1 --2- 172.21.15.196:0/1982530073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf800b23a0 0x7fdf80155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.501+0000 7fdf85e00640 1 --2- 172.21.15.196:0/1982530073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf680051c0 0x7fdf8014fa30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.501+0000 7fdf85e00640 1 -- 172.21.15.196:0/1982530073 >> 172.21.15.196:0/1982530073 conn(0x7fdf8006c6c0 msgr2=0x7fdf8015e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:28.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.501+0000 7fdf85e00640 1 -- 172.21.15.196:0/1982530073 shutdown_connections 2024-04-28T13:32:28.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.501+0000 7fdf85e00640 1 -- 172.21.15.196:0/1982530073 wait complete. 2024-04-28T13:32:28.520 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-28T13:32:28.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.706+0000 7fec0be00640 1 Processor -- start 2024-04-28T13:32:28.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.706+0000 7fec0be00640 1 -- start start 2024-04-28T13:32:28.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.707+0000 7fec0be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fec0c15a9a0 0x7fec0c0cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:28.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.708+0000 7fec0be00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec0c0d1790 0x7fec0c15d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:28.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.708+0000 7fec0be00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec0c160b00 con 0x7fec0c0add70 2024-04-28T13:32:28.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.708+0000 7fec0be00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fec0c160de0 con 0x7fec0c15a9a0 2024-04-28T13:32:28.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.708+0000 7fec0be00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fec0c1610c0 con 0x7fec0c0d1790 2024-04-28T13:32:28.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.708+0000 7fec0aa00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fec0c15a9a0 0x7fec0c0cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:28.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.708+0000 7fec0aa00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fec0c15a9a0 0x7fec0c0cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55234/0 (socket says 172.21.15.196:55234) 2024-04-28T13:32:28.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.708+0000 7fec0aa00640 1 -- 172.21.15.196:0/2123234330 learned_addr learned my addr 172.21.15.196:0/2123234330 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:28.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.708+0000 7fec0aa00640 1 -- 172.21.15.196:0/2123234330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec0c0d1790 msgr2=0x7fec0c15d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:28.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.708+0000 7fec0aa00640 1 --2- 172.21.15.196:0/2123234330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec0c0d1790 0x7fec0c15d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.708+0000 7fec0aa00640 1 -- 172.21.15.196:0/2123234330 >> v1:172.21.15.203:6789/0 conn(0x7fec0c0add70 legacy=0x7fec0c0d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:28.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.708+0000 7fec0aa00640 1 -- 172.21.15.196:0/2123234330 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0c1613a0 con 0x7fec0c15a9a0 2024-04-28T13:32:28.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.709+0000 7fec0aa00640 1 --2- 172.21.15.196:0/2123234330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c15a9a0 0x7fec0c0cfe90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7febfc009870 tx=0x7febfc02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d6da0e03b3ecdd server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:28.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.709+0000 7fec0a000640 1 -- 172.21.15.196:0/2123234330 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc037070 con 0x7fec0c15a9a0 2024-04-28T13:32:28.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.709+0000 7fec0a000640 1 -- 172.21.15.196:0/2123234330 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febfc032440 con 0x7fec0c15a9a0 2024-04-28T13:32:28.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.709+0000 7fec0a000640 1 -- 172.21.15.196:0/2123234330 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc032810 con 0x7fec0c15a9a0 2024-04-28T13:32:28.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.710+0000 7fec0be00640 1 -- 172.21.15.196:0/2123234330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c15a9a0 msgr2=0x7fec0c0cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:28.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.710+0000 7fec0be00640 1 --2- 172.21.15.196:0/2123234330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c15a9a0 0x7fec0c0cfe90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7febfc009870 tx=0x7febfc02a200 comp rx=0 tx=0).stop 2024-04-28T13:32:28.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.710+0000 7fec0be00640 1 -- 172.21.15.196:0/2123234330 shutdown_connections 2024-04-28T13:32:28.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.710+0000 7fec0be00640 1 --2- 172.21.15.196:0/2123234330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec0c0d1790 0x7fec0c15d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.710+0000 7fec0be00640 1 --2- 172.21.15.196:0/2123234330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c15a9a0 0x7fec0c0cfe90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.710+0000 7fec0be00640 1 -- 172.21.15.196:0/2123234330 >> 172.21.15.196:0/2123234330 conn(0x7fec0c06c6c0 msgr2=0x7fec0c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:28.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.710+0000 7fec0be00640 1 -- 172.21.15.196:0/2123234330 shutdown_connections 2024-04-28T13:32:28.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.710+0000 7fec0be00640 1 -- 172.21.15.196:0/2123234330 wait complete. 2024-04-28T13:32:28.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.712+0000 7fec0be00640 1 Processor -- start 2024-04-28T13:32:28.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.712+0000 7fec0be00640 1 -- start start 2024-04-28T13:32:28.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.713+0000 7fec0be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec0c0add70 0x7fec0c072a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:28.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.713+0000 7fec0be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c0d1790 0x7fec0c14c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:28.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.713+0000 7fec0be00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec0c15a9a0 0x7fec0c14cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:28.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.713+0000 7fec0be00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c0d06e0 con 0x7fec0c0d1790 2024-04-28T13:32:28.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.713+0000 7fec0be00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c1618c0 con 0x7fec0c0add70 2024-04-28T13:32:28.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.713+0000 7fec0be00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fec0c15ad80 con 0x7fec0c15a9a0 2024-04-28T13:32:28.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.713+0000 7fec03e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c0d1790 0x7fec0c14c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:28.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.713+0000 7fec03e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c0d1790 0x7fec0c14c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55248/0 (socket says 172.21.15.196:55248) 2024-04-28T13:32:28.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.713+0000 7fec03e00640 1 -- 172.21.15.196:0/3658339240 learned_addr learned my addr 172.21.15.196:0/3658339240 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:28.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.713+0000 7fec03e00640 1 -- 172.21.15.196:0/3658339240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec0c15a9a0 msgr2=0x7fec0c14cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:28.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.713+0000 7fec03e00640 1 --2- 172.21.15.196:0/3658339240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec0c15a9a0 0x7fec0c14cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.714+0000 7fec03e00640 1 -- 172.21.15.196:0/3658339240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec0c0add70 msgr2=0x7fec0c072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:28.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.714+0000 7fec03e00640 1 --2- 172.21.15.196:0/3658339240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec0c0add70 0x7fec0c072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.714+0000 7fec03e00640 1 -- 172.21.15.196:0/3658339240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0c072ff0 con 0x7fec0c0d1790 2024-04-28T13:32:28.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.714+0000 7fec03e00640 1 --2- 172.21.15.196:0/3658339240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c0d1790 0x7fec0c14c6c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7febf400b3c0 tx=0x7febf400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:28.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.714+0000 7fec03400640 1 -- 172.21.15.196:0/3658339240 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4011830 con 0x7fec0c0d1790 2024-04-28T13:32:28.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.714+0000 7fec03400640 1 -- 172.21.15.196:0/3658339240 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf4016070 con 0x7fec0c0d1790 2024-04-28T13:32:28.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.714+0000 7fec03400640 1 -- 172.21.15.196:0/3658339240 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4011c30 con 0x7fec0c0d1790 2024-04-28T13:32:28.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.714+0000 7fec0be00640 1 -- 172.21.15.196:0/3658339240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0c0732d0 con 0x7fec0c0d1790 2024-04-28T13:32:28.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.714+0000 7fec0be00640 1 -- 172.21.15.196:0/3658339240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec0c073890 con 0x7fec0c0d1790 2024-04-28T13:32:28.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.715+0000 7fec0be00640 1 -- 172.21.15.196:0/3658339240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0c09f0f0 con 0x7fec0c0d1790 2024-04-28T13:32:28.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.716+0000 7fec03400640 1 -- 172.21.15.196:0/3658339240 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febf401b460 con 0x7fec0c0d1790 2024-04-28T13:32:28.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.718+0000 7fec03400640 1 --2- 172.21.15.196:0/3658339240 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7febe806c8f0 0x7febe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:28.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.718+0000 7fec03400640 1 -- 172.21.15.196:0/3658339240 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7febf408b5b0 con 0x7fec0c0d1790 2024-04-28T13:32:28.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.718+0000 7fec0aa00640 1 --2- 172.21.15.196:0/3658339240 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7febe806c8f0 0x7febe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:28.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.719+0000 7fec0aa00640 1 --2- 172.21.15.196:0/3658339240 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7febe806c8f0 0x7febe806edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7febfc034000 tx=0x7febfc02c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:28.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.722+0000 7fec03400640 1 -- 172.21.15.196:0/3658339240 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7febf4059c90 con 0x7fec0c0d1790 2024-04-28T13:32:28.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.920+0000 7fec0be00640 1 -- 172.21.15.196:0/3658339240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fec0c06f790 con 0x7fec0c0d1790 2024-04-28T13:32:28.922 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.921+0000 7fec03400640 1 -- 172.21.15.196:0/3658339240 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7febf408b030 con 0x7fec0c0d1790 2024-04-28T13:32:28.922 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:28.922 INFO:teuthology.orchestra.run.smithi196.stdout:[ 2024-04-28T13:32:28.922 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:28.922 INFO:teuthology.orchestra.run.smithi196.stdout: "poolnum": 1, 2024-04-28T13:32:28.922 INFO:teuthology.orchestra.run.smithi196.stdout: "poolname": ".mgr" 2024-04-28T13:32:28.922 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:28.922 INFO:teuthology.orchestra.run.smithi196.stdout:] 2024-04-28T13:32:28.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.923+0000 7fec0be00640 1 -- 172.21.15.196:0/3658339240 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7febe806c8f0 msgr2=0x7febe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:28.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.923+0000 7fec0be00640 1 --2- 172.21.15.196:0/3658339240 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7febe806c8f0 0x7febe806edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7febfc034000 tx=0x7febfc02c000 comp rx=0 tx=0).stop 2024-04-28T13:32:28.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.923+0000 7fec0be00640 1 -- 172.21.15.196:0/3658339240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c0d1790 msgr2=0x7fec0c14c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:28.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.923+0000 7fec0be00640 1 --2- 172.21.15.196:0/3658339240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c0d1790 0x7fec0c14c6c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7febf400b3c0 tx=0x7febf400b880 comp rx=0 tx=0).stop 2024-04-28T13:32:28.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.924+0000 7fec0be00640 1 -- 172.21.15.196:0/3658339240 shutdown_connections 2024-04-28T13:32:28.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.924+0000 7fec0be00640 1 --2- 172.21.15.196:0/3658339240 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7febe806c8f0 0x7febe806edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.924+0000 7fec0be00640 1 --2- 172.21.15.196:0/3658339240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec0c15a9a0 0x7fec0c14cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.924+0000 7fec0be00640 1 --2- 172.21.15.196:0/3658339240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c0d1790 0x7fec0c14c6c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.924+0000 7fec0be00640 1 --2- 172.21.15.196:0/3658339240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec0c0add70 0x7fec0c072a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:28.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.924+0000 7fec0be00640 1 -- 172.21.15.196:0/3658339240 >> 172.21.15.196:0/3658339240 conn(0x7fec0c06c6c0 msgr2=0x7fec0c0cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:28.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.924+0000 7fec0be00640 1 -- 172.21.15.196:0/3658339240 shutdown_connections 2024-04-28T13:32:28.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:28.924+0000 7fec0be00640 1 -- 172.21.15.196:0/3658339240 wait complete. 2024-04-28T13:32:28.941 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-28T13:32:28.941 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-28T13:32:29.137 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.136+0000 7f5e98a00640 1 Processor -- start 2024-04-28T13:32:29.138 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.136+0000 7f5e98a00640 1 -- start start 2024-04-28T13:32:29.139 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.138+0000 7f5e98a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5e940b5ee0 0x7f5e940b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:29.139 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.138+0000 7f5e98a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5e9409ec80 0x7f5e940d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:29.139 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.138+0000 7f5e98a00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e9415e380 con 0x7f5e940add70 2024-04-28T13:32:29.139 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.138+0000 7f5e98a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5e940b4b20 con 0x7f5e940b5ee0 2024-04-28T13:32:29.139 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.138+0000 7f5e98a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f5e940b3f30 con 0x7f5e9409ec80 2024-04-28T13:32:29.140 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.139+0000 7f5e92800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5e940b5ee0 0x7f5e940b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:29.140 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.139+0000 7f5e92800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5e940b5ee0 0x7f5e940b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55262/0 (socket says 172.21.15.196:55262) 2024-04-28T13:32:29.140 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.139+0000 7f5e92800640 1 -- 172.21.15.196:0/1091731822 learned_addr learned my addr 172.21.15.196:0/1091731822 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:29.140 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.139+0000 7f5e91e00640 1 --2- 172.21.15.196:0/1091731822 >> v2:172.21.15.203:3300/0 conn(0x7f5e9409ec80 0x7f5e940d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:29.140 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.139+0000 7f5e92800640 1 -- 172.21.15.196:0/1091731822 >> v2:172.21.15.203:3300/0 conn(0x7f5e9409ec80 msgr2=0x7f5e940d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:29.140 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.139+0000 7f5e92800640 1 --2- 172.21.15.196:0/1091731822 >> v2:172.21.15.203:3300/0 conn(0x7f5e9409ec80 0x7f5e940d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:29.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.139+0000 7f5e92800640 1 -- 172.21.15.196:0/1091731822 >> v1:172.21.15.203:6789/0 conn(0x7f5e940add70 legacy=0x7f5e9406df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:29.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.139+0000 7f5e92800640 1 -- 172.21.15.196:0/1091731822 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e940b41f0 con 0x7f5e940b5ee0 2024-04-28T13:32:29.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.140+0000 7f5e92800640 1 --2- 172.21.15.196:0/1091731822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e940b5ee0 0x7f5e940b4f40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5e940cd5d0 tx=0x7f5e8402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b9df4c65001025 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:29.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.141+0000 7f5e91400640 1 -- 172.21.15.196:0/1091731822 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84037070 con 0x7f5e940b5ee0 2024-04-28T13:32:29.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.141+0000 7f5e91400640 1 -- 172.21.15.196:0/1091731822 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e84032440 con 0x7f5e940b5ee0 2024-04-28T13:32:29.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.141+0000 7f5e91400640 1 -- 172.21.15.196:0/1091731822 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84032810 con 0x7f5e940b5ee0 2024-04-28T13:32:29.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.141+0000 7f5e98a00640 1 -- 172.21.15.196:0/1091731822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e940b5ee0 msgr2=0x7f5e940b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:29.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.141+0000 7f5e98a00640 1 --2- 172.21.15.196:0/1091731822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e940b5ee0 0x7f5e940b4f40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5e940cd5d0 tx=0x7f5e8402a230 comp rx=0 tx=0).stop 2024-04-28T13:32:29.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.142+0000 7f5e98a00640 1 -- 172.21.15.196:0/1091731822 shutdown_connections 2024-04-28T13:32:29.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.142+0000 7f5e98a00640 1 --2- 172.21.15.196:0/1091731822 >> v2:172.21.15.203:3300/0 conn(0x7f5e9409ec80 0x7f5e940d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:29.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.142+0000 7f5e98a00640 1 --2- 172.21.15.196:0/1091731822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e940b5ee0 0x7f5e940b4f40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:29.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.142+0000 7f5e98a00640 1 -- 172.21.15.196:0/1091731822 >> 172.21.15.196:0/1091731822 conn(0x7f5e9406c6c0 msgr2=0x7f5e9406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:29.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.142+0000 7f5e98a00640 1 -- 172.21.15.196:0/1091731822 shutdown_connections 2024-04-28T13:32:29.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.142+0000 7f5e98a00640 1 -- 172.21.15.196:0/1091731822 wait complete. 2024-04-28T13:32:29.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.143+0000 7f5e98a00640 1 Processor -- start 2024-04-28T13:32:29.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.144+0000 7f5e98a00640 1 -- start start 2024-04-28T13:32:29.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.144+0000 7f5e98a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e9409ec80 0x7f5e940cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:29.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.144+0000 7f5e98a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e940add70 0x7f5e94149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:29.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.145+0000 7f5e98a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e940b5ee0 0x7f5e941565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:29.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.145+0000 7f5e98a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5e940b3c20 con 0x7f5e940add70 2024-04-28T13:32:29.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.145+0000 7f5e98a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5e940b47b0 con 0x7f5e9409ec80 2024-04-28T13:32:29.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.145+0000 7f5e98a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5e940d1590 con 0x7f5e940b5ee0 2024-04-28T13:32:29.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.145+0000 7f5e91e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e940add70 0x7f5e94149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:29.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.145+0000 7f5e91e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e940add70 0x7f5e94149840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55272/0 (socket says 172.21.15.196:55272) 2024-04-28T13:32:29.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.145+0000 7f5e91e00640 1 -- 172.21.15.196:0/622794904 learned_addr learned my addr 172.21.15.196:0/622794904 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:29.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.145+0000 7f5e91e00640 1 -- 172.21.15.196:0/622794904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e940b5ee0 msgr2=0x7f5e941565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:29.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.145+0000 7f5e91e00640 1 --2- 172.21.15.196:0/622794904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e940b5ee0 0x7f5e941565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:29.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.145+0000 7f5e92800640 1 --2- 172.21.15.196:0/622794904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e9409ec80 0x7f5e940cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:29.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.145+0000 7f5e91e00640 1 -- 172.21.15.196:0/622794904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e9409ec80 msgr2=0x7f5e940cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:29.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.145+0000 7f5e91e00640 1 --2- 172.21.15.196:0/622794904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e9409ec80 0x7f5e940cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:29.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.145+0000 7f5e91e00640 1 -- 172.21.15.196:0/622794904 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e94156b00 con 0x7f5e940add70 2024-04-28T13:32:29.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.146+0000 7f5e91e00640 1 --2- 172.21.15.196:0/622794904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e940add70 0x7f5e94149840 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000d7d0 tx=0x7f5e8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:29.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.146+0000 7f5e8aa00640 1 -- 172.21.15.196:0/622794904 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80004020 con 0x7f5e940add70 2024-04-28T13:32:29.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.146+0000 7f5e8aa00640 1 -- 172.21.15.196:0/622794904 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e800041e0 con 0x7f5e940add70 2024-04-28T13:32:29.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.146+0000 7f5e8aa00640 1 -- 172.21.15.196:0/622794904 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e800043d0 con 0x7f5e940add70 2024-04-28T13:32:29.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.146+0000 7f5e98a00640 1 -- 172.21.15.196:0/622794904 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e940ce2f0 con 0x7f5e940add70 2024-04-28T13:32:29.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.146+0000 7f5e98a00640 1 -- 172.21.15.196:0/622794904 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e940ce8b0 con 0x7f5e940add70 2024-04-28T13:32:29.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.147+0000 7f5e98a00640 1 -- 172.21.15.196:0/622794904 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e94050730 con 0x7f5e940add70 2024-04-28T13:32:29.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.148+0000 7f5e8aa00640 1 -- 172.21.15.196:0/622794904 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e8001b020 con 0x7f5e940add70 2024-04-28T13:32:29.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.149+0000 7f5e8aa00640 1 --2- 172.21.15.196:0/622794904 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e7406c920 0x7f5e7406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:29.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.149+0000 7f5e8aa00640 1 -- 172.21.15.196:0/622794904 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f5e8008b5f0 con 0x7f5e940add70 2024-04-28T13:32:29.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.149+0000 7f5e92800640 1 --2- 172.21.15.196:0/622794904 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e7406c920 0x7f5e7406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:29.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.151+0000 7f5e92800640 1 --2- 172.21.15.196:0/622794904 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e7406c920 0x7f5e7406ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5e84034000 tx=0x7f5e8402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:29.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.153+0000 7f5e8aa00640 1 -- 172.21.15.196:0/622794904 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5e80059d60 con 0x7f5e940add70 2024-04-28T13:32:29.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:29.338+0000 7f5e98a00640 1 -- 172.21.15.196:0/622794904 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5e9415ce00 con 0x7f5e940add70 2024-04-28T13:32:30.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.176+0000 7f5e8aa00640 1 -- 172.21.15.196:0/622794904 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7f5e80059660 con 0x7f5e940add70 2024-04-28T13:32:30.178 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'cephfs_metadata' created 2024-04-28T13:32:30.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.180+0000 7f5e98a00640 1 -- 172.21.15.196:0/622794904 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e7406c920 msgr2=0x7f5e7406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:30.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.180+0000 7f5e98a00640 1 --2- 172.21.15.196:0/622794904 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e7406c920 0x7f5e7406ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5e84034000 tx=0x7f5e8402e000 comp rx=0 tx=0).stop 2024-04-28T13:32:30.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.180+0000 7f5e98a00640 1 -- 172.21.15.196:0/622794904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e940add70 msgr2=0x7f5e94149840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:30.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.180+0000 7f5e98a00640 1 --2- 172.21.15.196:0/622794904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e940add70 0x7f5e94149840 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000d7d0 tx=0x7f5e8000dc90 comp rx=0 tx=0).stop 2024-04-28T13:32:30.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.181+0000 7f5e98a00640 1 -- 172.21.15.196:0/622794904 shutdown_connections 2024-04-28T13:32:30.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.181+0000 7f5e98a00640 1 --2- 172.21.15.196:0/622794904 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e7406c920 0x7f5e7406ede0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:30.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.181+0000 7f5e98a00640 1 --2- 172.21.15.196:0/622794904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e940b5ee0 0x7f5e941565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:30.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.181+0000 7f5e98a00640 1 --2- 172.21.15.196:0/622794904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e940add70 0x7f5e94149840 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:30.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.181+0000 7f5e98a00640 1 --2- 172.21.15.196:0/622794904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e9409ec80 0x7f5e940cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:30.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.181+0000 7f5e98a00640 1 -- 172.21.15.196:0/622794904 >> 172.21.15.196:0/622794904 conn(0x7f5e9406c6c0 msgr2=0x7f5e9415ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:30.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.181+0000 7f5e98a00640 1 -- 172.21.15.196:0/622794904 shutdown_connections 2024-04-28T13:32:30.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.181+0000 7f5e98a00640 1 -- 172.21.15.196:0/622794904 wait complete. 2024-04-28T13:32:30.210 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-28T13:32:30.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.404+0000 7fad50600640 1 Processor -- start 2024-04-28T13:32:30.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.404+0000 7fad50600640 1 -- start start 2024-04-28T13:32:30.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.406+0000 7fad50600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fad481459a0 0x7fad480b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:30.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.406+0000 7fad50600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fad4815d030 0x7fad480b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:30.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.406+0000 7fad50600640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad480b4480 con 0x7fad480b6130 2024-04-28T13:32:30.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.406+0000 7fad50600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fad480b65a0 con 0x7fad481459a0 2024-04-28T13:32:30.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.406+0000 7fad50600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fad480b4760 con 0x7fad4815d030 2024-04-28T13:32:30.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.406+0000 7fad4dc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fad481459a0 0x7fad480b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:30.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.407+0000 7fad4dc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fad481459a0 0x7fad480b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55286/0 (socket says 172.21.15.196:55286) 2024-04-28T13:32:30.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.407+0000 7fad4dc00640 1 -- 172.21.15.196:0/2753264228 learned_addr learned my addr 172.21.15.196:0/2753264228 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:30.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.407+0000 7fad4d200640 1 --2- 172.21.15.196:0/2753264228 >> v2:172.21.15.203:3300/0 conn(0x7fad4815d030 0x7fad480b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:30.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.407+0000 7fad4dc00640 1 -- 172.21.15.196:0/2753264228 >> v2:172.21.15.203:3300/0 conn(0x7fad4815d030 msgr2=0x7fad480b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:30.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.407+0000 7fad4dc00640 1 --2- 172.21.15.196:0/2753264228 >> v2:172.21.15.203:3300/0 conn(0x7fad4815d030 0x7fad480b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:30.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.407+0000 7fad4dc00640 1 -- 172.21.15.196:0/2753264228 >> v1:172.21.15.203:6789/0 conn(0x7fad480b6130 legacy=0x7fad4815fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:30.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.407+0000 7fad4dc00640 1 -- 172.21.15.196:0/2753264228 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad480b4a40 con 0x7fad481459a0 2024-04-28T13:32:30.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.407+0000 7fad4d200640 1 --2- 172.21.15.196:0/2753264228 >> v2:172.21.15.203:3300/0 conn(0x7fad4815d030 0x7fad480b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:30.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.408+0000 7fad4dc00640 1 --2- 172.21.15.196:0/2753264228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad481459a0 0x7fad480b3de0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fad4806eb70 tx=0x7fad3c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8aaf2e55bc336b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:30.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.408+0000 7fad47e00640 1 -- 172.21.15.196:0/2753264228 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c037070 con 0x7fad481459a0 2024-04-28T13:32:30.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.408+0000 7fad47e00640 1 -- 172.21.15.196:0/2753264228 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad3c032440 con 0x7fad481459a0 2024-04-28T13:32:30.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.409+0000 7fad47e00640 1 -- 172.21.15.196:0/2753264228 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c032810 con 0x7fad481459a0 2024-04-28T13:32:30.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.409+0000 7fad50600640 1 -- 172.21.15.196:0/2753264228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad481459a0 msgr2=0x7fad480b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:30.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.409+0000 7fad50600640 1 --2- 172.21.15.196:0/2753264228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad481459a0 0x7fad480b3de0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fad4806eb70 tx=0x7fad3c02a210 comp rx=0 tx=0).stop 2024-04-28T13:32:30.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.409+0000 7fad50600640 1 -- 172.21.15.196:0/2753264228 shutdown_connections 2024-04-28T13:32:30.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.409+0000 7fad50600640 1 --2- 172.21.15.196:0/2753264228 >> v2:172.21.15.203:3300/0 conn(0x7fad4815d030 0x7fad480b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:30.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.409+0000 7fad50600640 1 --2- 172.21.15.196:0/2753264228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad481459a0 0x7fad480b3de0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:30.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.409+0000 7fad50600640 1 -- 172.21.15.196:0/2753264228 >> 172.21.15.196:0/2753264228 conn(0x7fad480d0d30 msgr2=0x7fad480d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:30.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.409+0000 7fad50600640 1 -- 172.21.15.196:0/2753264228 shutdown_connections 2024-04-28T13:32:30.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.409+0000 7fad50600640 1 -- 172.21.15.196:0/2753264228 wait complete. 2024-04-28T13:32:30.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.411+0000 7fad50600640 1 Processor -- start 2024-04-28T13:32:30.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.411+0000 7fad50600640 1 -- start start 2024-04-28T13:32:30.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.412+0000 7fad50600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad480b6130 0x7fad481699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:30.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.412+0000 7fad50600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad481459a0 0x7fad48143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:30.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.412+0000 7fad50600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad4815d030 0x7fad48144330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:30.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.412+0000 7fad50600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad4815d470 con 0x7fad481459a0 2024-04-28T13:32:30.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.412+0000 7fad50600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fad480b5000 con 0x7fad4815d030 2024-04-28T13:32:30.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.412+0000 7fad50600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fad48073100 con 0x7fad480b6130 2024-04-28T13:32:30.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.412+0000 7fad4d200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad481459a0 0x7fad48143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:30.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.412+0000 7fad4d200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad481459a0 0x7fad48143df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55292/0 (socket says 172.21.15.196:55292) 2024-04-28T13:32:30.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.412+0000 7fad4d200640 1 -- 172.21.15.196:0/3906666731 learned_addr learned my addr 172.21.15.196:0/3906666731 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:30.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.413+0000 7fad4dc00640 1 --2- 172.21.15.196:0/3906666731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad480b6130 0x7fad481699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:30.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.413+0000 7fad4e600640 1 --2- 172.21.15.196:0/3906666731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad4815d030 0x7fad48144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:30.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.413+0000 7fad4d200640 1 -- 172.21.15.196:0/3906666731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad480b6130 msgr2=0x7fad481699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:30.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.413+0000 7fad4d200640 1 --2- 172.21.15.196:0/3906666731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad480b6130 0x7fad481699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:30.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.413+0000 7fad4d200640 1 -- 172.21.15.196:0/3906666731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad4815d030 msgr2=0x7fad48144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:30.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.413+0000 7fad4d200640 1 --2- 172.21.15.196:0/3906666731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad4815d030 0x7fad48144330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:30.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.413+0000 7fad4d200640 1 -- 172.21.15.196:0/3906666731 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad48169f10 con 0x7fad481459a0 2024-04-28T13:32:30.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.413+0000 7fad4e600640 1 --2- 172.21.15.196:0/3906666731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad4815d030 0x7fad48144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:30.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.413+0000 7fad4d200640 1 --2- 172.21.15.196:0/3906666731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad481459a0 0x7fad48143df0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fad3400c7f0 tx=0x7fad3400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:30.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.413+0000 7fad45600640 1 -- 172.21.15.196:0/3906666731 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34015070 con 0x7fad481459a0 2024-04-28T13:32:30.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.413+0000 7fad45600640 1 -- 172.21.15.196:0/3906666731 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad34007820 con 0x7fad481459a0 2024-04-28T13:32:30.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.413+0000 7fad45600640 1 -- 172.21.15.196:0/3906666731 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34007bd0 con 0x7fad481459a0 2024-04-28T13:32:30.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.413+0000 7fad50600640 1 -- 172.21.15.196:0/3906666731 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad48168930 con 0x7fad481459a0 2024-04-28T13:32:30.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.414+0000 7fad50600640 1 -- 172.21.15.196:0/3906666731 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad48168ef0 con 0x7fad481459a0 2024-04-28T13:32:30.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.415+0000 7fad50600640 1 -- 172.21.15.196:0/3906666731 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4806a290 con 0x7fad481459a0 2024-04-28T13:32:30.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.416+0000 7fad45600640 1 -- 172.21.15.196:0/3906666731 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad34018070 con 0x7fad481459a0 2024-04-28T13:32:30.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.417+0000 7fad45600640 1 --2- 172.21.15.196:0/3906666731 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad2806c8f0 0x7fad2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:30.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.417+0000 7fad45600640 1 -- 172.21.15.196:0/3906666731 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7fad3408bc10 con 0x7fad481459a0 2024-04-28T13:32:30.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.417+0000 7fad4dc00640 1 --2- 172.21.15.196:0/3906666731 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad2806c8f0 0x7fad2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:30.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.419+0000 7fad4dc00640 1 --2- 172.21.15.196:0/3906666731 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad2806c8f0 0x7fad2806edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fad3c034000 tx=0x7fad3c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:30.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.422+0000 7fad45600640 1 -- 172.21.15.196:0/3906666731 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad3405a0f0 con 0x7fad481459a0 2024-04-28T13:32:30.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:30.623+0000 7fad50600640 1 -- 172.21.15.196:0/3906666731 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fad48132170 con 0x7fad481459a0 2024-04-28T13:32:31.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.191+0000 7fad45600640 1 -- 172.21.15.196:0/3906666731 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7fad340599f0 con 0x7fad481459a0 2024-04-28T13:32:31.192 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'cephfs_data' created 2024-04-28T13:32:31.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.195+0000 7fad50600640 1 -- 172.21.15.196:0/3906666731 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad2806c8f0 msgr2=0x7fad2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:31.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.195+0000 7fad50600640 1 --2- 172.21.15.196:0/3906666731 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad2806c8f0 0x7fad2806edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fad3c034000 tx=0x7fad3c02e000 comp rx=0 tx=0).stop 2024-04-28T13:32:31.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.195+0000 7fad50600640 1 -- 172.21.15.196:0/3906666731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad481459a0 msgr2=0x7fad48143df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:31.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.195+0000 7fad50600640 1 --2- 172.21.15.196:0/3906666731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad481459a0 0x7fad48143df0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fad3400c7f0 tx=0x7fad3400ccb0 comp rx=0 tx=0).stop 2024-04-28T13:32:31.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.195+0000 7fad50600640 1 -- 172.21.15.196:0/3906666731 shutdown_connections 2024-04-28T13:32:31.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.195+0000 7fad50600640 1 --2- 172.21.15.196:0/3906666731 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad2806c8f0 0x7fad2806edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:31.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.195+0000 7fad50600640 1 --2- 172.21.15.196:0/3906666731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad4815d030 0x7fad48144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:31.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.195+0000 7fad50600640 1 --2- 172.21.15.196:0/3906666731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad481459a0 0x7fad48143df0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:31.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.195+0000 7fad50600640 1 --2- 172.21.15.196:0/3906666731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad480b6130 0x7fad481699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:31.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.195+0000 7fad50600640 1 -- 172.21.15.196:0/3906666731 >> 172.21.15.196:0/3906666731 conn(0x7fad480d0d30 msgr2=0x7fad4813bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:31.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.196+0000 7fad50600640 1 -- 172.21.15.196:0/3906666731 shutdown_connections 2024-04-28T13:32:31.198 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.197+0000 7fad50600640 1 -- 172.21.15.196:0/3906666731 wait complete. 2024-04-28T13:32:31.233 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-28T13:32:31.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.409+0000 7ff98fe00640 1 Processor -- start 2024-04-28T13:32:31.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.409+0000 7ff98fe00640 1 -- start start 2024-04-28T13:32:31.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.410+0000 7ff98fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff99015f150 0x7ff99006f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:31.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.410+0000 7ff98fe00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff99006df10 con 0x7ff9900b5df0 2024-04-28T13:32:31.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.410+0000 7ff98fe00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff99006e1f0 con 0x7ff9900d17b0 2024-04-28T13:32:31.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.410+0000 7ff98fe00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff99006fda0 con 0x7ff99015f150 2024-04-28T13:32:31.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.411+0000 7ff98ea00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff99015f150 0x7ff99006f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:31.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.411+0000 7ff987e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ff9900b5df0 0x7ff9900d10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:58146/0 (socket says 172.21.15.196:58146) 2024-04-28T13:32:31.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.411+0000 7ff98ea00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff99015f150 0x7ff99006f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55298/0 (socket says 172.21.15.196:55298) 2024-04-28T13:32:31.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.411+0000 7ff987e00640 1 -- 172.21.15.196:0/989191355 learned_addr learned my addr 172.21.15.196:0/989191355 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:31.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.411+0000 7ff98ea00640 1 -- 172.21.15.196:0/989191355 >> v1:172.21.15.203:6789/0 conn(0x7ff9900d17b0 legacy=0x7ff990072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:31.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.411+0000 7ff98ea00640 1 -- 172.21.15.196:0/989191355 >> v1:172.21.15.196:6789/0 conn(0x7ff9900b5df0 legacy=0x7ff9900d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:31.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.411+0000 7ff98ea00640 1 -- 172.21.15.196:0/989191355 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99006e4d0 con 0x7ff99015f150 2024-04-28T13:32:31.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.411+0000 7ff98e000640 1 -- 172.21.15.196:0/989191355 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2202543605 0 0) 0x7ff99006df10 con 0x7ff9900b5df0 2024-04-28T13:32:31.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.412+0000 7ff98ea00640 1 --2- 172.21.15.196:0/989191355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff99015f150 0x7ff99006f860 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff9800097a0 tx=0x7ff98002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa05a229879ed816 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:31.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.413+0000 7ff98e000640 1 -- 172.21.15.196:0/989191355 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980037070 con 0x7ff99015f150 2024-04-28T13:32:31.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.413+0000 7ff98e000640 1 -- 172.21.15.196:0/989191355 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff980032440 con 0x7ff99015f150 2024-04-28T13:32:31.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.413+0000 7ff98e000640 1 -- 172.21.15.196:0/989191355 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980032810 con 0x7ff99015f150 2024-04-28T13:32:31.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.413+0000 7ff98fe00640 1 -- 172.21.15.196:0/989191355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff99015f150 msgr2=0x7ff99006f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:31.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.413+0000 7ff98fe00640 1 --2- 172.21.15.196:0/989191355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff99015f150 0x7ff99006f860 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff9800097a0 tx=0x7ff98002a1e0 comp rx=0 tx=0).stop 2024-04-28T13:32:31.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.413+0000 7ff98fe00640 1 -- 172.21.15.196:0/989191355 shutdown_connections 2024-04-28T13:32:31.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.413+0000 7ff98fe00640 1 --2- 172.21.15.196:0/989191355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff99015f150 0x7ff99006f860 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:31.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.413+0000 7ff98fe00640 1 -- 172.21.15.196:0/989191355 >> 172.21.15.196:0/989191355 conn(0x7ff99006c6c0 msgr2=0x7ff99006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:31.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.413+0000 7ff98fe00640 1 -- 172.21.15.196:0/989191355 shutdown_connections 2024-04-28T13:32:31.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.413+0000 7ff98fe00640 1 -- 172.21.15.196:0/989191355 wait complete. 2024-04-28T13:32:31.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.415+0000 7ff98fe00640 1 Processor -- start 2024-04-28T13:32:31.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.415+0000 7ff98fe00640 1 -- start start 2024-04-28T13:32:31.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.416+0000 7ff98fe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9900b5df0 0x7ff99014ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:31.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.416+0000 7ff98fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9900d17b0 0x7ff99009e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:31.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.416+0000 7ff98fe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99015f150 0x7ff9900b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:31.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.416+0000 7ff98fe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff99015f530 con 0x7ff9900d17b0 2024-04-28T13:32:31.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.416+0000 7ff98fe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff99006ea90 con 0x7ff99015f150 2024-04-28T13:32:31.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.416+0000 7ff98fe00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff990072860 con 0x7ff9900b5df0 2024-04-28T13:32:31.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.417+0000 7ff987e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9900d17b0 0x7ff99009e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:31.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.417+0000 7ff987e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9900d17b0 0x7ff99009e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55304/0 (socket says 172.21.15.196:55304) 2024-04-28T13:32:31.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.417+0000 7ff987e00640 1 -- 172.21.15.196:0/2567655897 learned_addr learned my addr 172.21.15.196:0/2567655897 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:31.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.417+0000 7ff987e00640 1 -- 172.21.15.196:0/2567655897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9900b5df0 msgr2=0x7ff99014ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:31.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.417+0000 7ff98ea00640 1 --2- 172.21.15.196:0/2567655897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9900b5df0 0x7ff99014ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:31.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.417+0000 7ff98f400640 1 --2- 172.21.15.196:0/2567655897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99015f150 0x7ff9900b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:31.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.417+0000 7ff987e00640 1 --2- 172.21.15.196:0/2567655897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9900b5df0 0x7ff99014ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:31.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.417+0000 7ff987e00640 1 -- 172.21.15.196:0/2567655897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99015f150 msgr2=0x7ff9900b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:31.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.417+0000 7ff987e00640 1 --2- 172.21.15.196:0/2567655897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99015f150 0x7ff9900b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:31.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.417+0000 7ff987e00640 1 -- 172.21.15.196:0/2567655897 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff990156480 con 0x7ff9900d17b0 2024-04-28T13:32:31.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.417+0000 7ff987e00640 1 --2- 172.21.15.196:0/2567655897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9900d17b0 0x7ff99009e320 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c4b0 tx=0x7ff97800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:31.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.418+0000 7ff986a00640 1 -- 172.21.15.196:0/2567655897 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978018070 con 0x7ff9900d17b0 2024-04-28T13:32:31.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.418+0000 7ff986a00640 1 -- 172.21.15.196:0/2567655897 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff978007500 con 0x7ff9900d17b0 2024-04-28T13:32:31.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.418+0000 7ff986a00640 1 -- 172.21.15.196:0/2567655897 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978004a20 con 0x7ff9900d17b0 2024-04-28T13:32:31.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.418+0000 7ff98fe00640 1 -- 172.21.15.196:0/2567655897 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99014cfb0 con 0x7ff9900d17b0 2024-04-28T13:32:31.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.418+0000 7ff98fe00640 1 -- 172.21.15.196:0/2567655897 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff990158650 con 0x7ff9900d17b0 2024-04-28T13:32:31.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.419+0000 7ff98fe00640 1 -- 172.21.15.196:0/2567655897 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff990050730 con 0x7ff9900d17b0 2024-04-28T13:32:31.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.420+0000 7ff986a00640 1 -- 172.21.15.196:0/2567655897 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff97801b070 con 0x7ff9900d17b0 2024-04-28T13:32:31.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.421+0000 7ff986a00640 1 --2- 172.21.15.196:0/2567655897 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff96c06c940 0x7ff96c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:31.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.421+0000 7ff986a00640 1 -- 172.21.15.196:0/2567655897 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7ff97808aed0 con 0x7ff9900d17b0 2024-04-28T13:32:31.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.422+0000 7ff98ea00640 1 --2- 172.21.15.196:0/2567655897 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff96c06c940 0x7ff96c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:31.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.423+0000 7ff98ea00640 1 --2- 172.21.15.196:0/2567655897 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff96c06c940 0x7ff96c06ee00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff980034000 tx=0x7ff98002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:31.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.426+0000 7ff986a00640 1 -- 172.21.15.196:0/2567655897 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff978059490 con 0x7ff9900d17b0 2024-04-28T13:32:31.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:31.667+0000 7ff98fe00640 1 -- 172.21.15.196:0/2567655897 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff9900af1a0 con 0x7ff9900d17b0 2024-04-28T13:32:32.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.224+0000 7ff986a00640 1 -- 172.21.15.196:0/2567655897 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7ff978058d90 con 0x7ff9900d17b0 2024-04-28T13:32:32.228 INFO:teuthology.orchestra.run.smithi196.stderr:new fs with metadata pool 4 and data pool 5 2024-04-28T13:32:32.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.231+0000 7ff98fe00640 1 -- 172.21.15.196:0/2567655897 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff96c06c940 msgr2=0x7ff96c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:32.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.231+0000 7ff98fe00640 1 --2- 172.21.15.196:0/2567655897 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff96c06c940 0x7ff96c06ee00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff980034000 tx=0x7ff98002e000 comp rx=0 tx=0).stop 2024-04-28T13:32:32.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.231+0000 7ff98fe00640 1 -- 172.21.15.196:0/2567655897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9900d17b0 msgr2=0x7ff99009e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:32.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.231+0000 7ff98fe00640 1 --2- 172.21.15.196:0/2567655897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9900d17b0 0x7ff99009e320 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c4b0 tx=0x7ff97800c970 comp rx=0 tx=0).stop 2024-04-28T13:32:32.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.232+0000 7ff98fe00640 1 -- 172.21.15.196:0/2567655897 shutdown_connections 2024-04-28T13:32:32.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.232+0000 7ff98fe00640 1 --2- 172.21.15.196:0/2567655897 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff96c06c940 0x7ff96c06ee00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.232+0000 7ff98fe00640 1 --2- 172.21.15.196:0/2567655897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99015f150 0x7ff9900b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.232+0000 7ff98fe00640 1 --2- 172.21.15.196:0/2567655897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9900d17b0 0x7ff99009e320 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.232+0000 7ff98fe00640 1 --2- 172.21.15.196:0/2567655897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9900b5df0 0x7ff99014ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.232+0000 7ff98fe00640 1 -- 172.21.15.196:0/2567655897 >> 172.21.15.196:0/2567655897 conn(0x7ff99006c6c0 msgr2=0x7ff990072c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:32.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.233+0000 7ff98fe00640 1 -- 172.21.15.196:0/2567655897 shutdown_connections 2024-04-28T13:32:32.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.233+0000 7ff98fe00640 1 -- 172.21.15.196:0/2567655897 wait complete. 2024-04-28T13:32:32.264 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:32:32.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.434+0000 7fa3e0a00640 1 Processor -- start 2024-04-28T13:32:32.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.434+0000 7fa3e0a00640 1 -- start start 2024-04-28T13:32:32.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.435+0000 7fa3e0a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa3dc0d1190 0x7fa3dc0b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.435+0000 7fa3e0a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3dc160f40 con 0x7fa3dc1469f0 2024-04-28T13:32:32.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.435+0000 7fa3e0a00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3dc161220 con 0x7fa3dc0d0d00 2024-04-28T13:32:32.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.435+0000 7fa3e0a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa3dc0b5960 con 0x7fa3dc0d1190 2024-04-28T13:32:32.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.436+0000 7fa3d9e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa3dc1469f0 0x7fa3dc0ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:48558/0 (socket says 172.21.15.196:48558) 2024-04-28T13:32:32.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.436+0000 7fa3d9e00640 1 -- 172.21.15.196:0/2502039825 learned_addr learned my addr 172.21.15.196:0/2502039825 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:32.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.436+0000 7fa3da800640 1 --2- 172.21.15.196:0/2502039825 >> v2:172.21.15.203:3300/0 conn(0x7fa3dc0d1190 0x7fa3dc0b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:32.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.436+0000 7fa3d9400640 1 -- 172.21.15.196:0/2502039825 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3367622374 0 0) 0x7fa3dc160f40 con 0x7fa3dc1469f0 2024-04-28T13:32:32.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.436+0000 7fa3d9400640 1 -- 172.21.15.196:0/2502039825 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3bc003520 con 0x7fa3dc1469f0 2024-04-28T13:32:32.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9400640 1 -- 172.21.15.196:0/2502039825 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1112724503 0 0) 0x7fa3bc003520 con 0x7fa3dc1469f0 2024-04-28T13:32:32.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9400640 1 -- 172.21.15.196:0/2502039825 >> v2:172.21.15.203:3300/0 conn(0x7fa3dc0d1190 msgr2=0x7fa3dc0b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:32.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9400640 1 --2- 172.21.15.196:0/2502039825 >> v2:172.21.15.203:3300/0 conn(0x7fa3dc0d1190 0x7fa3dc0b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9400640 1 -- 172.21.15.196:0/2502039825 >> v1:172.21.15.203:6789/0 conn(0x7fa3dc0d0d00 legacy=0x7fa3dc0b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:32.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9400640 1 -- 172.21.15.196:0/2502039825 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3dc161500 con 0x7fa3dc1469f0 2024-04-28T13:32:32.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9400640 1 -- 172.21.15.196:0/2502039825 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fa3cc001670 con 0x7fa3dc1469f0 2024-04-28T13:32:32.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9400640 1 -- 172.21.15.196:0/2502039825 >> v1:172.21.15.196:6789/0 conn(0x7fa3dc1469f0 legacy=0x7fa3dc0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:32.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9400640 1 --2- 172.21.15.196:0/2502039825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3bc003a00 0x7fa3bc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9400640 1 --2- 172.21.15.196:0/2502039825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3bc004800 0x7fa3bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9400640 1 --2- 172.21.15.196:0/2502039825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3bc005210 0x7fa3bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9400640 1 -- 172.21.15.196:0/2502039825 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc00c130 con 0x7fa3bc003a00 2024-04-28T13:32:32.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9400640 1 -- 172.21.15.196:0/2502039825 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc00c2e0 con 0x7fa3bc005210 2024-04-28T13:32:32.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9400640 1 -- 172.21.15.196:0/2502039825 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa3bc00c4e0 con 0x7fa3bc004800 2024-04-28T13:32:32.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9e00640 1 --2- 172.21.15.196:0/2502039825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3bc003a00 0x7fa3bc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:32.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9e00640 1 -- 172.21.15.196:0/2502039825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3bc004800 msgr2=0x7fa3bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:32.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9e00640 1 --2- 172.21.15.196:0/2502039825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3bc004800 0x7fa3bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9e00640 1 -- 172.21.15.196:0/2502039825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3bc005210 msgr2=0x7fa3bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:32.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9e00640 1 --2- 172.21.15.196:0/2502039825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3bc005210 0x7fa3bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3d9e00640 1 -- 172.21.15.196:0/2502039825 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3bc003520 con 0x7fa3bc003a00 2024-04-28T13:32:32.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.437+0000 7fa3da800640 1 -- 172.21.15.196:0/2502039825 reap_dead start 2024-04-28T13:32:32.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.438+0000 7fa3d9e00640 1 --2- 172.21.15.196:0/2502039825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3bc003a00 0x7fa3bc003e20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00c110 tx=0x7fa3bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dcbb4299a1e552 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:32.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.439+0000 7fa3d9400640 1 -- 172.21.15.196:0/2502039825 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc007c40 con 0x7fa3bc003a00 2024-04-28T13:32:32.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.439+0000 7fa3d9400640 1 -- 172.21.15.196:0/2502039825 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3cc005af0 con 0x7fa3bc003a00 2024-04-28T13:32:32.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.439+0000 7fa3d9400640 1 -- 172.21.15.196:0/2502039825 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc007070 con 0x7fa3bc003a00 2024-04-28T13:32:32.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.439+0000 7fa3e0a00640 1 -- 172.21.15.196:0/2502039825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3bc003a00 msgr2=0x7fa3bc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:32.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.439+0000 7fa3e0a00640 1 --2- 172.21.15.196:0/2502039825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3bc003a00 0x7fa3bc003e20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00c110 tx=0x7fa3bc005830 comp rx=0 tx=0).stop 2024-04-28T13:32:32.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.439+0000 7fa3e0a00640 1 -- 172.21.15.196:0/2502039825 shutdown_connections 2024-04-28T13:32:32.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.439+0000 7fa3e0a00640 1 --2- 172.21.15.196:0/2502039825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3bc003a00 0x7fa3bc003e20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.439+0000 7fa3e0a00640 1 -- 172.21.15.196:0/2502039825 >> 172.21.15.196:0/2502039825 conn(0x7fa3dc0cd830 msgr2=0x7fa3dc0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:32.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.439+0000 7fa3e0a00640 1 -- 172.21.15.196:0/2502039825 shutdown_connections 2024-04-28T13:32:32.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.439+0000 7fa3e0a00640 1 -- 172.21.15.196:0/2502039825 wait complete. 2024-04-28T13:32:32.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.441+0000 7fa3e0a00640 1 Processor -- start 2024-04-28T13:32:32.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.441+0000 7fa3e0a00640 1 -- start start 2024-04-28T13:32:32.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.442+0000 7fa3e0a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3bc003a00 0x7fa3dc13c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.442+0000 7fa3e0a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3dc144d60 0x7fa3dc0b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.442+0000 7fa3e0a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3dc0cfac0 0x7fa3dc0cff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.442+0000 7fa3e0a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc072020 con 0x7fa3dc144d60 2024-04-28T13:32:32.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.442+0000 7fa3e0a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc161ac0 con 0x7fa3dc0cfac0 2024-04-28T13:32:32.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.442+0000 7fa3e0a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa3dc0d15c0 con 0x7fa3bc003a00 2024-04-28T13:32:32.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3d3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3dc144d60 0x7fa3dc0b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:32.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3d3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3dc144d60 0x7fa3dc0b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34204/0 (socket says 172.21.15.196:34204) 2024-04-28T13:32:32.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3d3e00640 1 -- 172.21.15.196:0/1294329004 learned_addr learned my addr 172.21.15.196:0/1294329004 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:32.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3d9e00640 1 --2- 172.21.15.196:0/1294329004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3bc003a00 0x7fa3dc13c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:32.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3da800640 1 --2- 172.21.15.196:0/1294329004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3dc0cfac0 0x7fa3dc0cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:32.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3d3e00640 1 -- 172.21.15.196:0/1294329004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3bc003a00 msgr2=0x7fa3dc13c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:32.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3d3e00640 1 --2- 172.21.15.196:0/1294329004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3bc003a00 0x7fa3dc13c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3d3e00640 1 -- 172.21.15.196:0/1294329004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3dc0cfac0 msgr2=0x7fa3dc0cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:32.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3d3e00640 1 --2- 172.21.15.196:0/1294329004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3dc0cfac0 0x7fa3dc0cff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3d3e00640 1 -- 172.21.15.196:0/1294329004 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3dc1601c0 con 0x7fa3dc144d60 2024-04-28T13:32:32.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3da800640 1 --2- 172.21.15.196:0/1294329004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3dc0cfac0 0x7fa3dc0cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:32.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3d3e00640 1 --2- 172.21.15.196:0/1294329004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3dc144d60 0x7fa3dc0b0db0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400c3c0 tx=0x7fa3c4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:32.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3d2000640 1 -- 172.21.15.196:0/1294329004 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4005220 con 0x7fa3dc144d60 2024-04-28T13:32:32.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3d2000640 1 -- 172.21.15.196:0/1294329004 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c40053e0 con 0x7fa3dc144d60 2024-04-28T13:32:32.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3d2000640 1 -- 172.21.15.196:0/1294329004 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4007270 con 0x7fa3dc144d60 2024-04-28T13:32:32.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3e0a00640 1 -- 172.21.15.196:0/1294329004 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3dc1604a0 con 0x7fa3dc144d60 2024-04-28T13:32:32.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.444+0000 7fa3e0a00640 1 -- 172.21.15.196:0/1294329004 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3dc0b1770 con 0x7fa3dc144d60 2024-04-28T13:32:32.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.445+0000 7fa3e0a00640 1 -- 172.21.15.196:0/1294329004 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3dc06a290 con 0x7fa3dc144d60 2024-04-28T13:32:32.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.446+0000 7fa3d2000640 1 -- 172.21.15.196:0/1294329004 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3c4019020 con 0x7fa3dc144d60 2024-04-28T13:32:32.448 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.447+0000 7fa3d2000640 1 --2- 172.21.15.196:0/1294329004 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa3b806c8f0 0x7fa3b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.448 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.447+0000 7fa3d2000640 1 -- 172.21.15.196:0/1294329004 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa3c408af10 con 0x7fa3dc144d60 2024-04-28T13:32:32.448 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.448+0000 7fa3d9e00640 1 --2- 172.21.15.196:0/1294329004 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa3b806c8f0 0x7fa3b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:32.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.449+0000 7fa3d9e00640 1 --2- 172.21.15.196:0/1294329004 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa3b806c8f0 0x7fa3b806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc032000 tx=0x7fa3cc009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:32.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.451+0000 7fa3d2000640 1 -- 172.21.15.196:0/1294329004 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa3c40592a0 con 0x7fa3dc144d60 2024-04-28T13:32:32.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.651+0000 7fa3e0a00640 1 -- 172.21.15.196:0/1294329004 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3dc0cdda0 con 0x7fa3dc144d60 2024-04-28T13:32:32.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.653+0000 7fa3d2000640 1 -- 172.21.15.196:0/1294329004 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16718 (secure 0 0 0) 0x7fa3c4058ba0 con 0x7fa3dc144d60 2024-04-28T13:32:32.654 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:32.654 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":32,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:31.669527+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:32:29.352348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-28T13:32:30.637498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:32:32.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.656+0000 7fa3e0a00640 1 -- 172.21.15.196:0/1294329004 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa3b806c8f0 msgr2=0x7fa3b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:32.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.656+0000 7fa3e0a00640 1 --2- 172.21.15.196:0/1294329004 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa3b806c8f0 0x7fa3b806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc032000 tx=0x7fa3cc009030 comp rx=0 tx=0).stop 2024-04-28T13:32:32.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.656+0000 7fa3e0a00640 1 -- 172.21.15.196:0/1294329004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3dc144d60 msgr2=0x7fa3dc0b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:32.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.656+0000 7fa3e0a00640 1 --2- 172.21.15.196:0/1294329004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3dc144d60 0x7fa3dc0b0db0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400c3c0 tx=0x7fa3c4001e70 comp rx=0 tx=0).stop 2024-04-28T13:32:32.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.657+0000 7fa3e0a00640 1 -- 172.21.15.196:0/1294329004 shutdown_connections 2024-04-28T13:32:32.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.657+0000 7fa3e0a00640 1 --2- 172.21.15.196:0/1294329004 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa3b806c8f0 0x7fa3b806edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.657+0000 7fa3e0a00640 1 --2- 172.21.15.196:0/1294329004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3dc0cfac0 0x7fa3dc0cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.657+0000 7fa3e0a00640 1 --2- 172.21.15.196:0/1294329004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3dc144d60 0x7fa3dc0b0db0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.657+0000 7fa3e0a00640 1 --2- 172.21.15.196:0/1294329004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3bc003a00 0x7fa3dc13c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.657+0000 7fa3e0a00640 1 -- 172.21.15.196:0/1294329004 >> 172.21.15.196:0/1294329004 conn(0x7fa3dc0cd830 msgr2=0x7fa3dc0b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:32.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.657+0000 7fa3e0a00640 1 -- 172.21.15.196:0/1294329004 shutdown_connections 2024-04-28T13:32:32.658 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.657+0000 7fa3e0a00640 1 -- 172.21.15.196:0/1294329004 wait complete. 2024-04-28T13:32:32.675 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:32:32.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.867+0000 7f7559800640 1 Processor -- start 2024-04-28T13:32:32.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.867+0000 7f7559800640 1 -- start start 2024-04-28T13:32:32.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.869+0000 7f7559800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75540d1760 0x7f75540b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.869+0000 7f7559800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f755415e380 con 0x7f75540b5ee0 2024-04-28T13:32:32.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.869+0000 7f7559800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75540b4b20 con 0x7f755409ec80 2024-04-28T13:32:32.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.869+0000 7f7559800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f75540b3f30 con 0x7f75540d1760 2024-04-28T13:32:32.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.869+0000 7f7553e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f75540b5ee0 0x7f75540d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:48574/0 (socket says 172.21.15.196:48574) 2024-04-28T13:32:32.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.869+0000 7f7553e00640 1 -- 172.21.15.196:0/3798675219 learned_addr learned my addr 172.21.15.196:0/3798675219 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:32.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7553400640 1 --2- 172.21.15.196:0/3798675219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75540d1760 0x7f75540b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:32.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7552a00640 1 -- 172.21.15.196:0/3798675219 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1031335875 0 0) 0x7f755415e380 con 0x7f75540b5ee0 2024-04-28T13:32:32.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7552a00640 1 -- 172.21.15.196:0/3798675219 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7538003520 con 0x7f75540b5ee0 2024-04-28T13:32:32.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7552a00640 1 -- 172.21.15.196:0/3798675219 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1066945499 0 0) 0x7f7538003520 con 0x7f75540b5ee0 2024-04-28T13:32:32.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7552a00640 1 -- 172.21.15.196:0/3798675219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75540d1760 msgr2=0x7f75540b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:32.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7552a00640 1 --2- 172.21.15.196:0/3798675219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75540d1760 0x7f75540b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7552a00640 1 -- 172.21.15.196:0/3798675219 >> v1:172.21.15.203:6789/0 conn(0x7f755409ec80 legacy=0x7f755406df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:32.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7552a00640 1 -- 172.21.15.196:0/3798675219 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75540b41f0 con 0x7f75540b5ee0 2024-04-28T13:32:32.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7552a00640 1 -- 172.21.15.196:0/3798675219 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f7544001670 con 0x7f75540b5ee0 2024-04-28T13:32:32.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7552a00640 1 -- 172.21.15.196:0/3798675219 >> v1:172.21.15.196:6789/0 conn(0x7f75540b5ee0 legacy=0x7f75540d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:32.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7552a00640 1 --2- 172.21.15.196:0/3798675219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75380039e0 0x7f7538003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7552a00640 1 --2- 172.21.15.196:0/3798675219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75540d1760 0x7f7538004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7552a00640 1 --2- 172.21.15.196:0/3798675219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7538004d40 0x7f75380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7552a00640 1 -- 172.21.15.196:0/3798675219 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75540b3f30 con 0x7f75380039e0 2024-04-28T13:32:32.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7552a00640 1 -- 172.21.15.196:0/3798675219 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f753800baa0 con 0x7f7538004d40 2024-04-28T13:32:32.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.870+0000 7f7552a00640 1 -- 172.21.15.196:0/3798675219 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f753800bca0 con 0x7f75540d1760 2024-04-28T13:32:32.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.871+0000 7f7553e00640 1 --2- 172.21.15.196:0/3798675219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75380039e0 0x7f7538003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:32.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.871+0000 7f7553400640 1 --2- 172.21.15.196:0/3798675219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75540d1760 0x7f7538004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:32.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.871+0000 7f7558e00640 1 --2- 172.21.15.196:0/3798675219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7538004d40 0x7f75380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:32.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.871+0000 7f7553e00640 1 -- 172.21.15.196:0/3798675219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75540d1760 msgr2=0x7f7538004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:32.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.871+0000 7f7553e00640 1 --2- 172.21.15.196:0/3798675219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75540d1760 0x7f7538004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.871+0000 7f7553e00640 1 -- 172.21.15.196:0/3798675219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7538004d40 msgr2=0x7f75380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:32.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.871+0000 7f7553e00640 1 --2- 172.21.15.196:0/3798675219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7538004d40 0x7f75380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.871+0000 7f7553e00640 1 -- 172.21.15.196:0/3798675219 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7538003520 con 0x7f75380039e0 2024-04-28T13:32:32.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.871+0000 7f7553400640 1 --2- 172.21.15.196:0/3798675219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75540d1760 0x7f7538004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:32.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.872+0000 7f7553e00640 1 --2- 172.21.15.196:0/3798675219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75380039e0 0x7f7538003e00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f754400c0e0 tx=0x7f7538005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83919568078bba90 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:32.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.872+0000 7f7552a00640 1 -- 172.21.15.196:0/3798675219 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544007c40 con 0x7f75380039e0 2024-04-28T13:32:32.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.873+0000 7f7552a00640 1 -- 172.21.15.196:0/3798675219 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7544005af0 con 0x7f75380039e0 2024-04-28T13:32:32.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.873+0000 7f7552a00640 1 -- 172.21.15.196:0/3798675219 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544007070 con 0x7f75380039e0 2024-04-28T13:32:32.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.873+0000 7f7559800640 1 -- 172.21.15.196:0/3798675219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75380039e0 msgr2=0x7f7538003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:32.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.873+0000 7f7559800640 1 --2- 172.21.15.196:0/3798675219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75380039e0 0x7f7538003e00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f754400c0e0 tx=0x7f7538005830 comp rx=0 tx=0).stop 2024-04-28T13:32:32.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.873+0000 7f7558e00640 1 -- 172.21.15.196:0/3798675219 reap_dead start 2024-04-28T13:32:32.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.873+0000 7f7559800640 1 -- 172.21.15.196:0/3798675219 shutdown_connections 2024-04-28T13:32:32.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.873+0000 7f7559800640 1 -- 172.21.15.196:0/3798675219 >> 172.21.15.196:0/3798675219 conn(0x7f755406c6c0 msgr2=0x7f755406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:32.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.873+0000 7f7559800640 1 -- 172.21.15.196:0/3798675219 shutdown_connections 2024-04-28T13:32:32.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.873+0000 7f7559800640 1 -- 172.21.15.196:0/3798675219 wait complete. 2024-04-28T13:32:32.875 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.875+0000 7f7559800640 1 Processor -- start 2024-04-28T13:32:32.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.875+0000 7f7559800640 1 -- start start 2024-04-28T13:32:32.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.876+0000 7f7559800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75380039e0 0x7f755413ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.876+0000 7f7559800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f755413f280 0x7f75541568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.876+0000 7f7559800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7554156e20 0x7f755409dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.876+0000 7f7559800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75540b3c20 con 0x7f755413f280 2024-04-28T13:32:32.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.876+0000 7f7559800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75540b47b0 con 0x7f75380039e0 2024-04-28T13:32:32.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.876+0000 7f7559800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f75540add70 con 0x7f7554156e20 2024-04-28T13:32:32.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.876+0000 7f7553400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f755413f280 0x7f75541568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:32.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.876+0000 7f7553400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f755413f280 0x7f75541568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34218/0 (socket says 172.21.15.196:34218) 2024-04-28T13:32:32.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.876+0000 7f7553400640 1 -- 172.21.15.196:0/1598505483 learned_addr learned my addr 172.21.15.196:0/1598505483 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:32.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.876+0000 7f7553400640 1 -- 172.21.15.196:0/1598505483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7554156e20 msgr2=0x7f755409dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:32.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.876+0000 7f7553400640 1 --2- 172.21.15.196:0/1598505483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7554156e20 0x7f755409dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.876+0000 7f7553400640 1 -- 172.21.15.196:0/1598505483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75380039e0 msgr2=0x7f755413ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:32.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.876+0000 7f7553e00640 1 --2- 172.21.15.196:0/1598505483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75380039e0 0x7f755413ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:32.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.876+0000 7f7553400640 1 --2- 172.21.15.196:0/1598505483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75380039e0 0x7f755413ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:32.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.876+0000 7f7553400640 1 -- 172.21.15.196:0/1598505483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7554157430 con 0x7f755413f280 2024-04-28T13:32:32.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.877+0000 7f7553400640 1 --2- 172.21.15.196:0/1598505483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f755413f280 0x7f75541568e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f75400108e0 tx=0x7f7540010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:32.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.877+0000 7f754be00640 1 -- 172.21.15.196:0/1598505483 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754000f040 con 0x7f755413f280 2024-04-28T13:32:32.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.877+0000 7f754be00640 1 -- 172.21.15.196:0/1598505483 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f754000ec20 con 0x7f755413f280 2024-04-28T13:32:32.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.877+0000 7f7559800640 1 -- 172.21.15.196:0/1598505483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7554157710 con 0x7f755413f280 2024-04-28T13:32:32.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.877+0000 7f754be00640 1 -- 172.21.15.196:0/1598505483 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754001c400 con 0x7f755413f280 2024-04-28T13:32:32.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.877+0000 7f7559800640 1 -- 172.21.15.196:0/1598505483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7554150a10 con 0x7f755413f280 2024-04-28T13:32:32.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.878+0000 7f7559800640 1 -- 172.21.15.196:0/1598505483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f755406a290 con 0x7f755413f280 2024-04-28T13:32:32.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.879+0000 7f754be00640 1 -- 172.21.15.196:0/1598505483 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f754001c5a0 con 0x7f755413f280 2024-04-28T13:32:32.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.880+0000 7f754be00640 1 --2- 172.21.15.196:0/1598505483 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f753406c8f0 0x7f753406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:32.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.881+0000 7f754be00640 1 -- 172.21.15.196:0/1598505483 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f754008c2c0 con 0x7f755413f280 2024-04-28T13:32:32.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.881+0000 7f7553e00640 1 --2- 172.21.15.196:0/1598505483 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f753406c8f0 0x7f753406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:32.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.882+0000 7f7553e00640 1 --2- 172.21.15.196:0/1598505483 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f753406c8f0 0x7f753406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7544032000 tx=0x7f754402d000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:32.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:32.885+0000 7f754be00640 1 -- 172.21.15.196:0/1598505483 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7540018030 con 0x7f755413f280 2024-04-28T13:32:33.069 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.069+0000 7f7559800640 1 -- 172.21.15.196:0/1598505483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f755415ac30 con 0x7f755413f280 2024-04-28T13:32:33.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.071+0000 7f754be00640 1 -- 172.21.15.196:0/1598505483 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16718 (secure 0 0 0) 0x7f754005a6e0 con 0x7f755413f280 2024-04-28T13:32:33.071 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:33.071 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":32,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:31.669527+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:32:29.352348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-28T13:32:30.637498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:32:33.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.073+0000 7f7559800640 1 -- 172.21.15.196:0/1598505483 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f753406c8f0 msgr2=0x7f753406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:33.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.073+0000 7f7559800640 1 --2- 172.21.15.196:0/1598505483 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f753406c8f0 0x7f753406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7544032000 tx=0x7f754402d000 comp rx=0 tx=0).stop 2024-04-28T13:32:33.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.073+0000 7f7559800640 1 -- 172.21.15.196:0/1598505483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f755413f280 msgr2=0x7f75541568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:33.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.073+0000 7f7559800640 1 --2- 172.21.15.196:0/1598505483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f755413f280 0x7f75541568e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f75400108e0 tx=0x7f7540010da0 comp rx=0 tx=0).stop 2024-04-28T13:32:33.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.073+0000 7f7559800640 1 -- 172.21.15.196:0/1598505483 shutdown_connections 2024-04-28T13:32:33.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.073+0000 7f7559800640 1 --2- 172.21.15.196:0/1598505483 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f753406c8f0 0x7f753406edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:33.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.073+0000 7f7559800640 1 --2- 172.21.15.196:0/1598505483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7554156e20 0x7f755409dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:33.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.073+0000 7f7559800640 1 --2- 172.21.15.196:0/1598505483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f755413f280 0x7f75541568e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:33.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.073+0000 7f7559800640 1 --2- 172.21.15.196:0/1598505483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75380039e0 0x7f755413ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:33.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.073+0000 7f7559800640 1 -- 172.21.15.196:0/1598505483 >> 172.21.15.196:0/1598505483 conn(0x7f755406c6c0 msgr2=0x7f75540726f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:33.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.074+0000 7f7559800640 1 -- 172.21.15.196:0/1598505483 shutdown_connections 2024-04-28T13:32:33.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.074+0000 7f7559800640 1 -- 172.21.15.196:0/1598505483 wait complete. 2024-04-28T13:32:33.091 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-28T13:32:33.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.293+0000 7fd5fc200640 1 Processor -- start 2024-04-28T13:32:33.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.293+0000 7fd5fc200640 1 -- start start 2024-04-28T13:32:33.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.295+0000 7fd5fc200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd5f40d3820 0x7fd5f40b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:33.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.295+0000 7fd5fc200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5f40b60b0 0x7fd5f40d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:33.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.295+0000 7fd5fc200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5f40d3c90 con 0x7fd5f415f680 2024-04-28T13:32:33.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.295+0000 7fd5fc200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd5f415fa60 con 0x7fd5f40d3820 2024-04-28T13:32:33.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.295+0000 7fd5fc200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f40d3f50 con 0x7fd5f40b60b0 2024-04-28T13:32:33.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.295+0000 7fd5f9800640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fd5f415f680 0x7fd5f4163aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:48584/0 (socket says 172.21.15.196:48584) 2024-04-28T13:32:33.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.295+0000 7fd5f9800640 1 -- 172.21.15.196:0/254310649 learned_addr learned my addr 172.21.15.196:0/254310649 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:33.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.295+0000 7fd5f8e00640 1 --2- 172.21.15.196:0/254310649 >> v2:172.21.15.203:3300/0 conn(0x7fd5f40d3820 0x7fd5f40b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:33.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5fa200640 1 --2- 172.21.15.196:0/254310649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5f40b60b0 0x7fd5f40d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:33.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 -- 172.21.15.196:0/254310649 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2263410644 0 0) 0x7fd5f40d3c90 con 0x7fd5f415f680 2024-04-28T13:32:33.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 -- 172.21.15.196:0/254310649 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5dc003520 con 0x7fd5f415f680 2024-04-28T13:32:33.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 -- 172.21.15.196:0/254310649 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2120340700 0 0) 0x7fd5dc003520 con 0x7fd5f415f680 2024-04-28T13:32:33.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 -- 172.21.15.196:0/254310649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5f40b60b0 msgr2=0x7fd5f40d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:33.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 --2- 172.21.15.196:0/254310649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5f40b60b0 0x7fd5f40d4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:33.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 -- 172.21.15.196:0/254310649 >> v2:172.21.15.203:3300/0 conn(0x7fd5f40d3820 msgr2=0x7fd5f40b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:33.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 --2- 172.21.15.196:0/254310649 >> v2:172.21.15.203:3300/0 conn(0x7fd5f40d3820 0x7fd5f40b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:33.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 -- 172.21.15.196:0/254310649 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f40b5090 con 0x7fd5f415f680 2024-04-28T13:32:33.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 -- 172.21.15.196:0/254310649 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fd5e8001690 con 0x7fd5f415f680 2024-04-28T13:32:33.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 -- 172.21.15.196:0/254310649 >> v1:172.21.15.196:6789/0 conn(0x7fd5f415f680 legacy=0x7fd5f4163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:33.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 --2- 172.21.15.196:0/254310649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5dc0039b0 0x7fd5dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:33.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 --2- 172.21.15.196:0/254310649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5dc004800 0x7fd5dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:33.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 --2- 172.21.15.196:0/254310649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5dc0051c0 0x7fd5dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:33.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 -- 172.21.15.196:0/254310649 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f415fa60 con 0x7fd5dc0039b0 2024-04-28T13:32:33.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 -- 172.21.15.196:0/254310649 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f40d3f50 con 0x7fd5dc004800 2024-04-28T13:32:33.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f3e00640 1 -- 172.21.15.196:0/254310649 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd5dc00bf30 con 0x7fd5dc0051c0 2024-04-28T13:32:33.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5fa200640 1 --2- 172.21.15.196:0/254310649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5f40b60b0 0x7fd5f40d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:32:33.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.296+0000 7fd5f9800640 1 --2- 172.21.15.196:0/254310649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5dc0039b0 0x7fd5dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:33.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.297+0000 7fd5f8e00640 1 --2- 172.21.15.196:0/254310649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5dc004800 0x7fd5dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:33.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.297+0000 7fd5f9800640 1 -- 172.21.15.196:0/254310649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5dc0051c0 msgr2=0x7fd5dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:33.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.297+0000 7fd5fa200640 1 --2- 172.21.15.196:0/254310649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5dc0051c0 0x7fd5dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:33.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.297+0000 7fd5f9800640 1 --2- 172.21.15.196:0/254310649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5dc0051c0 0x7fd5dc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:33.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.297+0000 7fd5f9800640 1 -- 172.21.15.196:0/254310649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5dc004800 msgr2=0x7fd5dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:33.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.297+0000 7fd5f9800640 1 --2- 172.21.15.196:0/254310649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5dc004800 0x7fd5dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:33.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.297+0000 7fd5f9800640 1 -- 172.21.15.196:0/254310649 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5dc003520 con 0x7fd5dc0039b0 2024-04-28T13:32:33.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.298+0000 7fd5f9800640 1 --2- 172.21.15.196:0/254310649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5dc0039b0 0x7fd5dc003dd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800c210 tx=0x7fd5dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8a2b71922eb69e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:33.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.298+0000 7fd5f3e00640 1 -- 172.21.15.196:0/254310649 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8007d70 con 0x7fd5dc0039b0 2024-04-28T13:32:33.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.298+0000 7fd5f3e00640 1 -- 172.21.15.196:0/254310649 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5e8005ad0 con 0x7fd5dc0039b0 2024-04-28T13:32:33.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.298+0000 7fd5f3e00640 1 -- 172.21.15.196:0/254310649 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8007070 con 0x7fd5dc0039b0 2024-04-28T13:32:33.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.299+0000 7fd5fc200640 1 -- 172.21.15.196:0/254310649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5dc0039b0 msgr2=0x7fd5dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:33.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.299+0000 7fd5fc200640 1 --2- 172.21.15.196:0/254310649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5dc0039b0 0x7fd5dc003dd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800c210 tx=0x7fd5dc005830 comp rx=0 tx=0).stop 2024-04-28T13:32:33.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.299+0000 7fd5fa200640 1 -- 172.21.15.196:0/254310649 reap_dead start 2024-04-28T13:32:33.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.299+0000 7fd5fc200640 1 -- 172.21.15.196:0/254310649 shutdown_connections 2024-04-28T13:32:33.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.299+0000 7fd5fc200640 1 -- 172.21.15.196:0/254310649 >> 172.21.15.196:0/254310649 conn(0x7fd5f408eca0 msgr2=0x7fd5f409e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:33.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.299+0000 7fd5fc200640 1 -- 172.21.15.196:0/254310649 shutdown_connections 2024-04-28T13:32:33.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.299+0000 7fd5fc200640 1 -- 172.21.15.196:0/254310649 wait complete. 2024-04-28T13:32:33.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.301+0000 7fd5fc200640 1 Processor -- start 2024-04-28T13:32:33.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.301+0000 7fd5fc200640 1 -- start start 2024-04-28T13:32:33.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.302+0000 7fd5fc200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5dc0039b0 0x7fd5f41612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:33.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.302+0000 7fd5fc200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5f41607b0 0x7fd5f40b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:33.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.302+0000 7fd5fc200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5f41565f0 0x7fd5f40d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:33.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.302+0000 7fd5fc200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f40d50a0 con 0x7fd5f41565f0 2024-04-28T13:32:33.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.302+0000 7fd5fc200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f40d4f20 con 0x7fd5f41607b0 2024-04-28T13:32:33.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.302+0000 7fd5fc200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f40d5220 con 0x7fd5dc0039b0 2024-04-28T13:32:33.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.302+0000 7fd5fa200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5f41565f0 0x7fd5f40d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:33.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.302+0000 7fd5fa200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5f41565f0 0x7fd5f40d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34246/0 (socket says 172.21.15.196:34246) 2024-04-28T13:32:33.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.302+0000 7fd5fa200640 1 -- 172.21.15.196:0/2117959513 learned_addr learned my addr 172.21.15.196:0/2117959513 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:33.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.302+0000 7fd5f8e00640 1 --2- 172.21.15.196:0/2117959513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5f41607b0 0x7fd5f40b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:33.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.302+0000 7fd5fa200640 1 -- 172.21.15.196:0/2117959513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5dc0039b0 msgr2=0x7fd5f41612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:33.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.302+0000 7fd5fa200640 1 --2- 172.21.15.196:0/2117959513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5dc0039b0 0x7fd5f41612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:33.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.302+0000 7fd5fa200640 1 -- 172.21.15.196:0/2117959513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5f41607b0 msgr2=0x7fd5f40b7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:33.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.302+0000 7fd5fa200640 1 --2- 172.21.15.196:0/2117959513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5f41607b0 0x7fd5f40b7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:33.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.302+0000 7fd5fa200640 1 -- 172.21.15.196:0/2117959513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f4160c30 con 0x7fd5f41565f0 2024-04-28T13:32:33.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.303+0000 7fd5fa200640 1 --2- 172.21.15.196:0/2117959513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5f41565f0 0x7fd5f40d0f60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40b6490 tx=0x7fd5ec004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:33.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.303+0000 7fd5f1600640 1 -- 172.21.15.196:0/2117959513 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec019070 con 0x7fd5f41565f0 2024-04-28T13:32:33.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.303+0000 7fd5f1600640 1 -- 172.21.15.196:0/2117959513 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5ec007d20 con 0x7fd5f41565f0 2024-04-28T13:32:33.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.303+0000 7fd5f1600640 1 -- 172.21.15.196:0/2117959513 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec014640 con 0x7fd5f41565f0 2024-04-28T13:32:33.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.303+0000 7fd5fc200640 1 -- 172.21.15.196:0/2117959513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f40cc3e0 con 0x7fd5f41565f0 2024-04-28T13:32:33.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.303+0000 7fd5fc200640 1 -- 172.21.15.196:0/2117959513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f40cc9a0 con 0x7fd5f41565f0 2024-04-28T13:32:33.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.304+0000 7fd5fc200640 1 -- 172.21.15.196:0/2117959513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f40d2e20 con 0x7fd5f41565f0 2024-04-28T13:32:33.305 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.305+0000 7fd5f1600640 1 -- 172.21.15.196:0/2117959513 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5ec01c070 con 0x7fd5f41565f0 2024-04-28T13:32:33.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.306+0000 7fd5f1600640 1 --2- 172.21.15.196:0/2117959513 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd5d406c8f0 0x7fd5d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:33.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.306+0000 7fd5f1600640 1 -- 172.21.15.196:0/2117959513 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd5ec08b690 con 0x7fd5f41565f0 2024-04-28T13:32:33.306 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.307+0000 7fd5f9800640 1 --2- 172.21.15.196:0/2117959513 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd5d406c8f0 0x7fd5d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:33.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.308+0000 7fd5f9800640 1 --2- 172.21.15.196:0/2117959513 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd5d406c8f0 0x7fd5d406edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8033000 tx=0x7fd5e8007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:33.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.310+0000 7fd5f1600640 1 -- 172.21.15.196:0/2117959513 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd5f40d2e20 con 0x7fd5f41565f0 2024-04-28T13:32:33.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:33.548+0000 7fd5fc200640 1 -- 172.21.15.196:0/2117959513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd5f4069ae0 con 0x7fd5f41565f0 2024-04-28T13:32:34.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.232+0000 7fd5f1600640 1 -- 172.21.15.196:0/2117959513 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fd5f4069ae0 con 0x7fd5f41565f0 2024-04-28T13:32:34.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.236+0000 7fd5fc200640 1 -- 172.21.15.196:0/2117959513 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd5d406c8f0 msgr2=0x7fd5d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:34.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.236+0000 7fd5fc200640 1 --2- 172.21.15.196:0/2117959513 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd5d406c8f0 0x7fd5d406edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8033000 tx=0x7fd5e8007000 comp rx=0 tx=0).stop 2024-04-28T13:32:34.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.236+0000 7fd5fc200640 1 -- 172.21.15.196:0/2117959513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5f41565f0 msgr2=0x7fd5f40d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:34.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.236+0000 7fd5fc200640 1 --2- 172.21.15.196:0/2117959513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5f41565f0 0x7fd5f40d0f60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40b6490 tx=0x7fd5ec004840 comp rx=0 tx=0).stop 2024-04-28T13:32:34.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.236+0000 7fd5fc200640 1 -- 172.21.15.196:0/2117959513 shutdown_connections 2024-04-28T13:32:34.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.236+0000 7fd5fc200640 1 --2- 172.21.15.196:0/2117959513 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd5d406c8f0 0x7fd5d406edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:34.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.236+0000 7fd5fc200640 1 --2- 172.21.15.196:0/2117959513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5f41565f0 0x7fd5f40d0f60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:34.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.236+0000 7fd5fc200640 1 --2- 172.21.15.196:0/2117959513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5f41607b0 0x7fd5f40b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:34.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.236+0000 7fd5fc200640 1 --2- 172.21.15.196:0/2117959513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5dc0039b0 0x7fd5f41612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:34.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.236+0000 7fd5fc200640 1 -- 172.21.15.196:0/2117959513 >> 172.21.15.196:0/2117959513 conn(0x7fd5f408eca0 msgr2=0x7fd5f415a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:34.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.237+0000 7fd5fc200640 1 -- 172.21.15.196:0/2117959513 shutdown_connections 2024-04-28T13:32:34.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.237+0000 7fd5fc200640 1 -- 172.21.15.196:0/2117959513 wait complete. 2024-04-28T13:32:34.259 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-28T13:32:34.260 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-28T13:32:34.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.464+0000 7f4cbe000640 1 Processor -- start 2024-04-28T13:32:34.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.464+0000 7f4cbe000640 1 -- start start 2024-04-28T13:32:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.466+0000 7f4cbe000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4cb815f150 0x7f4cb806f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.466+0000 7f4cbe000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4cb80d17b0 0x7f4cb8072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.466+0000 7f4cbe000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cb806df10 con 0x7f4cb80b5df0 2024-04-28T13:32:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.467+0000 7f4cbe000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4cb806fda0 con 0x7f4cb80d17b0 2024-04-28T13:32:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.467+0000 7f4cbe000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4cb806e1f0 con 0x7f4cb815f150 2024-04-28T13:32:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.467+0000 7f4cbd600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4cb80d17b0 0x7f4cb8072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.467+0000 7f4cbd600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4cb80d17b0 0x7f4cb8072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34254/0 (socket says 172.21.15.196:34254) 2024-04-28T13:32:34.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.467+0000 7f4cbd600640 1 -- 172.21.15.196:0/2139484783 learned_addr learned my addr 172.21.15.196:0/2139484783 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.467+0000 7f4cbd600640 1 -- 172.21.15.196:0/2139484783 >> v2:172.21.15.203:3300/0 conn(0x7f4cb815f150 msgr2=0x7f4cb806f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.467+0000 7f4cbd600640 1 --2- 172.21.15.196:0/2139484783 >> v2:172.21.15.203:3300/0 conn(0x7f4cb815f150 0x7f4cb806f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.467+0000 7f4cbd600640 1 -- 172.21.15.196:0/2139484783 >> v1:172.21.15.203:6789/0 conn(0x7f4cb80b5df0 legacy=0x7f4cb80d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.467+0000 7f4cbd600640 1 -- 172.21.15.196:0/2139484783 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb806e4d0 con 0x7f4cb80d17b0 2024-04-28T13:32:34.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.468+0000 7f4cbd600640 1 --2- 172.21.15.196:0/2139484783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb80d17b0 0x7f4cb8072150 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80d1cb0 tx=0x7f4cb002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc16906c0a70650d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:34.469 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.469+0000 7f4cb7400640 1 -- 172.21.15.196:0/2139484783 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0036070 con 0x7f4cb80d17b0 2024-04-28T13:32:34.469 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.469+0000 7f4cb7400640 1 -- 172.21.15.196:0/2139484783 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cb0031600 con 0x7f4cb80d17b0 2024-04-28T13:32:34.469 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.469+0000 7f4cb7400640 1 -- 172.21.15.196:0/2139484783 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00319d0 con 0x7f4cb80d17b0 2024-04-28T13:32:34.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.469+0000 7f4cbe000640 1 -- 172.21.15.196:0/2139484783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb80d17b0 msgr2=0x7f4cb8072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:34.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.469+0000 7f4cbe000640 1 --2- 172.21.15.196:0/2139484783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb80d17b0 0x7f4cb8072150 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80d1cb0 tx=0x7f4cb002a6f0 comp rx=0 tx=0).stop 2024-04-28T13:32:34.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.469+0000 7f4cbe000640 1 -- 172.21.15.196:0/2139484783 shutdown_connections 2024-04-28T13:32:34.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.469+0000 7f4cbe000640 1 --2- 172.21.15.196:0/2139484783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb80d17b0 0x7f4cb8072150 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:34.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.469+0000 7f4cbe000640 1 --2- 172.21.15.196:0/2139484783 >> v2:172.21.15.203:3300/0 conn(0x7f4cb815f150 0x7f4cb806f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:34.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.469+0000 7f4cbe000640 1 -- 172.21.15.196:0/2139484783 >> 172.21.15.196:0/2139484783 conn(0x7f4cb806c6c0 msgr2=0x7f4cb806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:34.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.470+0000 7f4cbe000640 1 -- 172.21.15.196:0/2139484783 shutdown_connections 2024-04-28T13:32:34.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.470+0000 7f4cbe000640 1 -- 172.21.15.196:0/2139484783 wait complete. 2024-04-28T13:32:34.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.471+0000 7f4cbe000640 1 Processor -- start 2024-04-28T13:32:34.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.471+0000 7f4cbe000640 1 -- start start 2024-04-28T13:32:34.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.472+0000 7f4cbe000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb80b5df0 0x7f4cb814a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.472+0000 7f4cbe000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cb80d17b0 0x7f4cb80b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.472+0000 7f4cbe000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cb815f150 0x7f4cb809d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.472+0000 7f4cbe000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb815f530 con 0x7f4cb80b5df0 2024-04-28T13:32:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.472+0000 7f4cbe000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb806ea90 con 0x7f4cb815f150 2024-04-28T13:32:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.472+0000 7f4cbe000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4cb8072690 con 0x7f4cb80d17b0 2024-04-28T13:32:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.473+0000 7f4cbcc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb80b5df0 0x7f4cb814a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.473+0000 7f4cbcc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb80b5df0 0x7f4cb814a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34266/0 (socket says 172.21.15.196:34266) 2024-04-28T13:32:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.473+0000 7f4cbcc00640 1 -- 172.21.15.196:0/1123713311 learned_addr learned my addr 172.21.15.196:0/1123713311 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.473+0000 7f4cbcc00640 1 -- 172.21.15.196:0/1123713311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cb80d17b0 msgr2=0x7f4cb80b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.473+0000 7f4cbcc00640 1 --2- 172.21.15.196:0/1123713311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cb80d17b0 0x7f4cb80b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.473+0000 7f4cbd600640 1 --2- 172.21.15.196:0/1123713311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cb815f150 0x7f4cb809d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.473+0000 7f4cbcc00640 1 -- 172.21.15.196:0/1123713311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cb815f150 msgr2=0x7f4cb809d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:34.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.473+0000 7f4cbcc00640 1 --2- 172.21.15.196:0/1123713311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cb815f150 0x7f4cb809d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.473+0000 7f4cbcc00640 1 -- 172.21.15.196:0/1123713311 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb814ac70 con 0x7f4cb80b5df0 2024-04-28T13:32:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.473+0000 7f4cbcc00640 1 --2- 172.21.15.196:0/1123713311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb80b5df0 0x7f4cb814a730 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800b410 tx=0x7f4ca800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.474+0000 7f4cb4c00640 1 -- 172.21.15.196:0/1123713311 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8015020 con 0x7f4cb80b5df0 2024-04-28T13:32:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.474+0000 7f4cb4c00640 1 -- 172.21.15.196:0/1123713311 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ca8011850 con 0x7f4cb80b5df0 2024-04-28T13:32:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.474+0000 7f4cb4c00640 1 -- 172.21.15.196:0/1123713311 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8011c40 con 0x7f4cb80b5df0 2024-04-28T13:32:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.474+0000 7f4cbe000640 1 -- 172.21.15.196:0/1123713311 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb80b28a0 con 0x7f4cb80b5df0 2024-04-28T13:32:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.474+0000 7f4cbe000640 1 -- 172.21.15.196:0/1123713311 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb80b2e60 con 0x7f4cb80b5df0 2024-04-28T13:32:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.475+0000 7f4cbe000640 1 -- 172.21.15.196:0/1123713311 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb8050730 con 0x7f4cb80b5df0 2024-04-28T13:32:34.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.476+0000 7f4cb4c00640 1 -- 172.21.15.196:0/1123713311 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ca8018020 con 0x7f4cb80b5df0 2024-04-28T13:32:34.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.477+0000 7f4cb4c00640 1 --2- 172.21.15.196:0/1123713311 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4c9806c8f0 0x7f4c9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:34.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.477+0000 7f4cb4c00640 1 -- 172.21.15.196:0/1123713311 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f4ca808b9e0 con 0x7f4cb80b5df0 2024-04-28T13:32:34.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.478+0000 7f4cb7e00640 1 --2- 172.21.15.196:0/1123713311 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4c9806c8f0 0x7f4c9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:34.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.479+0000 7f4cb7e00640 1 --2- 172.21.15.196:0/1123713311 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4c9806c8f0 0x7f4c9806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80b1bc0 tx=0x7f4ca4007990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:34.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.482+0000 7f4cb4c00640 1 -- 172.21.15.196:0/1123713311 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ca801f020 con 0x7f4cb80b5df0 2024-04-28T13:32:34.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:34.723+0000 7f4cbe000640 1 -- 172.21.15.196:0/1123713311 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4cb80d14d0 con 0x7f4cb80b5df0 2024-04-28T13:32:35.126 INFO:tasks.ceph.mds.b.smithi203.stderr:2024-04-28T13:32:35.126+0000 7f9483600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-28T13:32:35.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.245+0000 7f4cb4c00640 1 -- 172.21.15.196:0/1123713311 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f4cb814ac70 con 0x7f4cb80b5df0 2024-04-28T13:32:35.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.249+0000 7f4cbe000640 1 -- 172.21.15.196:0/1123713311 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4c9806c8f0 msgr2=0x7f4c9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:35.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.249+0000 7f4cbe000640 1 --2- 172.21.15.196:0/1123713311 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4c9806c8f0 0x7f4c9806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80b1bc0 tx=0x7f4ca4007990 comp rx=0 tx=0).stop 2024-04-28T13:32:35.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.249+0000 7f4cbe000640 1 -- 172.21.15.196:0/1123713311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb80b5df0 msgr2=0x7f4cb814a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:35.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.249+0000 7f4cbe000640 1 --2- 172.21.15.196:0/1123713311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb80b5df0 0x7f4cb814a730 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800b410 tx=0x7f4ca800b8d0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.249+0000 7f4cbe000640 1 -- 172.21.15.196:0/1123713311 shutdown_connections 2024-04-28T13:32:35.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.249+0000 7f4cbe000640 1 --2- 172.21.15.196:0/1123713311 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4c9806c8f0 0x7f4c9806edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.249+0000 7f4cbe000640 1 --2- 172.21.15.196:0/1123713311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cb815f150 0x7f4cb809d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.249+0000 7f4cbe000640 1 --2- 172.21.15.196:0/1123713311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cb80d17b0 0x7f4cb80b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.250+0000 7f4cbe000640 1 --2- 172.21.15.196:0/1123713311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb80b5df0 0x7f4cb814a730 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.250+0000 7f4cbe000640 1 -- 172.21.15.196:0/1123713311 >> 172.21.15.196:0/1123713311 conn(0x7f4cb806c6c0 msgr2=0x7f4cb815c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:35.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.250+0000 7f4cbe000640 1 -- 172.21.15.196:0/1123713311 shutdown_connections 2024-04-28T13:32:35.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.250+0000 7f4cbe000640 1 -- 172.21.15.196:0/1123713311 wait complete. 2024-04-28T13:32:35.271 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-28T13:32:35.271 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:32:35.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.472+0000 7f95d1000640 1 Processor -- start 2024-04-28T13:32:35.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.472+0000 7f95d1000640 1 -- start start 2024-04-28T13:32:35.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.473+0000 7f95d1000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95cc1469f0 0x7f95cc0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:35.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.473+0000 7f95d1000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f95cc0cd420 0x7f95cc0d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:35.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.474+0000 7f95d1000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95cc160f40 con 0x7f95cc0d1240 2024-04-28T13:32:35.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.474+0000 7f95d1000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f95cc0b5960 con 0x7f95cc0cd420 2024-04-28T13:32:35.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.474+0000 7f95d1000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f95cc161ac0 con 0x7f95cc1469f0 2024-04-28T13:32:35.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.474+0000 7f95c3e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f95cc0cd420 0x7f95cc0d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:35.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.474+0000 7f95c3e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f95cc0cd420 0x7f95cc0d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34272/0 (socket says 172.21.15.196:34272) 2024-04-28T13:32:35.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.474+0000 7f95c3e00640 1 -- 172.21.15.196:0/944312164 learned_addr learned my addr 172.21.15.196:0/944312164 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:35.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.474+0000 7f95c9e00640 1 --2- 172.21.15.196:0/944312164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95cc1469f0 0x7f95cc0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:35.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.474+0000 7f95c3e00640 1 -- 172.21.15.196:0/944312164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95cc1469f0 msgr2=0x7f95cc0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:35.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.475+0000 7f95c3e00640 1 --2- 172.21.15.196:0/944312164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95cc1469f0 0x7f95cc0ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.475+0000 7f95c3e00640 1 -- 172.21.15.196:0/944312164 >> v1:172.21.15.203:6789/0 conn(0x7f95cc0d1240 legacy=0x7f95cc06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:35.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.475+0000 7f95c3e00640 1 -- 172.21.15.196:0/944312164 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc161220 con 0x7f95cc0cd420 2024-04-28T13:32:35.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.475+0000 7f95c9e00640 1 --2- 172.21.15.196:0/944312164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95cc1469f0 0x7f95cc0ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:35.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.476+0000 7f95c3e00640 1 --2- 172.21.15.196:0/944312164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95cc0cd420 0x7f95cc0d0d00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009870 tx=0x7f95b402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92cd84479a1b354b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:35.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.476+0000 7f95c9400640 1 -- 172.21.15.196:0/944312164 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4037070 con 0x7f95cc0cd420 2024-04-28T13:32:35.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.476+0000 7f95c9400640 1 -- 172.21.15.196:0/944312164 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b4032440 con 0x7f95cc0cd420 2024-04-28T13:32:35.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.476+0000 7f95c9400640 1 -- 172.21.15.196:0/944312164 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4032810 con 0x7f95cc0cd420 2024-04-28T13:32:35.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.477+0000 7f95d1000640 1 -- 172.21.15.196:0/944312164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95cc0cd420 msgr2=0x7f95cc0d0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:35.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.477+0000 7f95d1000640 1 --2- 172.21.15.196:0/944312164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95cc0cd420 0x7f95cc0d0d00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009870 tx=0x7f95b402a200 comp rx=0 tx=0).stop 2024-04-28T13:32:35.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.477+0000 7f95d1000640 1 -- 172.21.15.196:0/944312164 shutdown_connections 2024-04-28T13:32:35.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.477+0000 7f95d1000640 1 --2- 172.21.15.196:0/944312164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95cc0cd420 0x7f95cc0d0d00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.477+0000 7f95d1000640 1 --2- 172.21.15.196:0/944312164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95cc1469f0 0x7f95cc0ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.477+0000 7f95d1000640 1 -- 172.21.15.196:0/944312164 >> 172.21.15.196:0/944312164 conn(0x7f95cc0cd830 msgr2=0x7f95cc0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:35.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.477+0000 7f95d1000640 1 -- 172.21.15.196:0/944312164 shutdown_connections 2024-04-28T13:32:35.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.477+0000 7f95d1000640 1 -- 172.21.15.196:0/944312164 wait complete. 2024-04-28T13:32:35.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.479+0000 7f95d1000640 1 Processor -- start 2024-04-28T13:32:35.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.479+0000 7f95d1000640 1 -- start start 2024-04-28T13:32:35.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.480+0000 7f95d1000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95cc0cd420 0x7f95cc0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:35.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.480+0000 7f95d1000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95cc0d1240 0x7f95cc13c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:35.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.480+0000 7f95d1000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95cc1469f0 0x7f95cc144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:35.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.480+0000 7f95d1000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc072020 con 0x7f95cc1469f0 2024-04-28T13:32:35.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.480+0000 7f95d1000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc1617e0 con 0x7f95cc0cd420 2024-04-28T13:32:35.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.480+0000 7f95d1000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f95cc0b6de0 con 0x7f95cc0d1240 2024-04-28T13:32:35.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.480+0000 7f95ca800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95cc1469f0 0x7f95cc144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:35.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.480+0000 7f95ca800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95cc1469f0 0x7f95cc144770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34274/0 (socket says 172.21.15.196:34274) 2024-04-28T13:32:35.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.480+0000 7f95ca800640 1 -- 172.21.15.196:0/3050926618 learned_addr learned my addr 172.21.15.196:0/3050926618 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.480+0000 7f95c9e00640 1 --2- 172.21.15.196:0/3050926618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95cc0cd420 0x7f95cc0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.480+0000 7f95ca800640 1 -- 172.21.15.196:0/3050926618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95cc0d1240 msgr2=0x7f95cc13c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.480+0000 7f95ca800640 1 --2- 172.21.15.196:0/3050926618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95cc0d1240 0x7f95cc13c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.480+0000 7f95ca800640 1 -- 172.21.15.196:0/3050926618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95cc0cd420 msgr2=0x7f95cc0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.480+0000 7f95ca800640 1 --2- 172.21.15.196:0/3050926618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95cc0cd420 0x7f95cc0ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.480+0000 7f95ca800640 1 -- 172.21.15.196:0/3050926618 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc0cfbe0 con 0x7f95cc1469f0 2024-04-28T13:32:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.481+0000 7f95ca800640 1 --2- 172.21.15.196:0/3050926618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95cc1469f0 0x7f95cc144770 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c780 tx=0x7f95c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.481+0000 7f95c2000640 1 -- 172.21.15.196:0/3050926618 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4004b10 con 0x7f95cc1469f0 2024-04-28T13:32:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.481+0000 7f95c2000640 1 -- 172.21.15.196:0/3050926618 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c4004cb0 con 0x7f95cc1469f0 2024-04-28T13:32:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.481+0000 7f95c2000640 1 -- 172.21.15.196:0/3050926618 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c400aa90 con 0x7f95cc1469f0 2024-04-28T13:32:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.481+0000 7f95d1000640 1 -- 172.21.15.196:0/3050926618 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95cc0b1bf0 con 0x7f95cc1469f0 2024-04-28T13:32:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.481+0000 7f95d1000640 1 -- 172.21.15.196:0/3050926618 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95cc0cf2b0 con 0x7f95cc1469f0 2024-04-28T13:32:35.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.482+0000 7f95d1000640 1 -- 172.21.15.196:0/3050926618 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95cc06a290 con 0x7f95cc1469f0 2024-04-28T13:32:35.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.483+0000 7f95c2000640 1 -- 172.21.15.196:0/3050926618 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95c400ac30 con 0x7f95cc1469f0 2024-04-28T13:32:35.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.484+0000 7f95c2000640 1 --2- 172.21.15.196:0/3050926618 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f95a806c8f0 0x7f95a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:35.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.484+0000 7f95c2000640 1 -- 172.21.15.196:0/3050926618 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f95c408bd70 con 0x7f95cc1469f0 2024-04-28T13:32:35.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.485+0000 7f95c9e00640 1 --2- 172.21.15.196:0/3050926618 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f95a806c8f0 0x7f95a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:35.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.486+0000 7f95c9e00640 1 --2- 172.21.15.196:0/3050926618 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f95a806c8f0 0x7f95a806edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00b840 tx=0x7f95bc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:35.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.488+0000 7f95c2000640 1 -- 172.21.15.196:0/3050926618 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f95c4020070 con 0x7f95cc1469f0 2024-04-28T13:32:35.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.731+0000 7f95d1000640 1 -- 172.21.15.196:0/3050926618 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95cc06fd90 con 0x7f95cc1469f0 2024-04-28T13:32:35.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.732+0000 7f95c2000640 1 -- 172.21.15.196:0/3050926618 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f95c405a190 con 0x7f95cc1469f0 2024-04-28T13:32:35.733 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:35.733 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/931076983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":931076983},{"type":"v1","addr":"172.21.15.196:6834","nonce":931076983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1114474485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1114474485},{"type":"v1","addr":"172.21.15.196:6835","nonce":1114474485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2070123371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2070123371},{"type":"v1","addr":"172.21.15.203:6837","nonce":2070123371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:32:31.669472+0000","modified":"2024-04-28T13:32:34.725159+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4735":{"gid":4735,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6836/459640695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":459640695},{"type":"v1","addr":"172.21.15.203:6836","nonce":459640695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4735,"qdb_cluster":[4735]},"id":2}]} 2024-04-28T13:32:35.733 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 16 2024-04-28T13:32:35.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.735+0000 7f95d1000640 1 -- 172.21.15.196:0/3050926618 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f95a806c8f0 msgr2=0x7f95a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:35.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.735+0000 7f95d1000640 1 --2- 172.21.15.196:0/3050926618 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f95a806c8f0 0x7f95a806edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00b840 tx=0x7f95bc008040 comp rx=0 tx=0).stop 2024-04-28T13:32:35.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.735+0000 7f95d1000640 1 -- 172.21.15.196:0/3050926618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95cc1469f0 msgr2=0x7f95cc144770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:35.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.735+0000 7f95d1000640 1 --2- 172.21.15.196:0/3050926618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95cc1469f0 0x7f95cc144770 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c780 tx=0x7f95c400cc40 comp rx=0 tx=0).stop 2024-04-28T13:32:35.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.735+0000 7f95d1000640 1 -- 172.21.15.196:0/3050926618 shutdown_connections 2024-04-28T13:32:35.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.735+0000 7f95d1000640 1 --2- 172.21.15.196:0/3050926618 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f95a806c8f0 0x7f95a806edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.735+0000 7f95d1000640 1 --2- 172.21.15.196:0/3050926618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95cc1469f0 0x7f95cc144770 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.735+0000 7f95d1000640 1 --2- 172.21.15.196:0/3050926618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95cc0d1240 0x7f95cc13c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.735+0000 7f95d1000640 1 --2- 172.21.15.196:0/3050926618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95cc0cd420 0x7f95cc0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.735+0000 7f95d1000640 1 -- 172.21.15.196:0/3050926618 >> 172.21.15.196:0/3050926618 conn(0x7f95cc0cd830 msgr2=0x7f95cc06eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:35.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.735+0000 7f95d1000640 1 -- 172.21.15.196:0/3050926618 shutdown_connections 2024-04-28T13:32:35.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.736+0000 7f95d1000640 1 -- 172.21.15.196:0/3050926618 wait complete. 2024-04-28T13:32:35.754 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:32:35.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.956+0000 7f86eec00640 1 Processor -- start 2024-04-28T13:32:35.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.956+0000 7f86eec00640 1 -- start start 2024-04-28T13:32:35.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.958+0000 7f86eec00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f86e8145870 0x7f86e806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:35.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.958+0000 7f86eec00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86e806f980 0x7f86e8072d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:35.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.958+0000 7f86eec00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86e8073a40 con 0x7f86e8073250 2024-04-28T13:32:35.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.958+0000 7f86eec00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f86e806fde0 con 0x7f86e8145870 2024-04-28T13:32:35.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.958+0000 7f86eec00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f86e815d8d0 con 0x7f86e806f980 2024-04-28T13:32:35.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.958+0000 7f86e7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f86e8145870 0x7f86e806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:35.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.958+0000 7f86e7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f86e8145870 0x7f86e806f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34282/0 (socket says 172.21.15.196:34282) 2024-04-28T13:32:35.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.958+0000 7f86e7e00640 1 -- 172.21.15.196:0/2725322649 learned_addr learned my addr 172.21.15.196:0/2725322649 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:35.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.959+0000 7f86e7400640 1 --2- 172.21.15.196:0/2725322649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86e806f980 0x7f86e8072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:35.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.959+0000 7f86e7e00640 1 -- 172.21.15.196:0/2725322649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86e806f980 msgr2=0x7f86e8072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:35.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.959+0000 7f86e7e00640 1 --2- 172.21.15.196:0/2725322649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86e806f980 0x7f86e8072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.959+0000 7f86e7e00640 1 -- 172.21.15.196:0/2725322649 >> v1:172.21.15.203:6789/0 conn(0x7f86e8073250 legacy=0x7f86e815fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:35.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.959+0000 7f86e7e00640 1 -- 172.21.15.196:0/2725322649 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e815dbb0 con 0x7f86e8145870 2024-04-28T13:32:35.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.961+0000 7f86e7e00640 1 --2- 172.21.15.196:0/2725322649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8145870 0x7f86e806f440 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009870 tx=0x7f86d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd400e871c0f495 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:35.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.961+0000 7f86e6a00640 1 -- 172.21.15.196:0/2725322649 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8037070 con 0x7f86e8145870 2024-04-28T13:32:35.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.961+0000 7f86e6a00640 1 -- 172.21.15.196:0/2725322649 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d8032440 con 0x7f86e8145870 2024-04-28T13:32:35.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.961+0000 7f86e6a00640 1 -- 172.21.15.196:0/2725322649 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8032810 con 0x7f86e8145870 2024-04-28T13:32:35.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.961+0000 7f86eec00640 1 -- 172.21.15.196:0/2725322649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8145870 msgr2=0x7f86e806f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:35.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.961+0000 7f86eec00640 1 --2- 172.21.15.196:0/2725322649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8145870 0x7f86e806f440 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009870 tx=0x7f86d802a270 comp rx=0 tx=0).stop 2024-04-28T13:32:35.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.962+0000 7f86eec00640 1 -- 172.21.15.196:0/2725322649 shutdown_connections 2024-04-28T13:32:35.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.962+0000 7f86eec00640 1 --2- 172.21.15.196:0/2725322649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86e806f980 0x7f86e8072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.962+0000 7f86eec00640 1 --2- 172.21.15.196:0/2725322649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8145870 0x7f86e806f440 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.962+0000 7f86eec00640 1 -- 172.21.15.196:0/2725322649 >> 172.21.15.196:0/2725322649 conn(0x7f86e80cd830 msgr2=0x7f86e80cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:35.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.962+0000 7f86eec00640 1 -- 172.21.15.196:0/2725322649 shutdown_connections 2024-04-28T13:32:35.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.962+0000 7f86eec00640 1 -- 172.21.15.196:0/2725322649 wait complete. 2024-04-28T13:32:35.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.963+0000 7f86eec00640 1 Processor -- start 2024-04-28T13:32:35.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.964+0000 7f86eec00640 1 -- start start 2024-04-28T13:32:35.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.964+0000 7f86eec00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86e806f980 0x7f86e81561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:35.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.964+0000 7f86eec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8073250 0x7f86e8159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:35.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.964+0000 7f86eec00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86e8145870 0x7f86e8153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:35.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.964+0000 7f86eec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86e8145c50 con 0x7f86e8073250 2024-04-28T13:32:35.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.964+0000 7f86eec00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f86e815d5d0 con 0x7f86e806f980 2024-04-28T13:32:35.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.964+0000 7f86eec00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f86e80736e0 con 0x7f86e8145870 2024-04-28T13:32:35.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.965+0000 7f86e7400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8073250 0x7f86e8159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:35.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.965+0000 7f86e7400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8073250 0x7f86e8159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34300/0 (socket says 172.21.15.196:34300) 2024-04-28T13:32:35.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.965+0000 7f86e7400640 1 -- 172.21.15.196:0/34897647 learned_addr learned my addr 172.21.15.196:0/34897647 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:35.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.965+0000 7f86e7e00640 1 --2- 172.21.15.196:0/34897647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86e806f980 0x7f86e81561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:35.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.965+0000 7f86ecc00640 1 --2- 172.21.15.196:0/34897647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86e8145870 0x7f86e8153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:35.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.965+0000 7f86e7400640 1 -- 172.21.15.196:0/34897647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86e8145870 msgr2=0x7f86e8153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:35.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.965+0000 7f86e7400640 1 --2- 172.21.15.196:0/34897647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86e8145870 0x7f86e8153830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.965+0000 7f86e7400640 1 -- 172.21.15.196:0/34897647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86e806f980 msgr2=0x7f86e81561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:35.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.965+0000 7f86e7400640 1 --2- 172.21.15.196:0/34897647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86e806f980 0x7f86e81561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:35.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.965+0000 7f86e7400640 1 -- 172.21.15.196:0/34897647 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e8159650 con 0x7f86e8073250 2024-04-28T13:32:35.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.965+0000 7f86e7e00640 1 --2- 172.21.15.196:0/34897647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86e806f980 0x7f86e81561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:35.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.966+0000 7f86e7400640 1 --2- 172.21.15.196:0/34897647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8073250 0x7f86e8159110 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f86d400e930 tx=0x7f86d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:35.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.966+0000 7f86dfe00640 1 -- 172.21.15.196:0/34897647 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4017070 con 0x7f86e8073250 2024-04-28T13:32:35.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.966+0000 7f86dfe00640 1 -- 172.21.15.196:0/34897647 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d400ca00 con 0x7f86e8073250 2024-04-28T13:32:35.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.966+0000 7f86dfe00640 1 -- 172.21.15.196:0/34897647 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d400cdf0 con 0x7f86e8073250 2024-04-28T13:32:35.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.966+0000 7f86eec00640 1 -- 172.21.15.196:0/34897647 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e8158070 con 0x7f86e8073250 2024-04-28T13:32:35.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.966+0000 7f86eec00640 1 -- 172.21.15.196:0/34897647 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e8158630 con 0x7f86e8073250 2024-04-28T13:32:35.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.967+0000 7f86eec00640 1 -- 172.21.15.196:0/34897647 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e806a290 con 0x7f86e8073250 2024-04-28T13:32:35.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.968+0000 7f86dfe00640 1 -- 172.21.15.196:0/34897647 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86d401a070 con 0x7f86e8073250 2024-04-28T13:32:35.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.970+0000 7f86dfe00640 1 --2- 172.21.15.196:0/34897647 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f86c806c8f0 0x7f86c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:35.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.970+0000 7f86dfe00640 1 -- 172.21.15.196:0/34897647 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f86d408bff0 con 0x7f86e8073250 2024-04-28T13:32:35.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.970+0000 7f86e7e00640 1 --2- 172.21.15.196:0/34897647 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f86c806c8f0 0x7f86c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:35.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.971+0000 7f86e7e00640 1 --2- 172.21.15.196:0/34897647 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f86c806c8f0 0x7f86c806edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f86d8034000 tx=0x7f86d802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:35.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:35.974+0000 7f86dfe00640 1 -- 172.21.15.196:0/34897647 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86d405a440 con 0x7f86e8073250 2024-04-28T13:32:36.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.177+0000 7f86eec00640 1 -- 172.21.15.196:0/34897647 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86e815d200 con 0x7f86e8073250 2024-04-28T13:32:36.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.178+0000 7f86dfe00640 1 -- 172.21.15.196:0/34897647 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16705 (secure 0 0 0) 0x7f86d4059d40 con 0x7f86e8073250 2024-04-28T13:32:36.179 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:36.179 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":33,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:33.217952+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:32:29.352348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-28T13:32:30.637498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:32:36.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.181+0000 7f86eec00640 1 -- 172.21.15.196:0/34897647 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f86c806c8f0 msgr2=0x7f86c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:36.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.181+0000 7f86eec00640 1 --2- 172.21.15.196:0/34897647 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f86c806c8f0 0x7f86c806edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f86d8034000 tx=0x7f86d802e000 comp rx=0 tx=0).stop 2024-04-28T13:32:36.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.181+0000 7f86eec00640 1 -- 172.21.15.196:0/34897647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8073250 msgr2=0x7f86e8159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:36.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.181+0000 7f86eec00640 1 --2- 172.21.15.196:0/34897647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8073250 0x7f86e8159110 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f86d400e930 tx=0x7f86d400edf0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.181+0000 7f86eec00640 1 -- 172.21.15.196:0/34897647 shutdown_connections 2024-04-28T13:32:36.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.181+0000 7f86eec00640 1 --2- 172.21.15.196:0/34897647 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f86c806c8f0 0x7f86c806edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.181+0000 7f86eec00640 1 --2- 172.21.15.196:0/34897647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86e8145870 0x7f86e8153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.181+0000 7f86eec00640 1 --2- 172.21.15.196:0/34897647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e8073250 0x7f86e8159110 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.181+0000 7f86eec00640 1 --2- 172.21.15.196:0/34897647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86e806f980 0x7f86e81561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.181+0000 7f86eec00640 1 -- 172.21.15.196:0/34897647 >> 172.21.15.196:0/34897647 conn(0x7f86e80cd830 msgr2=0x7f86e80d10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:36.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.181+0000 7f86eec00640 1 -- 172.21.15.196:0/34897647 shutdown_connections 2024-04-28T13:32:36.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.181+0000 7f86eec00640 1 -- 172.21.15.196:0/34897647 wait complete. 2024-04-28T13:32:36.201 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-28T13:32:36.201 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T13:32:36.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.397+0000 7f5ebc800640 1 Processor -- start 2024-04-28T13:32:36.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.397+0000 7f5ebc800640 1 -- start start 2024-04-28T13:32:36.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.398+0000 7f5ebc800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5eb406fb50 0x7f5eb4072d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:36.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.398+0000 7f5ebc800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eb4073a40 con 0x7f5eb4145870 2024-04-28T13:32:36.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.398+0000 7f5ebc800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eb415d8d0 con 0x7f5eb4073250 2024-04-28T13:32:36.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.398+0000 7f5ebc800640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb415dbb0 con 0x7f5eb406fb50 2024-04-28T13:32:36.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.398+0000 7f5eb9e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f5eb4145870 0x7f5eb406f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:48590/0 (socket says 172.21.15.196:48590) 2024-04-28T13:32:36.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.398+0000 7f5eb9e00640 1 -- 172.21.15.196:0/3469031844 learned_addr learned my addr 172.21.15.196:0/3469031844 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:36.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.399+0000 7f5eb9400640 1 --2- 172.21.15.196:0/3469031844 >> v2:172.21.15.203:3300/0 conn(0x7f5eb406fb50 0x7f5eb4072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:36.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.399+0000 7f5eb8a00640 1 -- 172.21.15.196:0/3469031844 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1852286592 0 0) 0x7f5eb4073a40 con 0x7f5eb4145870 2024-04-28T13:32:36.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.399+0000 7f5eb8a00640 1 -- 172.21.15.196:0/3469031844 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e9c003520 con 0x7f5eb4145870 2024-04-28T13:32:36.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.399+0000 7f5eb8a00640 1 -- 172.21.15.196:0/3469031844 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2611671001 0 0) 0x7f5e9c003520 con 0x7f5eb4145870 2024-04-28T13:32:36.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.399+0000 7f5eb8a00640 1 -- 172.21.15.196:0/3469031844 >> v2:172.21.15.203:3300/0 conn(0x7f5eb406fb50 msgr2=0x7f5eb4072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:36.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.399+0000 7f5eb8a00640 1 --2- 172.21.15.196:0/3469031844 >> v2:172.21.15.203:3300/0 conn(0x7f5eb406fb50 0x7f5eb4072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.399+0000 7f5eb8a00640 1 -- 172.21.15.196:0/3469031844 >> v1:172.21.15.203:6789/0 conn(0x7f5eb4073250 legacy=0x7f5eb415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:36.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.399+0000 7f5eb8a00640 1 -- 172.21.15.196:0/3469031844 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb415de90 con 0x7f5eb4145870 2024-04-28T13:32:36.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.399+0000 7f5eb8a00640 1 -- 172.21.15.196:0/3469031844 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f5ea8001670 con 0x7f5eb4145870 2024-04-28T13:32:36.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.399+0000 7f5eb9400640 1 --2- 172.21.15.196:0/3469031844 >> v2:172.21.15.203:3300/0 conn(0x7f5eb406fb50 0x7f5eb4072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:32:36.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.399+0000 7f5eb8a00640 1 -- 172.21.15.196:0/3469031844 >> v1:172.21.15.196:6789/0 conn(0x7f5eb4145870 legacy=0x7f5eb406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:36.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.399+0000 7f5eb8a00640 1 --2- 172.21.15.196:0/3469031844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e9c0039b0 0x7f5e9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:36.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.399+0000 7f5eb8a00640 1 --2- 172.21.15.196:0/3469031844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e9c004800 0x7f5e9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:36.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.399+0000 7f5eb8a00640 1 --2- 172.21.15.196:0/3469031844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e9c0051c0 0x7f5e9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:36.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.399+0000 7f5eb8a00640 1 -- 172.21.15.196:0/3469031844 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb415dbb0 con 0x7f5e9c004800 2024-04-28T13:32:36.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.400+0000 7f5eb8a00640 1 -- 172.21.15.196:0/3469031844 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c00c110 con 0x7f5e9c0039b0 2024-04-28T13:32:36.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.400+0000 7f5eb8a00640 1 -- 172.21.15.196:0/3469031844 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9c00c310 con 0x7f5e9c0051c0 2024-04-28T13:32:36.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.400+0000 7f5eb9400640 1 --2- 172.21.15.196:0/3469031844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e9c004800 0x7f5e9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:36.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.400+0000 7f5eb9e00640 1 --2- 172.21.15.196:0/3469031844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e9c0039b0 0x7f5e9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:36.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.400+0000 7f5eba800640 1 --2- 172.21.15.196:0/3469031844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e9c0051c0 0x7f5e9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:36.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.400+0000 7f5eb9400640 1 -- 172.21.15.196:0/3469031844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e9c0051c0 msgr2=0x7f5e9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:36.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.400+0000 7f5eb9400640 1 --2- 172.21.15.196:0/3469031844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e9c0051c0 0x7f5e9c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.400+0000 7f5eb9400640 1 -- 172.21.15.196:0/3469031844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e9c0039b0 msgr2=0x7f5e9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:36.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.400+0000 7f5eb9400640 1 --2- 172.21.15.196:0/3469031844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e9c0039b0 0x7f5e9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.400+0000 7f5eb9400640 1 -- 172.21.15.196:0/3469031844 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9c003520 con 0x7f5e9c004800 2024-04-28T13:32:36.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.400+0000 7f5eba800640 1 -- 172.21.15.196:0/3469031844 reap_dead start 2024-04-28T13:32:36.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.401+0000 7f5eb9400640 1 --2- 172.21.15.196:0/3469031844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e9c004800 0x7f5e9c004c80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40098a0 tx=0x7f5e9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb073282bef06fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:36.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.402+0000 7f5eb8a00640 1 -- 172.21.15.196:0/3469031844 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400f040 con 0x7f5e9c004800 2024-04-28T13:32:36.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.402+0000 7f5eb8a00640 1 -- 172.21.15.196:0/3469031844 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea402ace0 con 0x7f5e9c004800 2024-04-28T13:32:36.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.402+0000 7f5eb8a00640 1 -- 172.21.15.196:0/3469031844 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4004250 con 0x7f5e9c004800 2024-04-28T13:32:36.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.402+0000 7f5ebc800640 1 -- 172.21.15.196:0/3469031844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e9c004800 msgr2=0x7f5e9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:36.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.402+0000 7f5ebc800640 1 --2- 172.21.15.196:0/3469031844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e9c004800 0x7f5e9c004c80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40098a0 tx=0x7f5e9c005830 comp rx=0 tx=0).stop 2024-04-28T13:32:36.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.402+0000 7f5ebc800640 1 -- 172.21.15.196:0/3469031844 shutdown_connections 2024-04-28T13:32:36.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.402+0000 7f5ebc800640 1 --2- 172.21.15.196:0/3469031844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e9c004800 0x7f5e9c004c80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.402+0000 7f5ebc800640 1 -- 172.21.15.196:0/3469031844 >> 172.21.15.196:0/3469031844 conn(0x7f5eb40cd830 msgr2=0x7f5eb40cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:36.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.402+0000 7f5ebc800640 1 -- 172.21.15.196:0/3469031844 shutdown_connections 2024-04-28T13:32:36.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.402+0000 7f5ebc800640 1 -- 172.21.15.196:0/3469031844 wait complete. 2024-04-28T13:32:36.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.404+0000 7f5ebc800640 1 Processor -- start 2024-04-28T13:32:36.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.404+0000 7f5ebc800640 1 -- start start 2024-04-28T13:32:36.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.405+0000 7f5ebc800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e9c004800 0x7f5eb41595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:36.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.405+0000 7f5ebc800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb41429e0 0x7f5eb41385e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:36.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.405+0000 7f5ebc800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5eb4157bd0 0x7f5eb40d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:36.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.405+0000 7f5ebc800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb40736c0 con 0x7f5eb41429e0 2024-04-28T13:32:36.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.405+0000 7f5ebc800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb4145c50 con 0x7f5eb4157bd0 2024-04-28T13:32:36.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.405+0000 7f5ebc800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb4072d10 con 0x7f5e9c004800 2024-04-28T13:32:36.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.406+0000 7f5eb9400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb41429e0 0x7f5eb41385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:36.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.406+0000 7f5eb9400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb41429e0 0x7f5eb41385e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34306/0 (socket says 172.21.15.196:34306) 2024-04-28T13:32:36.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.406+0000 7f5eb9400640 1 -- 172.21.15.196:0/1458655942 learned_addr learned my addr 172.21.15.196:0/1458655942 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:36.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.406+0000 7f5eb9e00640 1 --2- 172.21.15.196:0/1458655942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e9c004800 0x7f5eb41595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:36.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.406+0000 7f5eba800640 1 --2- 172.21.15.196:0/1458655942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5eb4157bd0 0x7f5eb40d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:36.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.406+0000 7f5eb9400640 1 -- 172.21.15.196:0/1458655942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e9c004800 msgr2=0x7f5eb41595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:36.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.406+0000 7f5eb9400640 1 --2- 172.21.15.196:0/1458655942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e9c004800 0x7f5eb41595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.406+0000 7f5eb9400640 1 -- 172.21.15.196:0/1458655942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5eb4157bd0 msgr2=0x7f5eb40d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:36.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.406+0000 7f5eb9400640 1 --2- 172.21.15.196:0/1458655942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5eb4157bd0 0x7f5eb40d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.406+0000 7f5eb9400640 1 -- 172.21.15.196:0/1458655942 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb4158520 con 0x7f5eb41429e0 2024-04-28T13:32:36.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.406+0000 7f5eb9400640 1 --2- 172.21.15.196:0/1458655942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb41429e0 0x7f5eb41385e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40098a0 tx=0x7f5ea4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:36.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.407+0000 7f5eae000640 1 -- 172.21.15.196:0/1458655942 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4041070 con 0x7f5eb41429e0 2024-04-28T13:32:36.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.407+0000 7f5eae000640 1 -- 172.21.15.196:0/1458655942 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea402aae0 con 0x7f5eb41429e0 2024-04-28T13:32:36.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.407+0000 7f5ebc800640 1 -- 172.21.15.196:0/1458655942 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb4158800 con 0x7f5eb41429e0 2024-04-28T13:32:36.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.407+0000 7f5eae000640 1 -- 172.21.15.196:0/1458655942 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea403d430 con 0x7f5eb41429e0 2024-04-28T13:32:36.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.407+0000 7f5ebc800640 1 -- 172.21.15.196:0/1458655942 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb4158dc0 con 0x7f5eb41429e0 2024-04-28T13:32:36.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.408+0000 7f5ebc800640 1 -- 172.21.15.196:0/1458655942 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb406a290 con 0x7f5eb41429e0 2024-04-28T13:32:36.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.409+0000 7f5eae000640 1 -- 172.21.15.196:0/1458655942 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ea4044070 con 0x7f5eb41429e0 2024-04-28T13:32:36.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.410+0000 7f5eae000640 1 --2- 172.21.15.196:0/1458655942 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e9806c920 0x7f5e9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:36.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.410+0000 7f5eae000640 1 -- 172.21.15.196:0/1458655942 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5ea40ac4a0 con 0x7f5eb41429e0 2024-04-28T13:32:36.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.411+0000 7f5eb9e00640 1 --2- 172.21.15.196:0/1458655942 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e9806c920 0x7f5e9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:36.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.412+0000 7f5eb9e00640 1 --2- 172.21.15.196:0/1458655942 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e9806c920 0x7f5e9806ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8001050 tx=0x7f5ea8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:36.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.414+0000 7f5eae000640 1 -- 172.21.15.196:0/1458655942 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ea407a7a0 con 0x7f5eb41429e0 2024-04-28T13:32:36.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.608+0000 7f5ebc800640 1 -- 172.21.15.196:0/1458655942 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5eb415f6e0 con 0x7f5e9806c920 2024-04-28T13:32:36.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.629+0000 7f5eae000640 1 -- 172.21.15.196:0/1458655942 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7f5eb415f6e0 con 0x7f5e9806c920 2024-04-28T13:32:36.633 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:36.635 INFO:teuthology.orchestra.run.smithi196.stderr:dumped all 2024-04-28T13:32:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.635+0000 7f5ebc800640 1 -- 172.21.15.196:0/1458655942 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e9806c920 msgr2=0x7f5e9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.635+0000 7f5ebc800640 1 --2- 172.21.15.196:0/1458655942 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e9806c920 0x7f5e9806ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8001050 tx=0x7f5ea8009040 comp rx=0 tx=0).stop 2024-04-28T13:32:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.635+0000 7f5ebc800640 1 -- 172.21.15.196:0/1458655942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb41429e0 msgr2=0x7f5eb41385e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.635+0000 7f5ebc800640 1 --2- 172.21.15.196:0/1458655942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb41429e0 0x7f5eb41385e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40098a0 tx=0x7f5ea4004730 comp rx=0 tx=0).stop 2024-04-28T13:32:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.636+0000 7f5ebc800640 1 -- 172.21.15.196:0/1458655942 shutdown_connections 2024-04-28T13:32:36.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.636+0000 7f5ebc800640 1 --2- 172.21.15.196:0/1458655942 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e9806c920 0x7f5e9806ede0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.636+0000 7f5ebc800640 1 --2- 172.21.15.196:0/1458655942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5eb4157bd0 0x7f5eb40d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.636+0000 7f5ebc800640 1 --2- 172.21.15.196:0/1458655942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5eb41429e0 0x7f5eb41385e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.636+0000 7f5ebc800640 1 --2- 172.21.15.196:0/1458655942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e9c004800 0x7f5eb41595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.636+0000 7f5ebc800640 1 -- 172.21.15.196:0/1458655942 >> 172.21.15.196:0/1458655942 conn(0x7f5eb40cd830 msgr2=0x7f5eb41432b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:36.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.636+0000 7f5ebc800640 1 -- 172.21.15.196:0/1458655942 shutdown_connections 2024-04-28T13:32:36.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.636+0000 7f5ebc800640 1 -- 172.21.15.196:0/1458655942 wait complete. 2024-04-28T13:32:36.654 INFO:teuthology.orchestra.run.smithi196.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-28T13:32:36.207449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220328,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754392,"statfs":{"total":773094113280,"available":772868497408,"internally_reserved":0,"allocated":4235264,"data_stored":2358548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002093"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.565277+0000","last_change":"2024-04-28T13:32:32.250841+0000","last_active":"2024-04-28T13:32:33.565277+0000","last_peered":"2024-04-28T13:32:33.565277+0000","last_clean":"2024-04-28T13:32:33.565277+0000","last_became_active":"2024-04-28T13:32:32.250548+0000","last_became_peered":"2024-04-28T13:32:32.250548+0000","last_unstale":"2024-04-28T13:32:33.565277+0000","last_undegraded":"2024-04-28T13:32:33.565277+0000","last_fullsized":"2024-04-28T13:32:33.565277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:00.472207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564075+0000","last_change":"2024-04-28T13:32:32.237896+0000","last_active":"2024-04-28T13:32:33.564075+0000","last_peered":"2024-04-28T13:32:33.564075+0000","last_clean":"2024-04-28T13:32:33.564075+0000","last_became_active":"2024-04-28T13:32:32.234973+0000","last_became_peered":"2024-04-28T13:32:32.234973+0000","last_unstale":"2024-04-28T13:32:33.564075+0000","last_undegraded":"2024-04-28T13:32:33.564075+0000","last_fullsized":"2024-04-28T13:32:33.564075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:23:33.274183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234976+0000","last_change":"2024-04-28T13:32:32.236328+0000","last_active":"2024-04-28T13:32:33.234976+0000","last_peered":"2024-04-28T13:32:33.234976+0000","last_clean":"2024-04-28T13:32:33.234976+0000","last_became_active":"2024-04-28T13:32:32.231979+0000","last_became_peered":"2024-04-28T13:32:32.231979+0000","last_unstale":"2024-04-28T13:32:33.234976+0000","last_undegraded":"2024-04-28T13:32:33.234976+0000","last_fullsized":"2024-04-28T13:32:33.234976+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:00:57.153237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.246658+0000","last_change":"2024-04-28T13:32:32.238192+0000","last_active":"2024-04-28T13:32:33.246658+0000","last_peered":"2024-04-28T13:32:33.246658+0000","last_clean":"2024-04-28T13:32:33.246658+0000","last_became_active":"2024-04-28T13:32:32.236991+0000","last_became_peered":"2024-04-28T13:32:32.236991+0000","last_unstale":"2024-04-28T13:32:33.246658+0000","last_undegraded":"2024-04-28T13:32:33.246658+0000","last_fullsized":"2024-04-28T13:32:33.246658+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:32:57.197848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.560941+0000","last_change":"2024-04-28T13:32:32.244713+0000","last_active":"2024-04-28T13:32:33.560941+0000","last_peered":"2024-04-28T13:32:33.560941+0000","last_clean":"2024-04-28T13:32:33.560941+0000","last_became_active":"2024-04-28T13:32:32.237452+0000","last_became_peered":"2024-04-28T13:32:32.237452+0000","last_unstale":"2024-04-28T13:32:33.560941+0000","last_undegraded":"2024-04-28T13:32:33.560941+0000","last_fullsized":"2024-04-28T13:32:33.560941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:04:08.554194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563729+0000","last_change":"2024-04-28T13:32:32.227097+0000","last_active":"2024-04-28T13:32:33.563729+0000","last_peered":"2024-04-28T13:32:33.563729+0000","last_clean":"2024-04-28T13:32:33.563729+0000","last_became_active":"2024-04-28T13:32:32.225481+0000","last_became_peered":"2024-04-28T13:32:32.225481+0000","last_unstale":"2024-04-28T13:32:33.563729+0000","last_undegraded":"2024-04-28T13:32:33.563729+0000","last_fullsized":"2024-04-28T13:32:33.563729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:00:57.153237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.565672+0000","last_change":"2024-04-28T13:32:32.252961+0000","last_active":"2024-04-28T13:32:33.565672+0000","last_peered":"2024-04-28T13:32:33.565672+0000","last_clean":"2024-04-28T13:32:33.565672+0000","last_became_active":"2024-04-28T13:32:32.252325+0000","last_became_peered":"2024-04-28T13:32:32.252325+0000","last_unstale":"2024-04-28T13:32:33.565672+0000","last_undegraded":"2024-04-28T13:32:33.565672+0000","last_fullsized":"2024-04-28T13:32:33.565672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:23:16.417118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563655+0000","last_change":"2024-04-28T13:32:32.245757+0000","last_active":"2024-04-28T13:32:33.563655+0000","last_peered":"2024-04-28T13:32:33.563655+0000","last_clean":"2024-04-28T13:32:33.563655+0000","last_became_active":"2024-04-28T13:32:32.243796+0000","last_became_peered":"2024-04-28T13:32:32.243796+0000","last_unstale":"2024-04-28T13:32:33.563655+0000","last_undegraded":"2024-04-28T13:32:33.563655+0000","last_fullsized":"2024-04-28T13:32:33.563655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:22:03.865574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.237852+0000","last_change":"2024-04-28T13:32:32.235173+0000","last_active":"2024-04-28T13:32:33.237852+0000","last_peered":"2024-04-28T13:32:33.237852+0000","last_clean":"2024-04-28T13:32:33.237852+0000","last_became_active":"2024-04-28T13:32:32.231895+0000","last_became_peered":"2024-04-28T13:32:32.231895+0000","last_unstale":"2024-04-28T13:32:33.237852+0000","last_undegraded":"2024-04-28T13:32:33.237852+0000","last_fullsized":"2024-04-28T13:32:33.237852+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:45.905694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.247025+0000","last_change":"2024-04-28T13:32:32.234270+0000","last_active":"2024-04-28T13:32:33.247025+0000","last_peered":"2024-04-28T13:32:33.247025+0000","last_clean":"2024-04-28T13:32:33.247025+0000","last_became_active":"2024-04-28T13:32:32.233820+0000","last_became_peered":"2024-04-28T13:32:32.233820+0000","last_unstale":"2024-04-28T13:32:33.247025+0000","last_undegraded":"2024-04-28T13:32:33.247025+0000","last_fullsized":"2024-04-28T13:32:33.247025+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:45.905694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564931+0000","last_change":"2024-04-28T13:32:32.247706+0000","last_active":"2024-04-28T13:32:33.564931+0000","last_peered":"2024-04-28T13:32:33.564931+0000","last_clean":"2024-04-28T13:32:33.564931+0000","last_became_active":"2024-04-28T13:32:32.247153+0000","last_became_peered":"2024-04-28T13:32:32.247153+0000","last_unstale":"2024-04-28T13:32:33.564931+0000","last_undegraded":"2024-04-28T13:32:33.564931+0000","last_fullsized":"2024-04-28T13:32:33.564931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:23:33.274183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234962+0000","last_change":"2024-04-28T13:32:32.227012+0000","last_active":"2024-04-28T13:32:33.234962+0000","last_peered":"2024-04-28T13:32:33.234962+0000","last_clean":"2024-04-28T13:32:33.234962+0000","last_became_active":"2024-04-28T13:32:32.225919+0000","last_became_peered":"2024-04-28T13:32:32.225919+0000","last_unstale":"2024-04-28T13:32:33.234962+0000","last_undegraded":"2024-04-28T13:32:33.234962+0000","last_fullsized":"2024-04-28T13:32:33.234962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:05:13.660602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.561245+0000","last_change":"2024-04-28T13:32:32.231918+0000","last_active":"2024-04-28T13:32:33.561245+0000","last_peered":"2024-04-28T13:32:33.561245+0000","last_clean":"2024-04-28T13:32:33.561245+0000","last_became_active":"2024-04-28T13:32:32.231492+0000","last_became_peered":"2024-04-28T13:32:32.231492+0000","last_unstale":"2024-04-28T13:32:33.561245+0000","last_undegraded":"2024-04-28T13:32:33.561245+0000","last_fullsized":"2024-04-28T13:32:33.561245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:20:27.174252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563947+0000","last_change":"2024-04-28T13:32:32.226886+0000","last_active":"2024-04-28T13:32:33.563947+0000","last_peered":"2024-04-28T13:32:33.563947+0000","last_clean":"2024-04-28T13:32:33.563947+0000","last_became_active":"2024-04-28T13:32:32.226468+0000","last_became_peered":"2024-04-28T13:32:32.226468+0000","last_unstale":"2024-04-28T13:32:33.563947+0000","last_undegraded":"2024-04-28T13:32:33.563947+0000","last_fullsized":"2024-04-28T13:32:33.563947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:51:21.512396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234239+0000","last_change":"2024-04-28T13:32:32.243953+0000","last_active":"2024-04-28T13:32:33.234239+0000","last_peered":"2024-04-28T13:32:33.234239+0000","last_clean":"2024-04-28T13:32:33.234239+0000","last_became_active":"2024-04-28T13:32:32.243404+0000","last_became_peered":"2024-04-28T13:32:32.243404+0000","last_unstale":"2024-04-28T13:32:33.234239+0000","last_undegraded":"2024-04-28T13:32:33.234239+0000","last_fullsized":"2024-04-28T13:32:33.234239+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:45.905694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564577+0000","last_change":"2024-04-28T13:32:32.246203+0000","last_active":"2024-04-28T13:32:33.564577+0000","last_peered":"2024-04-28T13:32:33.564577+0000","last_clean":"2024-04-28T13:32:33.564577+0000","last_became_active":"2024-04-28T13:32:32.245488+0000","last_became_peered":"2024-04-28T13:32:32.245488+0000","last_unstale":"2024-04-28T13:32:33.564577+0000","last_undegraded":"2024-04-28T13:32:33.564577+0000","last_fullsized":"2024-04-28T13:32:33.564577+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:45:01.603947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.565532+0000","last_change":"2024-04-28T13:32:32.250595+0000","last_active":"2024-04-28T13:32:33.565532+0000","last_peered":"2024-04-28T13:32:33.565532+0000","last_clean":"2024-04-28T13:32:33.565532+0000","last_became_active":"2024-04-28T13:32:32.248441+0000","last_became_peered":"2024-04-28T13:32:32.248441+0000","last_unstale":"2024-04-28T13:32:33.565532+0000","last_undegraded":"2024-04-28T13:32:33.565532+0000","last_fullsized":"2024-04-28T13:32:33.565532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:32.105900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564255+0000","last_change":"2024-04-28T13:32:32.238288+0000","last_active":"2024-04-28T13:32:33.564255+0000","last_peered":"2024-04-28T13:32:33.564255+0000","last_clean":"2024-04-28T13:32:33.564255+0000","last_became_active":"2024-04-28T13:32:32.232177+0000","last_became_peered":"2024-04-28T13:32:32.232177+0000","last_unstale":"2024-04-28T13:32:33.564255+0000","last_undegraded":"2024-04-28T13:32:33.564255+0000","last_fullsized":"2024-04-28T13:32:33.564255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:32.105900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563960+0000","last_change":"2024-04-28T13:32:32.234087+0000","last_active":"2024-04-28T13:32:33.563960+0000","last_peered":"2024-04-28T13:32:33.563960+0000","last_clean":"2024-04-28T13:32:33.563960+0000","last_became_active":"2024-04-28T13:32:32.233753+0000","last_became_peered":"2024-04-28T13:32:32.233753+0000","last_unstale":"2024-04-28T13:32:33.563960+0000","last_undegraded":"2024-04-28T13:32:33.563960+0000","last_fullsized":"2024-04-28T13:32:33.563960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:59:15.390148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.560933+0000","last_change":"2024-04-28T13:32:32.240752+0000","last_active":"2024-04-28T13:32:33.560933+0000","last_peered":"2024-04-28T13:32:33.560933+0000","last_clean":"2024-04-28T13:32:33.560933+0000","last_became_active":"2024-04-28T13:32:32.240293+0000","last_became_peered":"2024-04-28T13:32:32.240293+0000","last_unstale":"2024-04-28T13:32:33.560933+0000","last_undegraded":"2024-04-28T13:32:33.560933+0000","last_fullsized":"2024-04-28T13:32:33.560933+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:13:47.894901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236143+0000","last_change":"2024-04-28T13:32:32.236745+0000","last_active":"2024-04-28T13:32:33.236143+0000","last_peered":"2024-04-28T13:32:33.236143+0000","last_clean":"2024-04-28T13:32:33.236143+0000","last_became_active":"2024-04-28T13:32:32.236460+0000","last_became_peered":"2024-04-28T13:32:32.236460+0000","last_unstale":"2024-04-28T13:32:33.236143+0000","last_undegraded":"2024-04-28T13:32:33.236143+0000","last_fullsized":"2024-04-28T13:32:33.236143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:03:42.106373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.237089+0000","last_change":"2024-04-28T13:32:32.235265+0000","last_active":"2024-04-28T13:32:33.237089+0000","last_peered":"2024-04-28T13:32:33.237089+0000","last_clean":"2024-04-28T13:32:33.237089+0000","last_became_active":"2024-04-28T13:32:32.234950+0000","last_became_peered":"2024-04-28T13:32:32.234950+0000","last_unstale":"2024-04-28T13:32:33.237089+0000","last_undegraded":"2024-04-28T13:32:33.237089+0000","last_fullsized":"2024-04-28T13:32:33.237089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:44:10.557521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234536+0000","last_change":"2024-04-28T13:32:32.250398+0000","last_active":"2024-04-28T13:32:33.234536+0000","last_peered":"2024-04-28T13:32:33.234536+0000","last_clean":"2024-04-28T13:32:33.234536+0000","last_became_active":"2024-04-28T13:32:32.249507+0000","last_became_peered":"2024-04-28T13:32:32.249507+0000","last_unstale":"2024-04-28T13:32:33.234536+0000","last_undegraded":"2024-04-28T13:32:33.234536+0000","last_fullsized":"2024-04-28T13:32:33.234536+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:00.472207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234127+0000","last_change":"2024-04-28T13:32:32.228861+0000","last_active":"2024-04-28T13:32:33.234127+0000","last_peered":"2024-04-28T13:32:33.234127+0000","last_clean":"2024-04-28T13:32:33.234127+0000","last_became_active":"2024-04-28T13:32:32.227846+0000","last_became_peered":"2024-04-28T13:32:32.227846+0000","last_unstale":"2024-04-28T13:32:33.234127+0000","last_undegraded":"2024-04-28T13:32:33.234127+0000","last_fullsized":"2024-04-28T13:32:33.234127+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:44:10.557521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.247025+0000","last_change":"2024-04-28T13:32:32.248140+0000","last_active":"2024-04-28T13:32:33.247025+0000","last_peered":"2024-04-28T13:32:33.247025+0000","last_clean":"2024-04-28T13:32:33.247025+0000","last_became_active":"2024-04-28T13:32:32.247780+0000","last_became_peered":"2024-04-28T13:32:32.247780+0000","last_unstale":"2024-04-28T13:32:33.247025+0000","last_undegraded":"2024-04-28T13:32:33.247025+0000","last_fullsized":"2024-04-28T13:32:33.247025+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:44:10.557521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564345+0000","last_change":"2024-04-28T13:32:32.244003+0000","last_active":"2024-04-28T13:32:33.564345+0000","last_peered":"2024-04-28T13:32:33.564345+0000","last_clean":"2024-04-28T13:32:33.564345+0000","last_became_active":"2024-04-28T13:32:32.243684+0000","last_became_peered":"2024-04-28T13:32:32.243684+0000","last_unstale":"2024-04-28T13:32:33.564345+0000","last_undegraded":"2024-04-28T13:32:33.564345+0000","last_fullsized":"2024-04-28T13:32:33.564345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:23:16.417118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.246849+0000","last_change":"2024-04-28T13:32:32.242488+0000","last_active":"2024-04-28T13:32:33.246849+0000","last_peered":"2024-04-28T13:32:33.246849+0000","last_clean":"2024-04-28T13:32:33.246849+0000","last_became_active":"2024-04-28T13:32:32.241505+0000","last_became_peered":"2024-04-28T13:32:32.241505+0000","last_unstale":"2024-04-28T13:32:33.246849+0000","last_undegraded":"2024-04-28T13:32:33.246849+0000","last_fullsized":"2024-04-28T13:32:33.246849+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:47:42.691584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235369+0000","last_change":"2024-04-28T13:32:32.230977+0000","last_active":"2024-04-28T13:32:33.235369+0000","last_peered":"2024-04-28T13:32:33.235369+0000","last_clean":"2024-04-28T13:32:33.235369+0000","last_became_active":"2024-04-28T13:32:32.230695+0000","last_became_peered":"2024-04-28T13:32:32.230695+0000","last_unstale":"2024-04-28T13:32:33.235369+0000","last_undegraded":"2024-04-28T13:32:33.235369+0000","last_fullsized":"2024-04-28T13:32:33.235369+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:45:01.603947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236302+0000","last_change":"2024-04-28T13:32:32.234401+0000","last_active":"2024-04-28T13:32:33.236302+0000","last_peered":"2024-04-28T13:32:33.236302+0000","last_clean":"2024-04-28T13:32:33.236302+0000","last_became_active":"2024-04-28T13:32:32.234087+0000","last_became_peered":"2024-04-28T13:32:32.234087+0000","last_unstale":"2024-04-28T13:32:33.236302+0000","last_undegraded":"2024-04-28T13:32:33.236302+0000","last_fullsized":"2024-04-28T13:32:33.236302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:32:57.197848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564749+0000","last_change":"2024-04-28T13:32:32.249322+0000","last_active":"2024-04-28T13:32:33.564749+0000","last_peered":"2024-04-28T13:32:33.564749+0000","last_clean":"2024-04-28T13:32:33.564749+0000","last_became_active":"2024-04-28T13:32:32.249015+0000","last_became_peered":"2024-04-28T13:32:32.249015+0000","last_unstale":"2024-04-28T13:32:33.564749+0000","last_undegraded":"2024-04-28T13:32:33.564749+0000","last_fullsized":"2024-04-28T13:32:33.564749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:51:21.512396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563635+0000","last_change":"2024-04-28T13:32:32.238482+0000","last_active":"2024-04-28T13:32:33.563635+0000","last_peered":"2024-04-28T13:32:33.563635+0000","last_clean":"2024-04-28T13:32:33.563635+0000","last_became_active":"2024-04-28T13:32:32.238169+0000","last_became_peered":"2024-04-28T13:32:32.238169+0000","last_unstale":"2024-04-28T13:32:33.563635+0000","last_undegraded":"2024-04-28T13:32:33.563635+0000","last_fullsized":"2024-04-28T13:32:33.563635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:33:54.706283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563598+0000","last_change":"2024-04-28T13:32:32.251390+0000","last_active":"2024-04-28T13:32:33.563598+0000","last_peered":"2024-04-28T13:32:33.563598+0000","last_clean":"2024-04-28T13:32:33.563598+0000","last_became_active":"2024-04-28T13:32:32.251076+0000","last_became_peered":"2024-04-28T13:32:32.251076+0000","last_unstale":"2024-04-28T13:32:33.563598+0000","last_undegraded":"2024-04-28T13:32:33.563598+0000","last_fullsized":"2024-04-28T13:32:33.563598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:05:13.660602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235273+0000","last_change":"2024-04-28T13:32:32.252823+0000","last_active":"2024-04-28T13:32:33.235273+0000","last_peered":"2024-04-28T13:32:33.235273+0000","last_clean":"2024-04-28T13:32:33.235273+0000","last_became_active":"2024-04-28T13:32:32.252488+0000","last_became_peered":"2024-04-28T13:32:32.252488+0000","last_unstale":"2024-04-28T13:32:33.235273+0000","last_undegraded":"2024-04-28T13:32:33.235273+0000","last_fullsized":"2024-04-28T13:32:33.235273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:33:41.458231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.565421+0000","last_change":"2024-04-28T13:32:31.228570+0000","last_active":"2024-04-28T13:32:33.565421+0000","last_peered":"2024-04-28T13:32:33.565421+0000","last_clean":"2024-04-28T13:32:33.565421+0000","last_became_active":"2024-04-28T13:32:31.228251+0000","last_became_peered":"2024-04-28T13:32:31.228251+0000","last_unstale":"2024-04-28T13:32:33.565421+0000","last_undegraded":"2024-04-28T13:32:33.565421+0000","last_fullsized":"2024-04-28T13:32:33.565421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:32:13.306868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.565437+0000","last_change":"2024-04-28T13:32:32.243756+0000","last_active":"2024-04-28T13:32:33.565437+0000","last_peered":"2024-04-28T13:32:33.565437+0000","last_clean":"2024-04-28T13:32:33.565437+0000","last_became_active":"2024-04-28T13:32:32.243474+0000","last_became_peered":"2024-04-28T13:32:32.243474+0000","last_unstale":"2024-04-28T13:32:33.565437+0000","last_undegraded":"2024-04-28T13:32:33.565437+0000","last_fullsized":"2024-04-28T13:32:33.565437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:41:57.017489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235017+0000","last_change":"2024-04-28T13:32:31.229138+0000","last_active":"2024-04-28T13:32:33.235017+0000","last_peered":"2024-04-28T13:32:33.235017+0000","last_clean":"2024-04-28T13:32:33.235017+0000","last_became_active":"2024-04-28T13:32:31.228859+0000","last_became_peered":"2024-04-28T13:32:31.228859+0000","last_unstale":"2024-04-28T13:32:33.235017+0000","last_undegraded":"2024-04-28T13:32:33.235017+0000","last_fullsized":"2024-04-28T13:32:33.235017+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:20:26.159036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236652+0000","last_change":"2024-04-28T13:32:32.237721+0000","last_active":"2024-04-28T13:32:33.236652+0000","last_peered":"2024-04-28T13:32:33.236652+0000","last_clean":"2024-04-28T13:32:33.236652+0000","last_became_active":"2024-04-28T13:32:32.237247+0000","last_became_peered":"2024-04-28T13:32:32.237247+0000","last_unstale":"2024-04-28T13:32:33.236652+0000","last_undegraded":"2024-04-28T13:32:33.236652+0000","last_fullsized":"2024-04-28T13:32:33.236652+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:47:42.691584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563963+0000","last_change":"2024-04-28T13:32:31.234407+0000","last_active":"2024-04-28T13:32:33.563963+0000","last_peered":"2024-04-28T13:32:33.563963+0000","last_clean":"2024-04-28T13:32:33.563963+0000","last_became_active":"2024-04-28T13:32:31.234100+0000","last_became_peered":"2024-04-28T13:32:31.234100+0000","last_unstale":"2024-04-28T13:32:33.563963+0000","last_undegraded":"2024-04-28T13:32:33.563963+0000","last_fullsized":"2024-04-28T13:32:33.563963+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:28:59.456991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.246947+0000","last_change":"2024-04-28T13:32:32.228659+0000","last_active":"2024-04-28T13:32:33.246947+0000","last_peered":"2024-04-28T13:32:33.246947+0000","last_clean":"2024-04-28T13:32:33.246947+0000","last_became_active":"2024-04-28T13:32:32.228041+0000","last_became_peered":"2024-04-28T13:32:32.228041+0000","last_unstale":"2024-04-28T13:32:33.246947+0000","last_undegraded":"2024-04-28T13:32:33.246947+0000","last_fullsized":"2024-04-28T13:32:33.246947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:34:54.602566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.561029+0000","last_change":"2024-04-28T13:32:31.228812+0000","last_active":"2024-04-28T13:32:33.561029+0000","last_peered":"2024-04-28T13:32:33.561029+0000","last_clean":"2024-04-28T13:32:33.561029+0000","last_became_active":"2024-04-28T13:32:31.228397+0000","last_became_peered":"2024-04-28T13:32:31.228397+0000","last_unstale":"2024-04-28T13:32:33.561029+0000","last_undegraded":"2024-04-28T13:32:33.561029+0000","last_fullsized":"2024-04-28T13:32:33.561029+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:22:52.708165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564775+0000","last_change":"2024-04-28T13:32:32.244639+0000","last_active":"2024-04-28T13:32:33.564775+0000","last_peered":"2024-04-28T13:32:33.564775+0000","last_clean":"2024-04-28T13:32:33.564775+0000","last_became_active":"2024-04-28T13:32:32.244274+0000","last_became_peered":"2024-04-28T13:32:32.244274+0000","last_unstale":"2024-04-28T13:32:33.564775+0000","last_undegraded":"2024-04-28T13:32:33.564775+0000","last_fullsized":"2024-04-28T13:32:33.564775+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:59:15.390148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236213+0000","last_change":"2024-04-28T13:32:31.223055+0000","last_active":"2024-04-28T13:32:33.236213+0000","last_peered":"2024-04-28T13:32:33.236213+0000","last_clean":"2024-04-28T13:32:33.236213+0000","last_became_active":"2024-04-28T13:32:31.222713+0000","last_became_peered":"2024-04-28T13:32:31.222713+0000","last_unstale":"2024-04-28T13:32:33.236213+0000","last_undegraded":"2024-04-28T13:32:33.236213+0000","last_fullsized":"2024-04-28T13:32:33.236213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:32:13.306868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234789+0000","last_change":"2024-04-28T13:32:32.235548+0000","last_active":"2024-04-28T13:32:33.234789+0000","last_peered":"2024-04-28T13:32:33.234789+0000","last_clean":"2024-04-28T13:32:33.234789+0000","last_became_active":"2024-04-28T13:32:32.233102+0000","last_became_peered":"2024-04-28T13:32:32.233102+0000","last_unstale":"2024-04-28T13:32:33.234789+0000","last_undegraded":"2024-04-28T13:32:33.234789+0000","last_fullsized":"2024-04-28T13:32:33.234789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:14:48.471231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236137+0000","last_change":"2024-04-28T13:32:31.226888+0000","last_active":"2024-04-28T13:32:33.236137+0000","last_peered":"2024-04-28T13:32:33.236137+0000","last_clean":"2024-04-28T13:32:33.236137+0000","last_became_active":"2024-04-28T13:32:31.225183+0000","last_became_peered":"2024-04-28T13:32:31.225183+0000","last_unstale":"2024-04-28T13:32:33.236137+0000","last_undegraded":"2024-04-28T13:32:33.236137+0000","last_fullsized":"2024-04-28T13:32:33.236137+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:58.653633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235625+0000","last_change":"2024-04-28T13:32:32.243447+0000","last_active":"2024-04-28T13:32:33.235625+0000","last_peered":"2024-04-28T13:32:33.235625+0000","last_clean":"2024-04-28T13:32:33.235625+0000","last_became_active":"2024-04-28T13:32:32.243074+0000","last_became_peered":"2024-04-28T13:32:32.243074+0000","last_unstale":"2024-04-28T13:32:33.235625+0000","last_undegraded":"2024-04-28T13:32:33.235625+0000","last_fullsized":"2024-04-28T13:32:33.235625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:01:56.621985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563331+0000","last_change":"2024-04-28T13:32:31.223766+0000","last_active":"2024-04-28T13:32:33.563331+0000","last_peered":"2024-04-28T13:32:33.563331+0000","last_clean":"2024-04-28T13:32:33.563331+0000","last_became_active":"2024-04-28T13:32:31.223546+0000","last_became_peered":"2024-04-28T13:32:31.223546+0000","last_unstale":"2024-04-28T13:32:33.563331+0000","last_undegraded":"2024-04-28T13:32:33.563331+0000","last_fullsized":"2024-04-28T13:32:33.563331+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:33:53.691067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.245939+0000","last_change":"2024-04-28T13:32:32.242078+0000","last_active":"2024-04-28T13:32:33.245939+0000","last_peered":"2024-04-28T13:32:33.245939+0000","last_clean":"2024-04-28T13:32:33.245939+0000","last_became_active":"2024-04-28T13:32:32.241566+0000","last_became_peered":"2024-04-28T13:32:32.241566+0000","last_unstale":"2024-04-28T13:32:33.245939+0000","last_undegraded":"2024-04-28T13:32:33.245939+0000","last_fullsized":"2024-04-28T13:32:33.245939+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:03:42.106373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235617+0000","last_change":"2024-04-28T13:32:31.226887+0000","last_active":"2024-04-28T13:32:33.235617+0000","last_peered":"2024-04-28T13:32:33.235617+0000","last_clean":"2024-04-28T13:32:33.235617+0000","last_became_active":"2024-04-28T13:32:31.224680+0000","last_became_peered":"2024-04-28T13:32:31.224680+0000","last_unstale":"2024-04-28T13:32:33.235617+0000","last_undegraded":"2024-04-28T13:32:33.235617+0000","last_fullsized":"2024-04-28T13:32:33.235617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:04:07.538978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235508+0000","last_change":"2024-04-28T13:32:32.230272+0000","last_active":"2024-04-28T13:32:33.235508+0000","last_peered":"2024-04-28T13:32:33.235508+0000","last_clean":"2024-04-28T13:32:33.235508+0000","last_became_active":"2024-04-28T13:32:32.229747+0000","last_became_peered":"2024-04-28T13:32:32.229747+0000","last_unstale":"2024-04-28T13:32:33.235508+0000","last_undegraded":"2024-04-28T13:32:33.235508+0000","last_fullsized":"2024-04-28T13:32:33.235508+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:51:21.512396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564421+0000","last_change":"2024-04-28T13:32:31.232102+0000","last_active":"2024-04-28T13:32:33.564421+0000","last_peered":"2024-04-28T13:32:33.564421+0000","last_clean":"2024-04-28T13:32:33.564421+0000","last_became_active":"2024-04-28T13:32:31.231791+0000","last_became_peered":"2024-04-28T13:32:31.231791+0000","last_unstale":"2024-04-28T13:32:33.564421+0000","last_undegraded":"2024-04-28T13:32:33.564421+0000","last_fullsized":"2024-04-28T13:32:33.564421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:41:56.002273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564344+0000","last_change":"2024-04-28T13:32:32.233805+0000","last_active":"2024-04-28T13:32:33.564344+0000","last_peered":"2024-04-28T13:32:33.564344+0000","last_clean":"2024-04-28T13:32:33.564344+0000","last_became_active":"2024-04-28T13:32:32.233506+0000","last_became_peered":"2024-04-28T13:32:32.233506+0000","last_unstale":"2024-04-28T13:32:33.564344+0000","last_undegraded":"2024-04-28T13:32:33.564344+0000","last_fullsized":"2024-04-28T13:32:33.564344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:02:39.028860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235130+0000","last_change":"2024-04-28T13:32:31.224724+0000","last_active":"2024-04-28T13:32:33.235130+0000","last_peered":"2024-04-28T13:32:33.235130+0000","last_clean":"2024-04-28T13:32:33.235130+0000","last_became_active":"2024-04-28T13:32:31.222741+0000","last_became_peered":"2024-04-28T13:32:31.222741+0000","last_unstale":"2024-04-28T13:32:33.235130+0000","last_undegraded":"2024-04-28T13:32:33.235130+0000","last_fullsized":"2024-04-28T13:32:33.235130+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:24:06.561968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564263+0000","last_change":"2024-04-28T13:32:32.245208+0000","last_active":"2024-04-28T13:32:33.564263+0000","last_peered":"2024-04-28T13:32:33.564263+0000","last_clean":"2024-04-28T13:32:33.564263+0000","last_became_active":"2024-04-28T13:32:32.244813+0000","last_became_peered":"2024-04-28T13:32:32.244813+0000","last_unstale":"2024-04-28T13:32:33.564263+0000","last_undegraded":"2024-04-28T13:32:33.564263+0000","last_fullsized":"2024-04-28T13:32:33.564263+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:22:58.887460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235099+0000","last_change":"2024-04-28T13:32:31.229553+0000","last_active":"2024-04-28T13:32:33.235099+0000","last_peered":"2024-04-28T13:32:33.235099+0000","last_clean":"2024-04-28T13:32:33.235099+0000","last_became_active":"2024-04-28T13:32:31.229130+0000","last_became_peered":"2024-04-28T13:32:31.229130+0000","last_unstale":"2024-04-28T13:32:33.235099+0000","last_undegraded":"2024-04-28T13:32:33.235099+0000","last_fullsized":"2024-04-28T13:32:33.235099+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:08:17.119169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234962+0000","last_change":"2024-04-28T13:32:32.231690+0000","last_active":"2024-04-28T13:32:33.234962+0000","last_peered":"2024-04-28T13:32:33.234962+0000","last_clean":"2024-04-28T13:32:33.234962+0000","last_became_active":"2024-04-28T13:32:32.231444+0000","last_became_peered":"2024-04-28T13:32:32.231444+0000","last_unstale":"2024-04-28T13:32:33.234962+0000","last_undegraded":"2024-04-28T13:32:33.234962+0000","last_fullsized":"2024-04-28T13:32:33.234962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:22:03.865574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564339+0000","last_change":"2024-04-28T13:32:31.228938+0000","last_active":"2024-04-28T13:32:33.564339+0000","last_peered":"2024-04-28T13:32:33.564339+0000","last_clean":"2024-04-28T13:32:33.564339+0000","last_became_active":"2024-04-28T13:32:31.228699+0000","last_became_peered":"2024-04-28T13:32:31.228699+0000","last_unstale":"2024-04-28T13:32:33.564339+0000","last_undegraded":"2024-04-28T13:32:33.564339+0000","last_fullsized":"2024-04-28T13:32:33.564339+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:33:40.443015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.246794+0000","last_change":"2024-04-28T13:32:32.236711+0000","last_active":"2024-04-28T13:32:33.246794+0000","last_peered":"2024-04-28T13:32:33.246794+0000","last_clean":"2024-04-28T13:32:33.246794+0000","last_became_active":"2024-04-28T13:32:32.236383+0000","last_became_peered":"2024-04-28T13:32:32.236383+0000","last_unstale":"2024-04-28T13:32:33.246794+0000","last_undegraded":"2024-04-28T13:32:33.246794+0000","last_fullsized":"2024-04-28T13:32:33.246794+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:53:48.013290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236347+0000","last_change":"2024-04-28T13:32:31.224596+0000","last_active":"2024-04-28T13:32:33.236347+0000","last_peered":"2024-04-28T13:32:33.236347+0000","last_clean":"2024-04-28T13:32:33.236347+0000","last_became_active":"2024-04-28T13:32:31.224300+0000","last_became_peered":"2024-04-28T13:32:31.224300+0000","last_unstale":"2024-04-28T13:32:33.236347+0000","last_undegraded":"2024-04-28T13:32:33.236347+0000","last_fullsized":"2024-04-28T13:32:33.236347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:12:10.305333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563792+0000","last_change":"2024-04-28T13:32:32.232144+0000","last_active":"2024-04-28T13:32:33.563792+0000","last_peered":"2024-04-28T13:32:33.563792+0000","last_clean":"2024-04-28T13:32:33.563792+0000","last_became_active":"2024-04-28T13:32:32.231852+0000","last_became_peered":"2024-04-28T13:32:32.231852+0000","last_unstale":"2024-04-28T13:32:33.563792+0000","last_undegraded":"2024-04-28T13:32:33.563792+0000","last_fullsized":"2024-04-28T13:32:33.563792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:45:01.603947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.245825+0000","last_change":"2024-04-28T13:32:31.230105+0000","last_active":"2024-04-28T13:32:33.245825+0000","last_peered":"2024-04-28T13:32:33.245825+0000","last_clean":"2024-04-28T13:32:33.245825+0000","last_became_active":"2024-04-28T13:32:31.229764+0000","last_became_peered":"2024-04-28T13:32:31.229764+0000","last_unstale":"2024-04-28T13:32:33.245825+0000","last_undegraded":"2024-04-28T13:32:33.245825+0000","last_fullsized":"2024-04-28T13:32:33.245825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:58.653633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564019+0000","last_change":"2024-04-28T13:32:32.245969+0000","last_active":"2024-04-28T13:32:33.564019+0000","last_peered":"2024-04-28T13:32:33.564019+0000","last_clean":"2024-04-28T13:32:33.564019+0000","last_became_active":"2024-04-28T13:32:32.245095+0000","last_became_peered":"2024-04-28T13:32:32.245095+0000","last_unstale":"2024-04-28T13:32:33.564019+0000","last_undegraded":"2024-04-28T13:32:33.564019+0000","last_fullsized":"2024-04-28T13:32:33.564019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:00:57.153237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563352+0000","last_change":"2024-04-28T13:32:31.230987+0000","last_active":"2024-04-28T13:32:33.563352+0000","last_peered":"2024-04-28T13:32:33.563352+0000","last_clean":"2024-04-28T13:32:33.563352+0000","last_became_active":"2024-04-28T13:32:31.230756+0000","last_became_peered":"2024-04-28T13:32:31.230756+0000","last_unstale":"2024-04-28T13:32:33.563352+0000","last_undegraded":"2024-04-28T13:32:33.563352+0000","last_fullsized":"2024-04-28T13:32:33.563352+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:08:17.119169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563814+0000","last_change":"2024-04-28T13:32:32.235161+0000","last_active":"2024-04-28T13:32:33.563814+0000","last_peered":"2024-04-28T13:32:33.563814+0000","last_clean":"2024-04-28T13:32:33.563814+0000","last_became_active":"2024-04-28T13:32:32.234838+0000","last_became_peered":"2024-04-28T13:32:32.234838+0000","last_unstale":"2024-04-28T13:32:33.563814+0000","last_undegraded":"2024-04-28T13:32:33.563814+0000","last_fullsized":"2024-04-28T13:32:33.563814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:08:18.134385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563369+0000","last_change":"2024-04-28T13:32:31.233385+0000","last_active":"2024-04-28T13:32:33.563369+0000","last_peered":"2024-04-28T13:32:33.563369+0000","last_clean":"2024-04-28T13:32:33.563369+0000","last_became_active":"2024-04-28T13:32:31.233075+0000","last_became_peered":"2024-04-28T13:32:31.233075+0000","last_unstale":"2024-04-28T13:32:33.563369+0000","last_undegraded":"2024-04-28T13:32:33.563369+0000","last_fullsized":"2024-04-28T13:32:33.563369+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:20:51.755318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236569+0000","last_change":"2024-04-28T13:32:32.237097+0000","last_active":"2024-04-28T13:32:33.236569+0000","last_peered":"2024-04-28T13:32:33.236569+0000","last_clean":"2024-04-28T13:32:33.236569+0000","last_became_active":"2024-04-28T13:32:32.236693+0000","last_became_peered":"2024-04-28T13:32:32.236693+0000","last_unstale":"2024-04-28T13:32:33.236569+0000","last_undegraded":"2024-04-28T13:32:33.236569+0000","last_fullsized":"2024-04-28T13:32:33.236569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:53:48.013290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.246997+0000","last_change":"2024-04-28T13:32:31.223266+0000","last_active":"2024-04-28T13:32:33.246997+0000","last_peered":"2024-04-28T13:32:33.246997+0000","last_clean":"2024-04-28T13:32:33.246997+0000","last_became_active":"2024-04-28T13:32:31.222951+0000","last_became_peered":"2024-04-28T13:32:31.222951+0000","last_unstale":"2024-04-28T13:32:33.246997+0000","last_undegraded":"2024-04-28T13:32:33.246997+0000","last_fullsized":"2024-04-28T13:32:33.246997+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:24:06.561968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235772+0000","last_change":"2024-04-28T13:32:32.229844+0000","last_active":"2024-04-28T13:32:33.235772+0000","last_peered":"2024-04-28T13:32:33.235772+0000","last_clean":"2024-04-28T13:32:33.235772+0000","last_became_active":"2024-04-28T13:32:32.228009+0000","last_became_peered":"2024-04-28T13:32:32.228009+0000","last_unstale":"2024-04-28T13:32:33.235772+0000","last_undegraded":"2024-04-28T13:32:33.235772+0000","last_fullsized":"2024-04-28T13:32:33.235772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:23:33.274183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236655+0000","last_change":"2024-04-28T13:32:31.226304+0000","last_active":"2024-04-28T13:32:33.236655+0000","last_peered":"2024-04-28T13:32:33.236655+0000","last_clean":"2024-04-28T13:32:33.236655+0000","last_became_active":"2024-04-28T13:32:31.225972+0000","last_became_peered":"2024-04-28T13:32:31.225972+0000","last_unstale":"2024-04-28T13:32:33.236655+0000","last_undegraded":"2024-04-28T13:32:33.236655+0000","last_fullsized":"2024-04-28T13:32:33.236655+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:13:46.879685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.247117+0000","last_change":"2024-04-28T13:32:32.238416+0000","last_active":"2024-04-28T13:32:33.247117+0000","last_peered":"2024-04-28T13:32:33.247117+0000","last_clean":"2024-04-28T13:32:33.247117+0000","last_became_active":"2024-04-28T13:32:32.237247+0000","last_became_peered":"2024-04-28T13:32:32.237247+0000","last_unstale":"2024-04-28T13:32:33.247117+0000","last_undegraded":"2024-04-28T13:32:33.247117+0000","last_fullsized":"2024-04-28T13:32:33.247117+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:14:48.471231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236749+0000","last_change":"2024-04-28T13:32:31.216936+0000","last_active":"2024-04-28T13:32:33.236749+0000","last_peered":"2024-04-28T13:32:33.236749+0000","last_clean":"2024-04-28T13:32:33.236749+0000","last_became_active":"2024-04-28T13:32:31.214638+0000","last_became_peered":"2024-04-28T13:32:31.214638+0000","last_unstale":"2024-04-28T13:32:33.236749+0000","last_undegraded":"2024-04-28T13:32:33.236749+0000","last_fullsized":"2024-04-28T13:32:33.236749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:20:26.159036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.561202+0000","last_change":"2024-04-28T13:32:32.245218+0000","last_active":"2024-04-28T13:32:33.561202+0000","last_peered":"2024-04-28T13:32:33.561202+0000","last_clean":"2024-04-28T13:32:33.561202+0000","last_became_active":"2024-04-28T13:32:32.244867+0000","last_became_peered":"2024-04-28T13:32:32.244867+0000","last_unstale":"2024-04-28T13:32:33.561202+0000","last_undegraded":"2024-04-28T13:32:33.561202+0000","last_fullsized":"2024-04-28T13:32:33.561202+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:12:11.320549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236582+0000","last_change":"2024-04-28T13:32:31.225015+0000","last_active":"2024-04-28T13:32:33.236582+0000","last_peered":"2024-04-28T13:32:33.236582+0000","last_clean":"2024-04-28T13:32:33.236582+0000","last_became_active":"2024-04-28T13:32:31.224704+0000","last_became_peered":"2024-04-28T13:32:31.224704+0000","last_unstale":"2024-04-28T13:32:33.236582+0000","last_undegraded":"2024-04-28T13:32:33.236582+0000","last_fullsized":"2024-04-28T13:32:33.236582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:24:06.561968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.560877+0000","last_change":"2024-04-28T13:32:32.232786+0000","last_active":"2024-04-28T13:32:33.560877+0000","last_peered":"2024-04-28T13:32:33.560877+0000","last_clean":"2024-04-28T13:32:33.560877+0000","last_became_active":"2024-04-28T13:32:32.232492+0000","last_became_peered":"2024-04-28T13:32:32.232492+0000","last_unstale":"2024-04-28T13:32:33.560877+0000","last_undegraded":"2024-04-28T13:32:33.560877+0000","last_fullsized":"2024-04-28T13:32:33.560877+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:11:02.849386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235178+0000","last_change":"2024-04-28T13:32:31.221816+0000","last_active":"2024-04-28T13:32:33.235178+0000","last_peered":"2024-04-28T13:32:33.235178+0000","last_clean":"2024-04-28T13:32:33.235178+0000","last_became_active":"2024-04-28T13:32:31.221471+0000","last_became_peered":"2024-04-28T13:32:31.221471+0000","last_unstale":"2024-04-28T13:32:33.235178+0000","last_undegraded":"2024-04-28T13:32:33.235178+0000","last_fullsized":"2024-04-28T13:32:33.235178+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:12:10.305333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.561108+0000","last_change":"2024-04-28T13:32:32.235301+0000","last_active":"2024-04-28T13:32:33.561108+0000","last_peered":"2024-04-28T13:32:33.561108+0000","last_clean":"2024-04-28T13:32:33.561108+0000","last_became_active":"2024-04-28T13:32:32.235033+0000","last_became_peered":"2024-04-28T13:32:32.235033+0000","last_unstale":"2024-04-28T13:32:33.561108+0000","last_undegraded":"2024-04-28T13:32:33.561108+0000","last_fullsized":"2024-04-28T13:32:33.561108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:32:14.322085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563862+0000","last_change":"2024-04-28T13:32:31.233004+0000","last_active":"2024-04-28T13:32:33.563862+0000","last_peered":"2024-04-28T13:32:33.563862+0000","last_clean":"2024-04-28T13:32:33.563862+0000","last_became_active":"2024-04-28T13:32:31.232714+0000","last_became_peered":"2024-04-28T13:32:31.232714+0000","last_unstale":"2024-04-28T13:32:33.563862+0000","last_undegraded":"2024-04-28T13:32:33.563862+0000","last_fullsized":"2024-04-28T13:32:33.563862+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:58.653633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.561280+0000","last_change":"2024-04-28T13:32:32.231264+0000","last_active":"2024-04-28T13:32:33.561280+0000","last_peered":"2024-04-28T13:32:33.561280+0000","last_clean":"2024-04-28T13:32:33.561280+0000","last_became_active":"2024-04-28T13:32:32.230470+0000","last_became_peered":"2024-04-28T13:32:32.230470+0000","last_unstale":"2024-04-28T13:32:33.561280+0000","last_undegraded":"2024-04-28T13:32:33.561280+0000","last_fullsized":"2024-04-28T13:32:33.561280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:24:07.577184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235903+0000","last_change":"2024-04-28T13:32:31.215630+0000","last_active":"2024-04-28T13:32:33.235903+0000","last_peered":"2024-04-28T13:32:33.235903+0000","last_clean":"2024-04-28T13:32:33.235903+0000","last_became_active":"2024-04-28T13:32:31.215172+0000","last_became_peered":"2024-04-28T13:32:31.215172+0000","last_unstale":"2024-04-28T13:32:33.235903+0000","last_undegraded":"2024-04-28T13:32:33.235903+0000","last_fullsized":"2024-04-28T13:32:33.235903+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:30:48.400845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.565267+0000","last_change":"2024-04-28T13:32:32.253268+0000","last_active":"2024-04-28T13:32:33.565267+0000","last_peered":"2024-04-28T13:32:33.565267+0000","last_clean":"2024-04-28T13:32:33.565267+0000","last_became_active":"2024-04-28T13:32:32.251933+0000","last_became_peered":"2024-04-28T13:32:32.251933+0000","last_unstale":"2024-04-28T13:32:33.565267+0000","last_undegraded":"2024-04-28T13:32:33.565267+0000","last_fullsized":"2024-04-28T13:32:33.565267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:22:58.887460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.233962+0000","last_change":"2024-04-28T13:32:31.225740+0000","last_active":"2024-04-28T13:32:33.233962+0000","last_peered":"2024-04-28T13:32:33.233962+0000","last_clean":"2024-04-28T13:32:33.233962+0000","last_became_active":"2024-04-28T13:32:31.225458+0000","last_became_peered":"2024-04-28T13:32:31.225458+0000","last_unstale":"2024-04-28T13:32:33.233962+0000","last_undegraded":"2024-04-28T13:32:33.233962+0000","last_fullsized":"2024-04-28T13:32:33.233962+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:33:53.691067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.561244+0000","last_change":"2024-04-28T13:32:32.242450+0000","last_active":"2024-04-28T13:32:33.561244+0000","last_peered":"2024-04-28T13:32:33.561244+0000","last_clean":"2024-04-28T13:32:33.561244+0000","last_became_active":"2024-04-28T13:32:32.242148+0000","last_became_peered":"2024-04-28T13:32:32.242148+0000","last_unstale":"2024-04-28T13:32:33.561244+0000","last_undegraded":"2024-04-28T13:32:33.561244+0000","last_fullsized":"2024-04-28T13:32:33.561244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:59.668849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.247125+0000","last_change":"2024-04-28T13:32:31.227326+0000","last_active":"2024-04-28T13:32:33.247125+0000","last_peered":"2024-04-28T13:32:33.247125+0000","last_clean":"2024-04-28T13:32:33.247125+0000","last_became_active":"2024-04-28T13:32:31.226143+0000","last_became_peered":"2024-04-28T13:32:31.226143+0000","last_unstale":"2024-04-28T13:32:33.247125+0000","last_undegraded":"2024-04-28T13:32:33.247125+0000","last_fullsized":"2024-04-28T13:32:33.247125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:30:34.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235619+0000","last_change":"2024-04-28T13:32:32.231308+0000","last_active":"2024-04-28T13:32:33.235619+0000","last_peered":"2024-04-28T13:32:33.235619+0000","last_clean":"2024-04-28T13:32:33.235619+0000","last_became_active":"2024-04-28T13:32:32.230914+0000","last_became_peered":"2024-04-28T13:32:32.230914+0000","last_unstale":"2024-04-28T13:32:33.235619+0000","last_undegraded":"2024-04-28T13:32:33.235619+0000","last_fullsized":"2024-04-28T13:32:33.235619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:59:15.390148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235269+0000","last_change":"2024-04-28T13:32:31.227035+0000","last_active":"2024-04-28T13:32:33.235269+0000","last_peered":"2024-04-28T13:32:33.235269+0000","last_clean":"2024-04-28T13:32:33.235269+0000","last_became_active":"2024-04-28T13:32:31.226312+0000","last_became_peered":"2024-04-28T13:32:31.226312+0000","last_unstale":"2024-04-28T13:32:33.235269+0000","last_undegraded":"2024-04-28T13:32:33.235269+0000","last_fullsized":"2024-04-28T13:32:33.235269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:30:34.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234727+0000","last_change":"2024-04-28T13:32:32.233696+0000","last_active":"2024-04-28T13:32:33.234727+0000","last_peered":"2024-04-28T13:32:33.234727+0000","last_clean":"2024-04-28T13:32:33.234727+0000","last_became_active":"2024-04-28T13:32:32.233372+0000","last_became_peered":"2024-04-28T13:32:32.233372+0000","last_unstale":"2024-04-28T13:32:33.234727+0000","last_undegraded":"2024-04-28T13:32:33.234727+0000","last_fullsized":"2024-04-28T13:32:33.234727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:41:57.017489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.246994+0000","last_change":"2024-04-28T13:32:31.226997+0000","last_active":"2024-04-28T13:32:33.246994+0000","last_peered":"2024-04-28T13:32:33.246994+0000","last_clean":"2024-04-28T13:32:33.246994+0000","last_became_active":"2024-04-28T13:32:31.226705+0000","last_became_peered":"2024-04-28T13:32:31.226705+0000","last_unstale":"2024-04-28T13:32:33.246994+0000","last_undegraded":"2024-04-28T13:32:33.246994+0000","last_fullsized":"2024-04-28T13:32:33.246994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:12:10.305333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564089+0000","last_change":"2024-04-28T13:32:32.247886+0000","last_active":"2024-04-28T13:32:33.564089+0000","last_peered":"2024-04-28T13:32:33.564089+0000","last_clean":"2024-04-28T13:32:33.564089+0000","last_became_active":"2024-04-28T13:32:32.246316+0000","last_became_peered":"2024-04-28T13:32:32.246316+0000","last_unstale":"2024-04-28T13:32:33.564089+0000","last_undegraded":"2024-04-28T13:32:33.564089+0000","last_fullsized":"2024-04-28T13:32:33.564089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:20:52.770534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235483+0000","last_change":"2024-04-28T13:32:31.216200+0000","last_active":"2024-04-28T13:32:33.235483+0000","last_peered":"2024-04-28T13:32:33.235483+0000","last_clean":"2024-04-28T13:32:33.235483+0000","last_became_active":"2024-04-28T13:32:31.215800+0000","last_became_peered":"2024-04-28T13:32:31.215800+0000","last_unstale":"2024-04-28T13:32:33.235483+0000","last_undegraded":"2024-04-28T13:32:33.235483+0000","last_fullsized":"2024-04-28T13:32:33.235483+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:58.653633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235371+0000","last_change":"2024-04-28T13:32:32.241745+0000","last_active":"2024-04-28T13:32:33.235371+0000","last_peered":"2024-04-28T13:32:33.235371+0000","last_clean":"2024-04-28T13:32:33.235371+0000","last_became_active":"2024-04-28T13:32:32.241438+0000","last_became_peered":"2024-04-28T13:32:32.241438+0000","last_unstale":"2024-04-28T13:32:33.235371+0000","last_undegraded":"2024-04-28T13:32:33.235371+0000","last_fullsized":"2024-04-28T13:32:33.235371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:01:56.621985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.246042+0000","last_change":"2024-04-28T13:32:31.220706+0000","last_active":"2024-04-28T13:32:33.246042+0000","last_peered":"2024-04-28T13:32:33.246042+0000","last_clean":"2024-04-28T13:32:33.246042+0000","last_became_active":"2024-04-28T13:32:31.220451+0000","last_became_peered":"2024-04-28T13:32:31.220451+0000","last_unstale":"2024-04-28T13:32:33.246042+0000","last_undegraded":"2024-04-28T13:32:33.246042+0000","last_fullsized":"2024-04-28T13:32:33.246042+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:20:26.159036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563517+0000","last_change":"2024-04-28T13:32:32.231453+0000","last_active":"2024-04-28T13:32:33.563517+0000","last_peered":"2024-04-28T13:32:33.563517+0000","last_clean":"2024-04-28T13:32:33.563517+0000","last_became_active":"2024-04-28T13:32:32.231093+0000","last_became_peered":"2024-04-28T13:32:32.231093+0000","last_unstale":"2024-04-28T13:32:33.563517+0000","last_undegraded":"2024-04-28T13:32:33.563517+0000","last_fullsized":"2024-04-28T13:32:33.563517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:22:03.865574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236749+0000","last_change":"2024-04-28T13:32:31.225974+0000","last_active":"2024-04-28T13:32:33.236749+0000","last_peered":"2024-04-28T13:32:33.236749+0000","last_clean":"2024-04-28T13:32:33.236749+0000","last_became_active":"2024-04-28T13:32:31.219380+0000","last_became_peered":"2024-04-28T13:32:31.219380+0000","last_unstale":"2024-04-28T13:32:33.236749+0000","last_undegraded":"2024-04-28T13:32:33.236749+0000","last_fullsized":"2024-04-28T13:32:33.236749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:11:01.834169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236753+0000","last_change":"2024-04-28T13:32:32.234417+0000","last_active":"2024-04-28T13:32:33.236753+0000","last_peered":"2024-04-28T13:32:33.236753+0000","last_clean":"2024-04-28T13:32:33.236753+0000","last_became_active":"2024-04-28T13:32:32.233680+0000","last_became_peered":"2024-04-28T13:32:32.233680+0000","last_unstale":"2024-04-28T13:32:33.236753+0000","last_undegraded":"2024-04-28T13:32:33.236753+0000","last_fullsized":"2024-04-28T13:32:33.236753+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:34:54.602566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563203+0000","last_change":"2024-04-28T13:32:31.228304+0000","last_active":"2024-04-28T13:32:33.563203+0000","last_peered":"2024-04-28T13:32:33.563203+0000","last_clean":"2024-04-28T13:32:33.563203+0000","last_became_active":"2024-04-28T13:32:31.227982+0000","last_became_peered":"2024-04-28T13:32:31.227982+0000","last_unstale":"2024-04-28T13:32:33.563203+0000","last_undegraded":"2024-04-28T13:32:33.563203+0000","last_fullsized":"2024-04-28T13:32:33.563203+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:23:30.639925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.566285+0000","last_change":"2024-04-28T13:32:32.242809+0000","last_active":"2024-04-28T13:32:33.566285+0000","last_peered":"2024-04-28T13:32:33.566285+0000","last_clean":"2024-04-28T13:32:33.566285+0000","last_became_active":"2024-04-28T13:32:32.242516+0000","last_became_peered":"2024-04-28T13:32:32.242516+0000","last_unstale":"2024-04-28T13:32:33.566285+0000","last_undegraded":"2024-04-28T13:32:33.566285+0000","last_fullsized":"2024-04-28T13:32:33.566285+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:02:39.028860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564036+0000","last_change":"2024-04-28T13:32:31.228458+0000","last_active":"2024-04-28T13:32:33.564036+0000","last_peered":"2024-04-28T13:32:33.564036+0000","last_clean":"2024-04-28T13:32:33.564036+0000","last_became_active":"2024-04-28T13:32:31.228166+0000","last_became_peered":"2024-04-28T13:32:31.228166+0000","last_unstale":"2024-04-28T13:32:33.564036+0000","last_undegraded":"2024-04-28T13:32:33.564036+0000","last_fullsized":"2024-04-28T13:32:33.564036+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:13:46.879685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564733+0000","last_change":"2024-04-28T13:31:11.831538+0000","last_active":"2024-04-28T13:32:33.564733+0000","last_peered":"2024-04-28T13:32:33.564733+0000","last_clean":"2024-04-28T13:32:33.564733+0000","last_became_active":"2024-04-28T13:31:11.828959+0000","last_became_peered":"2024-04-28T13:31:11.828959+0000","last_unstale":"2024-04-28T13:32:33.564733+0000","last_undegraded":"2024-04-28T13:32:33.564733+0000","last_fullsized":"2024-04-28T13:32:33.564733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_clean_scrub_stamp":"2024-04-28T13:31:10.444388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:25:27.078911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":737082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":736992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T13:32:36.658 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T13:32:36.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.900+0000 7f9695600640 1 Processor -- start 2024-04-28T13:32:36.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.900+0000 7f9695600640 1 -- start start 2024-04-28T13:32:36.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.902+0000 7f9695600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9690163bf0 0x7f9690163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:36.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.902+0000 7f9695600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9690163570 0x7f96900b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:36.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.902+0000 7f9695600640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9690162da0 con 0x7f96901629c0 2024-04-28T13:32:36.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.902+0000 7f9695600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f96900d3e20 con 0x7f9690163570 2024-04-28T13:32:36.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.902+0000 7f9695600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f96900b6190 con 0x7f9690163bf0 2024-04-28T13:32:36.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.903+0000 7f9694c00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9690163570 0x7f96900b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:36.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.903+0000 7f968f400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9690163bf0 0x7f9690163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:36.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.903+0000 7f9694c00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9690163570 0x7f96900b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:56176/0 (socket says 172.21.15.196:56176) 2024-04-28T13:32:36.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.903+0000 7f9694c00640 1 -- 172.21.15.196:0/1075278317 learned_addr learned my addr 172.21.15.196:0/1075278317 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:36.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.904+0000 7f968f400640 1 -- 172.21.15.196:0/1075278317 >> v2:172.21.15.203:3300/0 conn(0x7f9690163570 msgr2=0x7f96900b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:36.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.904+0000 7f968f400640 1 --2- 172.21.15.196:0/1075278317 >> v2:172.21.15.203:3300/0 conn(0x7f9690163570 0x7f96900b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.904+0000 7f968f400640 1 -- 172.21.15.196:0/1075278317 >> v1:172.21.15.203:6789/0 conn(0x7f96901629c0 legacy=0x7f96900d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:36.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.904+0000 7f968f400640 1 -- 172.21.15.196:0/1075278317 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96900b6470 con 0x7f9690163bf0 2024-04-28T13:32:36.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.904+0000 7f9694c00640 1 --2- 172.21.15.196:0/1075278317 >> v2:172.21.15.203:3300/0 conn(0x7f9690163570 0x7f96900b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:32:36.905 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.905+0000 7f968f400640 1 --2- 172.21.15.196:0/1075278317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9690163bf0 0x7f9690163030 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9678009d90 tx=0x7f967802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2459b214680e5e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:36.905 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.905+0000 7f968ea00640 1 -- 172.21.15.196:0/1075278317 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678007740 con 0x7f9690163bf0 2024-04-28T13:32:36.905 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.906+0000 7f968ea00640 1 -- 172.21.15.196:0/1075278317 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96780078e0 con 0x7f9690163bf0 2024-04-28T13:32:36.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.906+0000 7f968ea00640 1 -- 172.21.15.196:0/1075278317 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678007cb0 con 0x7f9690163bf0 2024-04-28T13:32:36.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.906+0000 7f9695600640 1 -- 172.21.15.196:0/1075278317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9690163bf0 msgr2=0x7f9690163030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:36.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.906+0000 7f9695600640 1 --2- 172.21.15.196:0/1075278317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9690163bf0 0x7f9690163030 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9678009d90 tx=0x7f967802a660 comp rx=0 tx=0).stop 2024-04-28T13:32:36.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.906+0000 7f9695600640 1 -- 172.21.15.196:0/1075278317 shutdown_connections 2024-04-28T13:32:36.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.906+0000 7f9695600640 1 --2- 172.21.15.196:0/1075278317 >> v2:172.21.15.203:3300/0 conn(0x7f9690163570 0x7f96900b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.906+0000 7f9695600640 1 --2- 172.21.15.196:0/1075278317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9690163bf0 0x7f9690163030 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.906+0000 7f9695600640 1 -- 172.21.15.196:0/1075278317 >> 172.21.15.196:0/1075278317 conn(0x7f969009e570 msgr2=0x7f96900d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:36.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.906+0000 7f9695600640 1 -- 172.21.15.196:0/1075278317 shutdown_connections 2024-04-28T13:32:36.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.906+0000 7f9695600640 1 -- 172.21.15.196:0/1075278317 wait complete. 2024-04-28T13:32:36.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.908+0000 7f9695600640 1 Processor -- start 2024-04-28T13:32:36.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.908+0000 7f9695600640 1 -- start start 2024-04-28T13:32:36.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.909+0000 7f9695600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96901629c0 0x7f96900caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:36.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.909+0000 7f9695600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9690163570 0x7f96900ba850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:36.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.909+0000 7f9695600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9690163bf0 0x7f96900ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:36.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.909+0000 7f9695600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f96900b1e60 con 0x7f96901629c0 2024-04-28T13:32:36.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.909+0000 7f9695600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96900b6a30 con 0x7f9690163bf0 2024-04-28T13:32:36.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.909+0000 7f9695600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f96900b28c0 con 0x7f9690163570 2024-04-28T13:32:36.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.910+0000 7f968fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96901629c0 0x7f96900caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:36.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.910+0000 7f968fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96901629c0 0x7f96900caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34318/0 (socket says 172.21.15.196:34318) 2024-04-28T13:32:36.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.910+0000 7f968fe00640 1 -- 172.21.15.196:0/1261954948 learned_addr learned my addr 172.21.15.196:0/1261954948 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:36.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.910+0000 7f9694c00640 1 --2- 172.21.15.196:0/1261954948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9690163bf0 0x7f96900ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:36.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.910+0000 7f968f400640 1 --2- 172.21.15.196:0/1261954948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9690163570 0x7f96900ba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:36.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.910+0000 7f968fe00640 1 -- 172.21.15.196:0/1261954948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9690163570 msgr2=0x7f96900ba850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:36.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.910+0000 7f968fe00640 1 --2- 172.21.15.196:0/1261954948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9690163570 0x7f96900ba850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.910+0000 7f968fe00640 1 -- 172.21.15.196:0/1261954948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9690163bf0 msgr2=0x7f96900ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:36.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.910+0000 7f968fe00640 1 --2- 172.21.15.196:0/1261954948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9690163bf0 0x7f96900ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:36.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.910+0000 7f968fe00640 1 -- 172.21.15.196:0/1261954948 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96900b9530 con 0x7f96901629c0 2024-04-28T13:32:36.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.910+0000 7f9694c00640 1 --2- 172.21.15.196:0/1261954948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9690163bf0 0x7f96900ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:32:36.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.911+0000 7f968fe00640 1 --2- 172.21.15.196:0/1261954948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96901629c0 0x7f96900caeb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f968000c400 tx=0x7f968000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:36.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.911+0000 7f9687e00640 1 -- 172.21.15.196:0/1261954948 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96800052d0 con 0x7f96901629c0 2024-04-28T13:32:36.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.911+0000 7f9687e00640 1 -- 172.21.15.196:0/1261954948 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9680005490 con 0x7f96901629c0 2024-04-28T13:32:36.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.911+0000 7f9687e00640 1 -- 172.21.15.196:0/1261954948 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9680007260 con 0x7f96901629c0 2024-04-28T13:32:36.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.911+0000 7f9695600640 1 -- 172.21.15.196:0/1261954948 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96900b97c0 con 0x7f96901629c0 2024-04-28T13:32:36.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.911+0000 7f9695600640 1 -- 172.21.15.196:0/1261954948 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96900b9d80 con 0x7f96901629c0 2024-04-28T13:32:36.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.912+0000 7f9695600640 1 -- 172.21.15.196:0/1261954948 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f969006a290 con 0x7f96901629c0 2024-04-28T13:32:36.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.913+0000 7f9687e00640 1 -- 172.21.15.196:0/1261954948 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9680007400 con 0x7f96901629c0 2024-04-28T13:32:36.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.914+0000 7f9687e00640 1 --2- 172.21.15.196:0/1261954948 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f966c06c8f0 0x7f966c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:36.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.914+0000 7f9687e00640 1 -- 172.21.15.196:0/1261954948 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f968008b180 con 0x7f96901629c0 2024-04-28T13:32:36.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.915+0000 7f968f400640 1 --2- 172.21.15.196:0/1261954948 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f966c06c8f0 0x7f966c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:36.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.916+0000 7f968f400640 1 --2- 172.21.15.196:0/1261954948 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f966c06c8f0 0x7f966c06edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9678034000 tx=0x7f9678033040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:36.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:36.918+0000 7f9687e00640 1 -- 172.21.15.196:0/1261954948 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9680059510 con 0x7f96901629c0 2024-04-28T13:32:37.115 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.115+0000 7f9695600640 1 -- 172.21.15.196:0/1261954948 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f969009ea50 con 0x7f966c06c8f0 2024-04-28T13:32:37.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.136+0000 7f9687e00640 1 -- 172.21.15.196:0/1261954948 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7f969009ea50 con 0x7f966c06c8f0 2024-04-28T13:32:37.139 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:37.140 INFO:teuthology.orchestra.run.smithi196.stderr:dumped all 2024-04-28T13:32:37.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.142+0000 7f9695600640 1 -- 172.21.15.196:0/1261954948 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f966c06c8f0 msgr2=0x7f966c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:37.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.142+0000 7f9695600640 1 --2- 172.21.15.196:0/1261954948 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f966c06c8f0 0x7f966c06edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9678034000 tx=0x7f9678033040 comp rx=0 tx=0).stop 2024-04-28T13:32:37.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.142+0000 7f9695600640 1 -- 172.21.15.196:0/1261954948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96901629c0 msgr2=0x7f96900caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:37.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.142+0000 7f9695600640 1 --2- 172.21.15.196:0/1261954948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96901629c0 0x7f96900caeb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f968000c400 tx=0x7f968000c8c0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.142+0000 7f9695600640 1 -- 172.21.15.196:0/1261954948 shutdown_connections 2024-04-28T13:32:37.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.142+0000 7f9695600640 1 --2- 172.21.15.196:0/1261954948 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f966c06c8f0 0x7f966c06edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.142+0000 7f9695600640 1 --2- 172.21.15.196:0/1261954948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9690163bf0 0x7f96900ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.142+0000 7f9695600640 1 --2- 172.21.15.196:0/1261954948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9690163570 0x7f96900ba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.142+0000 7f9695600640 1 --2- 172.21.15.196:0/1261954948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96901629c0 0x7f96900caeb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.142+0000 7f9695600640 1 -- 172.21.15.196:0/1261954948 >> 172.21.15.196:0/1261954948 conn(0x7f969009e570 msgr2=0x7f969006d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:37.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.142+0000 7f9695600640 1 -- 172.21.15.196:0/1261954948 shutdown_connections 2024-04-28T13:32:37.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.142+0000 7f9695600640 1 -- 172.21.15.196:0/1261954948 wait complete. 2024-04-28T13:32:37.161 INFO:teuthology.orchestra.run.smithi196.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-28T13:32:36.207449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220328,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754392,"statfs":{"total":773094113280,"available":772868497408,"internally_reserved":0,"allocated":4235264,"data_stored":2358548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002093"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.565277+0000","last_change":"2024-04-28T13:32:32.250841+0000","last_active":"2024-04-28T13:32:33.565277+0000","last_peered":"2024-04-28T13:32:33.565277+0000","last_clean":"2024-04-28T13:32:33.565277+0000","last_became_active":"2024-04-28T13:32:32.250548+0000","last_became_peered":"2024-04-28T13:32:32.250548+0000","last_unstale":"2024-04-28T13:32:33.565277+0000","last_undegraded":"2024-04-28T13:32:33.565277+0000","last_fullsized":"2024-04-28T13:32:33.565277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:00.472207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564075+0000","last_change":"2024-04-28T13:32:32.237896+0000","last_active":"2024-04-28T13:32:33.564075+0000","last_peered":"2024-04-28T13:32:33.564075+0000","last_clean":"2024-04-28T13:32:33.564075+0000","last_became_active":"2024-04-28T13:32:32.234973+0000","last_became_peered":"2024-04-28T13:32:32.234973+0000","last_unstale":"2024-04-28T13:32:33.564075+0000","last_undegraded":"2024-04-28T13:32:33.564075+0000","last_fullsized":"2024-04-28T13:32:33.564075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:23:33.274183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234976+0000","last_change":"2024-04-28T13:32:32.236328+0000","last_active":"2024-04-28T13:32:33.234976+0000","last_peered":"2024-04-28T13:32:33.234976+0000","last_clean":"2024-04-28T13:32:33.234976+0000","last_became_active":"2024-04-28T13:32:32.231979+0000","last_became_peered":"2024-04-28T13:32:32.231979+0000","last_unstale":"2024-04-28T13:32:33.234976+0000","last_undegraded":"2024-04-28T13:32:33.234976+0000","last_fullsized":"2024-04-28T13:32:33.234976+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:00:57.153237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.246658+0000","last_change":"2024-04-28T13:32:32.238192+0000","last_active":"2024-04-28T13:32:33.246658+0000","last_peered":"2024-04-28T13:32:33.246658+0000","last_clean":"2024-04-28T13:32:33.246658+0000","last_became_active":"2024-04-28T13:32:32.236991+0000","last_became_peered":"2024-04-28T13:32:32.236991+0000","last_unstale":"2024-04-28T13:32:33.246658+0000","last_undegraded":"2024-04-28T13:32:33.246658+0000","last_fullsized":"2024-04-28T13:32:33.246658+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:32:57.197848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.560941+0000","last_change":"2024-04-28T13:32:32.244713+0000","last_active":"2024-04-28T13:32:33.560941+0000","last_peered":"2024-04-28T13:32:33.560941+0000","last_clean":"2024-04-28T13:32:33.560941+0000","last_became_active":"2024-04-28T13:32:32.237452+0000","last_became_peered":"2024-04-28T13:32:32.237452+0000","last_unstale":"2024-04-28T13:32:33.560941+0000","last_undegraded":"2024-04-28T13:32:33.560941+0000","last_fullsized":"2024-04-28T13:32:33.560941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:04:08.554194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563729+0000","last_change":"2024-04-28T13:32:32.227097+0000","last_active":"2024-04-28T13:32:33.563729+0000","last_peered":"2024-04-28T13:32:33.563729+0000","last_clean":"2024-04-28T13:32:33.563729+0000","last_became_active":"2024-04-28T13:32:32.225481+0000","last_became_peered":"2024-04-28T13:32:32.225481+0000","last_unstale":"2024-04-28T13:32:33.563729+0000","last_undegraded":"2024-04-28T13:32:33.563729+0000","last_fullsized":"2024-04-28T13:32:33.563729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:00:57.153237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.565672+0000","last_change":"2024-04-28T13:32:32.252961+0000","last_active":"2024-04-28T13:32:33.565672+0000","last_peered":"2024-04-28T13:32:33.565672+0000","last_clean":"2024-04-28T13:32:33.565672+0000","last_became_active":"2024-04-28T13:32:32.252325+0000","last_became_peered":"2024-04-28T13:32:32.252325+0000","last_unstale":"2024-04-28T13:32:33.565672+0000","last_undegraded":"2024-04-28T13:32:33.565672+0000","last_fullsized":"2024-04-28T13:32:33.565672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:23:16.417118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563655+0000","last_change":"2024-04-28T13:32:32.245757+0000","last_active":"2024-04-28T13:32:33.563655+0000","last_peered":"2024-04-28T13:32:33.563655+0000","last_clean":"2024-04-28T13:32:33.563655+0000","last_became_active":"2024-04-28T13:32:32.243796+0000","last_became_peered":"2024-04-28T13:32:32.243796+0000","last_unstale":"2024-04-28T13:32:33.563655+0000","last_undegraded":"2024-04-28T13:32:33.563655+0000","last_fullsized":"2024-04-28T13:32:33.563655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:22:03.865574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.237852+0000","last_change":"2024-04-28T13:32:32.235173+0000","last_active":"2024-04-28T13:32:33.237852+0000","last_peered":"2024-04-28T13:32:33.237852+0000","last_clean":"2024-04-28T13:32:33.237852+0000","last_became_active":"2024-04-28T13:32:32.231895+0000","last_became_peered":"2024-04-28T13:32:32.231895+0000","last_unstale":"2024-04-28T13:32:33.237852+0000","last_undegraded":"2024-04-28T13:32:33.237852+0000","last_fullsized":"2024-04-28T13:32:33.237852+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:45.905694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.247025+0000","last_change":"2024-04-28T13:32:32.234270+0000","last_active":"2024-04-28T13:32:33.247025+0000","last_peered":"2024-04-28T13:32:33.247025+0000","last_clean":"2024-04-28T13:32:33.247025+0000","last_became_active":"2024-04-28T13:32:32.233820+0000","last_became_peered":"2024-04-28T13:32:32.233820+0000","last_unstale":"2024-04-28T13:32:33.247025+0000","last_undegraded":"2024-04-28T13:32:33.247025+0000","last_fullsized":"2024-04-28T13:32:33.247025+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:45.905694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564931+0000","last_change":"2024-04-28T13:32:32.247706+0000","last_active":"2024-04-28T13:32:33.564931+0000","last_peered":"2024-04-28T13:32:33.564931+0000","last_clean":"2024-04-28T13:32:33.564931+0000","last_became_active":"2024-04-28T13:32:32.247153+0000","last_became_peered":"2024-04-28T13:32:32.247153+0000","last_unstale":"2024-04-28T13:32:33.564931+0000","last_undegraded":"2024-04-28T13:32:33.564931+0000","last_fullsized":"2024-04-28T13:32:33.564931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:23:33.274183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234962+0000","last_change":"2024-04-28T13:32:32.227012+0000","last_active":"2024-04-28T13:32:33.234962+0000","last_peered":"2024-04-28T13:32:33.234962+0000","last_clean":"2024-04-28T13:32:33.234962+0000","last_became_active":"2024-04-28T13:32:32.225919+0000","last_became_peered":"2024-04-28T13:32:32.225919+0000","last_unstale":"2024-04-28T13:32:33.234962+0000","last_undegraded":"2024-04-28T13:32:33.234962+0000","last_fullsized":"2024-04-28T13:32:33.234962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:05:13.660602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.561245+0000","last_change":"2024-04-28T13:32:32.231918+0000","last_active":"2024-04-28T13:32:33.561245+0000","last_peered":"2024-04-28T13:32:33.561245+0000","last_clean":"2024-04-28T13:32:33.561245+0000","last_became_active":"2024-04-28T13:32:32.231492+0000","last_became_peered":"2024-04-28T13:32:32.231492+0000","last_unstale":"2024-04-28T13:32:33.561245+0000","last_undegraded":"2024-04-28T13:32:33.561245+0000","last_fullsized":"2024-04-28T13:32:33.561245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:20:27.174252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563947+0000","last_change":"2024-04-28T13:32:32.226886+0000","last_active":"2024-04-28T13:32:33.563947+0000","last_peered":"2024-04-28T13:32:33.563947+0000","last_clean":"2024-04-28T13:32:33.563947+0000","last_became_active":"2024-04-28T13:32:32.226468+0000","last_became_peered":"2024-04-28T13:32:32.226468+0000","last_unstale":"2024-04-28T13:32:33.563947+0000","last_undegraded":"2024-04-28T13:32:33.563947+0000","last_fullsized":"2024-04-28T13:32:33.563947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:51:21.512396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234239+0000","last_change":"2024-04-28T13:32:32.243953+0000","last_active":"2024-04-28T13:32:33.234239+0000","last_peered":"2024-04-28T13:32:33.234239+0000","last_clean":"2024-04-28T13:32:33.234239+0000","last_became_active":"2024-04-28T13:32:32.243404+0000","last_became_peered":"2024-04-28T13:32:32.243404+0000","last_unstale":"2024-04-28T13:32:33.234239+0000","last_undegraded":"2024-04-28T13:32:33.234239+0000","last_fullsized":"2024-04-28T13:32:33.234239+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:27:45.905694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564577+0000","last_change":"2024-04-28T13:32:32.246203+0000","last_active":"2024-04-28T13:32:33.564577+0000","last_peered":"2024-04-28T13:32:33.564577+0000","last_clean":"2024-04-28T13:32:33.564577+0000","last_became_active":"2024-04-28T13:32:32.245488+0000","last_became_peered":"2024-04-28T13:32:32.245488+0000","last_unstale":"2024-04-28T13:32:33.564577+0000","last_undegraded":"2024-04-28T13:32:33.564577+0000","last_fullsized":"2024-04-28T13:32:33.564577+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:45:01.603947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.565532+0000","last_change":"2024-04-28T13:32:32.250595+0000","last_active":"2024-04-28T13:32:33.565532+0000","last_peered":"2024-04-28T13:32:33.565532+0000","last_clean":"2024-04-28T13:32:33.565532+0000","last_became_active":"2024-04-28T13:32:32.248441+0000","last_became_peered":"2024-04-28T13:32:32.248441+0000","last_unstale":"2024-04-28T13:32:33.565532+0000","last_undegraded":"2024-04-28T13:32:33.565532+0000","last_fullsized":"2024-04-28T13:32:33.565532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:32.105900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564255+0000","last_change":"2024-04-28T13:32:32.238288+0000","last_active":"2024-04-28T13:32:33.564255+0000","last_peered":"2024-04-28T13:32:33.564255+0000","last_clean":"2024-04-28T13:32:33.564255+0000","last_became_active":"2024-04-28T13:32:32.232177+0000","last_became_peered":"2024-04-28T13:32:32.232177+0000","last_unstale":"2024-04-28T13:32:33.564255+0000","last_undegraded":"2024-04-28T13:32:33.564255+0000","last_fullsized":"2024-04-28T13:32:33.564255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:32.105900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563960+0000","last_change":"2024-04-28T13:32:32.234087+0000","last_active":"2024-04-28T13:32:33.563960+0000","last_peered":"2024-04-28T13:32:33.563960+0000","last_clean":"2024-04-28T13:32:33.563960+0000","last_became_active":"2024-04-28T13:32:32.233753+0000","last_became_peered":"2024-04-28T13:32:32.233753+0000","last_unstale":"2024-04-28T13:32:33.563960+0000","last_undegraded":"2024-04-28T13:32:33.563960+0000","last_fullsized":"2024-04-28T13:32:33.563960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:59:15.390148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.560933+0000","last_change":"2024-04-28T13:32:32.240752+0000","last_active":"2024-04-28T13:32:33.560933+0000","last_peered":"2024-04-28T13:32:33.560933+0000","last_clean":"2024-04-28T13:32:33.560933+0000","last_became_active":"2024-04-28T13:32:32.240293+0000","last_became_peered":"2024-04-28T13:32:32.240293+0000","last_unstale":"2024-04-28T13:32:33.560933+0000","last_undegraded":"2024-04-28T13:32:33.560933+0000","last_fullsized":"2024-04-28T13:32:33.560933+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:13:47.894901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236143+0000","last_change":"2024-04-28T13:32:32.236745+0000","last_active":"2024-04-28T13:32:33.236143+0000","last_peered":"2024-04-28T13:32:33.236143+0000","last_clean":"2024-04-28T13:32:33.236143+0000","last_became_active":"2024-04-28T13:32:32.236460+0000","last_became_peered":"2024-04-28T13:32:32.236460+0000","last_unstale":"2024-04-28T13:32:33.236143+0000","last_undegraded":"2024-04-28T13:32:33.236143+0000","last_fullsized":"2024-04-28T13:32:33.236143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:03:42.106373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.237089+0000","last_change":"2024-04-28T13:32:32.235265+0000","last_active":"2024-04-28T13:32:33.237089+0000","last_peered":"2024-04-28T13:32:33.237089+0000","last_clean":"2024-04-28T13:32:33.237089+0000","last_became_active":"2024-04-28T13:32:32.234950+0000","last_became_peered":"2024-04-28T13:32:32.234950+0000","last_unstale":"2024-04-28T13:32:33.237089+0000","last_undegraded":"2024-04-28T13:32:33.237089+0000","last_fullsized":"2024-04-28T13:32:33.237089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:44:10.557521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234536+0000","last_change":"2024-04-28T13:32:32.250398+0000","last_active":"2024-04-28T13:32:33.234536+0000","last_peered":"2024-04-28T13:32:33.234536+0000","last_clean":"2024-04-28T13:32:33.234536+0000","last_became_active":"2024-04-28T13:32:32.249507+0000","last_became_peered":"2024-04-28T13:32:32.249507+0000","last_unstale":"2024-04-28T13:32:33.234536+0000","last_undegraded":"2024-04-28T13:32:33.234536+0000","last_fullsized":"2024-04-28T13:32:33.234536+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:29:00.472207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234127+0000","last_change":"2024-04-28T13:32:32.228861+0000","last_active":"2024-04-28T13:32:33.234127+0000","last_peered":"2024-04-28T13:32:33.234127+0000","last_clean":"2024-04-28T13:32:33.234127+0000","last_became_active":"2024-04-28T13:32:32.227846+0000","last_became_peered":"2024-04-28T13:32:32.227846+0000","last_unstale":"2024-04-28T13:32:33.234127+0000","last_undegraded":"2024-04-28T13:32:33.234127+0000","last_fullsized":"2024-04-28T13:32:33.234127+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:44:10.557521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.247025+0000","last_change":"2024-04-28T13:32:32.248140+0000","last_active":"2024-04-28T13:32:33.247025+0000","last_peered":"2024-04-28T13:32:33.247025+0000","last_clean":"2024-04-28T13:32:33.247025+0000","last_became_active":"2024-04-28T13:32:32.247780+0000","last_became_peered":"2024-04-28T13:32:32.247780+0000","last_unstale":"2024-04-28T13:32:33.247025+0000","last_undegraded":"2024-04-28T13:32:33.247025+0000","last_fullsized":"2024-04-28T13:32:33.247025+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:44:10.557521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564345+0000","last_change":"2024-04-28T13:32:32.244003+0000","last_active":"2024-04-28T13:32:33.564345+0000","last_peered":"2024-04-28T13:32:33.564345+0000","last_clean":"2024-04-28T13:32:33.564345+0000","last_became_active":"2024-04-28T13:32:32.243684+0000","last_became_peered":"2024-04-28T13:32:32.243684+0000","last_unstale":"2024-04-28T13:32:33.564345+0000","last_undegraded":"2024-04-28T13:32:33.564345+0000","last_fullsized":"2024-04-28T13:32:33.564345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:23:16.417118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.246849+0000","last_change":"2024-04-28T13:32:32.242488+0000","last_active":"2024-04-28T13:32:33.246849+0000","last_peered":"2024-04-28T13:32:33.246849+0000","last_clean":"2024-04-28T13:32:33.246849+0000","last_became_active":"2024-04-28T13:32:32.241505+0000","last_became_peered":"2024-04-28T13:32:32.241505+0000","last_unstale":"2024-04-28T13:32:33.246849+0000","last_undegraded":"2024-04-28T13:32:33.246849+0000","last_fullsized":"2024-04-28T13:32:33.246849+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:47:42.691584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235369+0000","last_change":"2024-04-28T13:32:32.230977+0000","last_active":"2024-04-28T13:32:33.235369+0000","last_peered":"2024-04-28T13:32:33.235369+0000","last_clean":"2024-04-28T13:32:33.235369+0000","last_became_active":"2024-04-28T13:32:32.230695+0000","last_became_peered":"2024-04-28T13:32:32.230695+0000","last_unstale":"2024-04-28T13:32:33.235369+0000","last_undegraded":"2024-04-28T13:32:33.235369+0000","last_fullsized":"2024-04-28T13:32:33.235369+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:45:01.603947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236302+0000","last_change":"2024-04-28T13:32:32.234401+0000","last_active":"2024-04-28T13:32:33.236302+0000","last_peered":"2024-04-28T13:32:33.236302+0000","last_clean":"2024-04-28T13:32:33.236302+0000","last_became_active":"2024-04-28T13:32:32.234087+0000","last_became_peered":"2024-04-28T13:32:32.234087+0000","last_unstale":"2024-04-28T13:32:33.236302+0000","last_undegraded":"2024-04-28T13:32:33.236302+0000","last_fullsized":"2024-04-28T13:32:33.236302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:32:57.197848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564749+0000","last_change":"2024-04-28T13:32:32.249322+0000","last_active":"2024-04-28T13:32:33.564749+0000","last_peered":"2024-04-28T13:32:33.564749+0000","last_clean":"2024-04-28T13:32:33.564749+0000","last_became_active":"2024-04-28T13:32:32.249015+0000","last_became_peered":"2024-04-28T13:32:32.249015+0000","last_unstale":"2024-04-28T13:32:33.564749+0000","last_undegraded":"2024-04-28T13:32:33.564749+0000","last_fullsized":"2024-04-28T13:32:33.564749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:51:21.512396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563635+0000","last_change":"2024-04-28T13:32:32.238482+0000","last_active":"2024-04-28T13:32:33.563635+0000","last_peered":"2024-04-28T13:32:33.563635+0000","last_clean":"2024-04-28T13:32:33.563635+0000","last_became_active":"2024-04-28T13:32:32.238169+0000","last_became_peered":"2024-04-28T13:32:32.238169+0000","last_unstale":"2024-04-28T13:32:33.563635+0000","last_undegraded":"2024-04-28T13:32:33.563635+0000","last_fullsized":"2024-04-28T13:32:33.563635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:33:54.706283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563598+0000","last_change":"2024-04-28T13:32:32.251390+0000","last_active":"2024-04-28T13:32:33.563598+0000","last_peered":"2024-04-28T13:32:33.563598+0000","last_clean":"2024-04-28T13:32:33.563598+0000","last_became_active":"2024-04-28T13:32:32.251076+0000","last_became_peered":"2024-04-28T13:32:32.251076+0000","last_unstale":"2024-04-28T13:32:33.563598+0000","last_undegraded":"2024-04-28T13:32:33.563598+0000","last_fullsized":"2024-04-28T13:32:33.563598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:05:13.660602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235273+0000","last_change":"2024-04-28T13:32:32.252823+0000","last_active":"2024-04-28T13:32:33.235273+0000","last_peered":"2024-04-28T13:32:33.235273+0000","last_clean":"2024-04-28T13:32:33.235273+0000","last_became_active":"2024-04-28T13:32:32.252488+0000","last_became_peered":"2024-04-28T13:32:32.252488+0000","last_unstale":"2024-04-28T13:32:33.235273+0000","last_undegraded":"2024-04-28T13:32:33.235273+0000","last_fullsized":"2024-04-28T13:32:33.235273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:33:41.458231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.565421+0000","last_change":"2024-04-28T13:32:31.228570+0000","last_active":"2024-04-28T13:32:33.565421+0000","last_peered":"2024-04-28T13:32:33.565421+0000","last_clean":"2024-04-28T13:32:33.565421+0000","last_became_active":"2024-04-28T13:32:31.228251+0000","last_became_peered":"2024-04-28T13:32:31.228251+0000","last_unstale":"2024-04-28T13:32:33.565421+0000","last_undegraded":"2024-04-28T13:32:33.565421+0000","last_fullsized":"2024-04-28T13:32:33.565421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:32:13.306868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.565437+0000","last_change":"2024-04-28T13:32:32.243756+0000","last_active":"2024-04-28T13:32:33.565437+0000","last_peered":"2024-04-28T13:32:33.565437+0000","last_clean":"2024-04-28T13:32:33.565437+0000","last_became_active":"2024-04-28T13:32:32.243474+0000","last_became_peered":"2024-04-28T13:32:32.243474+0000","last_unstale":"2024-04-28T13:32:33.565437+0000","last_undegraded":"2024-04-28T13:32:33.565437+0000","last_fullsized":"2024-04-28T13:32:33.565437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:41:57.017489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235017+0000","last_change":"2024-04-28T13:32:31.229138+0000","last_active":"2024-04-28T13:32:33.235017+0000","last_peered":"2024-04-28T13:32:33.235017+0000","last_clean":"2024-04-28T13:32:33.235017+0000","last_became_active":"2024-04-28T13:32:31.228859+0000","last_became_peered":"2024-04-28T13:32:31.228859+0000","last_unstale":"2024-04-28T13:32:33.235017+0000","last_undegraded":"2024-04-28T13:32:33.235017+0000","last_fullsized":"2024-04-28T13:32:33.235017+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:20:26.159036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236652+0000","last_change":"2024-04-28T13:32:32.237721+0000","last_active":"2024-04-28T13:32:33.236652+0000","last_peered":"2024-04-28T13:32:33.236652+0000","last_clean":"2024-04-28T13:32:33.236652+0000","last_became_active":"2024-04-28T13:32:32.237247+0000","last_became_peered":"2024-04-28T13:32:32.237247+0000","last_unstale":"2024-04-28T13:32:33.236652+0000","last_undegraded":"2024-04-28T13:32:33.236652+0000","last_fullsized":"2024-04-28T13:32:33.236652+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:47:42.691584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563963+0000","last_change":"2024-04-28T13:32:31.234407+0000","last_active":"2024-04-28T13:32:33.563963+0000","last_peered":"2024-04-28T13:32:33.563963+0000","last_clean":"2024-04-28T13:32:33.563963+0000","last_became_active":"2024-04-28T13:32:31.234100+0000","last_became_peered":"2024-04-28T13:32:31.234100+0000","last_unstale":"2024-04-28T13:32:33.563963+0000","last_undegraded":"2024-04-28T13:32:33.563963+0000","last_fullsized":"2024-04-28T13:32:33.563963+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:28:59.456991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.246947+0000","last_change":"2024-04-28T13:32:32.228659+0000","last_active":"2024-04-28T13:32:33.246947+0000","last_peered":"2024-04-28T13:32:33.246947+0000","last_clean":"2024-04-28T13:32:33.246947+0000","last_became_active":"2024-04-28T13:32:32.228041+0000","last_became_peered":"2024-04-28T13:32:32.228041+0000","last_unstale":"2024-04-28T13:32:33.246947+0000","last_undegraded":"2024-04-28T13:32:33.246947+0000","last_fullsized":"2024-04-28T13:32:33.246947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:34:54.602566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.561029+0000","last_change":"2024-04-28T13:32:31.228812+0000","last_active":"2024-04-28T13:32:33.561029+0000","last_peered":"2024-04-28T13:32:33.561029+0000","last_clean":"2024-04-28T13:32:33.561029+0000","last_became_active":"2024-04-28T13:32:31.228397+0000","last_became_peered":"2024-04-28T13:32:31.228397+0000","last_unstale":"2024-04-28T13:32:33.561029+0000","last_undegraded":"2024-04-28T13:32:33.561029+0000","last_fullsized":"2024-04-28T13:32:33.561029+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:22:52.708165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564775+0000","last_change":"2024-04-28T13:32:32.244639+0000","last_active":"2024-04-28T13:32:33.564775+0000","last_peered":"2024-04-28T13:32:33.564775+0000","last_clean":"2024-04-28T13:32:33.564775+0000","last_became_active":"2024-04-28T13:32:32.244274+0000","last_became_peered":"2024-04-28T13:32:32.244274+0000","last_unstale":"2024-04-28T13:32:33.564775+0000","last_undegraded":"2024-04-28T13:32:33.564775+0000","last_fullsized":"2024-04-28T13:32:33.564775+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:59:15.390148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236213+0000","last_change":"2024-04-28T13:32:31.223055+0000","last_active":"2024-04-28T13:32:33.236213+0000","last_peered":"2024-04-28T13:32:33.236213+0000","last_clean":"2024-04-28T13:32:33.236213+0000","last_became_active":"2024-04-28T13:32:31.222713+0000","last_became_peered":"2024-04-28T13:32:31.222713+0000","last_unstale":"2024-04-28T13:32:33.236213+0000","last_undegraded":"2024-04-28T13:32:33.236213+0000","last_fullsized":"2024-04-28T13:32:33.236213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:32:13.306868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234789+0000","last_change":"2024-04-28T13:32:32.235548+0000","last_active":"2024-04-28T13:32:33.234789+0000","last_peered":"2024-04-28T13:32:33.234789+0000","last_clean":"2024-04-28T13:32:33.234789+0000","last_became_active":"2024-04-28T13:32:32.233102+0000","last_became_peered":"2024-04-28T13:32:32.233102+0000","last_unstale":"2024-04-28T13:32:33.234789+0000","last_undegraded":"2024-04-28T13:32:33.234789+0000","last_fullsized":"2024-04-28T13:32:33.234789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:14:48.471231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236137+0000","last_change":"2024-04-28T13:32:31.226888+0000","last_active":"2024-04-28T13:32:33.236137+0000","last_peered":"2024-04-28T13:32:33.236137+0000","last_clean":"2024-04-28T13:32:33.236137+0000","last_became_active":"2024-04-28T13:32:31.225183+0000","last_became_peered":"2024-04-28T13:32:31.225183+0000","last_unstale":"2024-04-28T13:32:33.236137+0000","last_undegraded":"2024-04-28T13:32:33.236137+0000","last_fullsized":"2024-04-28T13:32:33.236137+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:58.653633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235625+0000","last_change":"2024-04-28T13:32:32.243447+0000","last_active":"2024-04-28T13:32:33.235625+0000","last_peered":"2024-04-28T13:32:33.235625+0000","last_clean":"2024-04-28T13:32:33.235625+0000","last_became_active":"2024-04-28T13:32:32.243074+0000","last_became_peered":"2024-04-28T13:32:32.243074+0000","last_unstale":"2024-04-28T13:32:33.235625+0000","last_undegraded":"2024-04-28T13:32:33.235625+0000","last_fullsized":"2024-04-28T13:32:33.235625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:01:56.621985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563331+0000","last_change":"2024-04-28T13:32:31.223766+0000","last_active":"2024-04-28T13:32:33.563331+0000","last_peered":"2024-04-28T13:32:33.563331+0000","last_clean":"2024-04-28T13:32:33.563331+0000","last_became_active":"2024-04-28T13:32:31.223546+0000","last_became_peered":"2024-04-28T13:32:31.223546+0000","last_unstale":"2024-04-28T13:32:33.563331+0000","last_undegraded":"2024-04-28T13:32:33.563331+0000","last_fullsized":"2024-04-28T13:32:33.563331+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:33:53.691067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.245939+0000","last_change":"2024-04-28T13:32:32.242078+0000","last_active":"2024-04-28T13:32:33.245939+0000","last_peered":"2024-04-28T13:32:33.245939+0000","last_clean":"2024-04-28T13:32:33.245939+0000","last_became_active":"2024-04-28T13:32:32.241566+0000","last_became_peered":"2024-04-28T13:32:32.241566+0000","last_unstale":"2024-04-28T13:32:33.245939+0000","last_undegraded":"2024-04-28T13:32:33.245939+0000","last_fullsized":"2024-04-28T13:32:33.245939+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:03:42.106373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235617+0000","last_change":"2024-04-28T13:32:31.226887+0000","last_active":"2024-04-28T13:32:33.235617+0000","last_peered":"2024-04-28T13:32:33.235617+0000","last_clean":"2024-04-28T13:32:33.235617+0000","last_became_active":"2024-04-28T13:32:31.224680+0000","last_became_peered":"2024-04-28T13:32:31.224680+0000","last_unstale":"2024-04-28T13:32:33.235617+0000","last_undegraded":"2024-04-28T13:32:33.235617+0000","last_fullsized":"2024-04-28T13:32:33.235617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:04:07.538978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235508+0000","last_change":"2024-04-28T13:32:32.230272+0000","last_active":"2024-04-28T13:32:33.235508+0000","last_peered":"2024-04-28T13:32:33.235508+0000","last_clean":"2024-04-28T13:32:33.235508+0000","last_became_active":"2024-04-28T13:32:32.229747+0000","last_became_peered":"2024-04-28T13:32:32.229747+0000","last_unstale":"2024-04-28T13:32:33.235508+0000","last_undegraded":"2024-04-28T13:32:33.235508+0000","last_fullsized":"2024-04-28T13:32:33.235508+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:51:21.512396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564421+0000","last_change":"2024-04-28T13:32:31.232102+0000","last_active":"2024-04-28T13:32:33.564421+0000","last_peered":"2024-04-28T13:32:33.564421+0000","last_clean":"2024-04-28T13:32:33.564421+0000","last_became_active":"2024-04-28T13:32:31.231791+0000","last_became_peered":"2024-04-28T13:32:31.231791+0000","last_unstale":"2024-04-28T13:32:33.564421+0000","last_undegraded":"2024-04-28T13:32:33.564421+0000","last_fullsized":"2024-04-28T13:32:33.564421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:41:56.002273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564344+0000","last_change":"2024-04-28T13:32:32.233805+0000","last_active":"2024-04-28T13:32:33.564344+0000","last_peered":"2024-04-28T13:32:33.564344+0000","last_clean":"2024-04-28T13:32:33.564344+0000","last_became_active":"2024-04-28T13:32:32.233506+0000","last_became_peered":"2024-04-28T13:32:32.233506+0000","last_unstale":"2024-04-28T13:32:33.564344+0000","last_undegraded":"2024-04-28T13:32:33.564344+0000","last_fullsized":"2024-04-28T13:32:33.564344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:02:39.028860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235130+0000","last_change":"2024-04-28T13:32:31.224724+0000","last_active":"2024-04-28T13:32:33.235130+0000","last_peered":"2024-04-28T13:32:33.235130+0000","last_clean":"2024-04-28T13:32:33.235130+0000","last_became_active":"2024-04-28T13:32:31.222741+0000","last_became_peered":"2024-04-28T13:32:31.222741+0000","last_unstale":"2024-04-28T13:32:33.235130+0000","last_undegraded":"2024-04-28T13:32:33.235130+0000","last_fullsized":"2024-04-28T13:32:33.235130+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:24:06.561968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564263+0000","last_change":"2024-04-28T13:32:32.245208+0000","last_active":"2024-04-28T13:32:33.564263+0000","last_peered":"2024-04-28T13:32:33.564263+0000","last_clean":"2024-04-28T13:32:33.564263+0000","last_became_active":"2024-04-28T13:32:32.244813+0000","last_became_peered":"2024-04-28T13:32:32.244813+0000","last_unstale":"2024-04-28T13:32:33.564263+0000","last_undegraded":"2024-04-28T13:32:33.564263+0000","last_fullsized":"2024-04-28T13:32:33.564263+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:22:58.887460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235099+0000","last_change":"2024-04-28T13:32:31.229553+0000","last_active":"2024-04-28T13:32:33.235099+0000","last_peered":"2024-04-28T13:32:33.235099+0000","last_clean":"2024-04-28T13:32:33.235099+0000","last_became_active":"2024-04-28T13:32:31.229130+0000","last_became_peered":"2024-04-28T13:32:31.229130+0000","last_unstale":"2024-04-28T13:32:33.235099+0000","last_undegraded":"2024-04-28T13:32:33.235099+0000","last_fullsized":"2024-04-28T13:32:33.235099+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:08:17.119169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234962+0000","last_change":"2024-04-28T13:32:32.231690+0000","last_active":"2024-04-28T13:32:33.234962+0000","last_peered":"2024-04-28T13:32:33.234962+0000","last_clean":"2024-04-28T13:32:33.234962+0000","last_became_active":"2024-04-28T13:32:32.231444+0000","last_became_peered":"2024-04-28T13:32:32.231444+0000","last_unstale":"2024-04-28T13:32:33.234962+0000","last_undegraded":"2024-04-28T13:32:33.234962+0000","last_fullsized":"2024-04-28T13:32:33.234962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:22:03.865574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564339+0000","last_change":"2024-04-28T13:32:31.228938+0000","last_active":"2024-04-28T13:32:33.564339+0000","last_peered":"2024-04-28T13:32:33.564339+0000","last_clean":"2024-04-28T13:32:33.564339+0000","last_became_active":"2024-04-28T13:32:31.228699+0000","last_became_peered":"2024-04-28T13:32:31.228699+0000","last_unstale":"2024-04-28T13:32:33.564339+0000","last_undegraded":"2024-04-28T13:32:33.564339+0000","last_fullsized":"2024-04-28T13:32:33.564339+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:33:40.443015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.246794+0000","last_change":"2024-04-28T13:32:32.236711+0000","last_active":"2024-04-28T13:32:33.246794+0000","last_peered":"2024-04-28T13:32:33.246794+0000","last_clean":"2024-04-28T13:32:33.246794+0000","last_became_active":"2024-04-28T13:32:32.236383+0000","last_became_peered":"2024-04-28T13:32:32.236383+0000","last_unstale":"2024-04-28T13:32:33.246794+0000","last_undegraded":"2024-04-28T13:32:33.246794+0000","last_fullsized":"2024-04-28T13:32:33.246794+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:53:48.013290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236347+0000","last_change":"2024-04-28T13:32:31.224596+0000","last_active":"2024-04-28T13:32:33.236347+0000","last_peered":"2024-04-28T13:32:33.236347+0000","last_clean":"2024-04-28T13:32:33.236347+0000","last_became_active":"2024-04-28T13:32:31.224300+0000","last_became_peered":"2024-04-28T13:32:31.224300+0000","last_unstale":"2024-04-28T13:32:33.236347+0000","last_undegraded":"2024-04-28T13:32:33.236347+0000","last_fullsized":"2024-04-28T13:32:33.236347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:12:10.305333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563792+0000","last_change":"2024-04-28T13:32:32.232144+0000","last_active":"2024-04-28T13:32:33.563792+0000","last_peered":"2024-04-28T13:32:33.563792+0000","last_clean":"2024-04-28T13:32:33.563792+0000","last_became_active":"2024-04-28T13:32:32.231852+0000","last_became_peered":"2024-04-28T13:32:32.231852+0000","last_unstale":"2024-04-28T13:32:33.563792+0000","last_undegraded":"2024-04-28T13:32:33.563792+0000","last_fullsized":"2024-04-28T13:32:33.563792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:45:01.603947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.245825+0000","last_change":"2024-04-28T13:32:31.230105+0000","last_active":"2024-04-28T13:32:33.245825+0000","last_peered":"2024-04-28T13:32:33.245825+0000","last_clean":"2024-04-28T13:32:33.245825+0000","last_became_active":"2024-04-28T13:32:31.229764+0000","last_became_peered":"2024-04-28T13:32:31.229764+0000","last_unstale":"2024-04-28T13:32:33.245825+0000","last_undegraded":"2024-04-28T13:32:33.245825+0000","last_fullsized":"2024-04-28T13:32:33.245825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:58.653633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564019+0000","last_change":"2024-04-28T13:32:32.245969+0000","last_active":"2024-04-28T13:32:33.564019+0000","last_peered":"2024-04-28T13:32:33.564019+0000","last_clean":"2024-04-28T13:32:33.564019+0000","last_became_active":"2024-04-28T13:32:32.245095+0000","last_became_peered":"2024-04-28T13:32:32.245095+0000","last_unstale":"2024-04-28T13:32:33.564019+0000","last_undegraded":"2024-04-28T13:32:33.564019+0000","last_fullsized":"2024-04-28T13:32:33.564019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:00:57.153237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563352+0000","last_change":"2024-04-28T13:32:31.230987+0000","last_active":"2024-04-28T13:32:33.563352+0000","last_peered":"2024-04-28T13:32:33.563352+0000","last_clean":"2024-04-28T13:32:33.563352+0000","last_became_active":"2024-04-28T13:32:31.230756+0000","last_became_peered":"2024-04-28T13:32:31.230756+0000","last_unstale":"2024-04-28T13:32:33.563352+0000","last_undegraded":"2024-04-28T13:32:33.563352+0000","last_fullsized":"2024-04-28T13:32:33.563352+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:08:17.119169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563814+0000","last_change":"2024-04-28T13:32:32.235161+0000","last_active":"2024-04-28T13:32:33.563814+0000","last_peered":"2024-04-28T13:32:33.563814+0000","last_clean":"2024-04-28T13:32:33.563814+0000","last_became_active":"2024-04-28T13:32:32.234838+0000","last_became_peered":"2024-04-28T13:32:32.234838+0000","last_unstale":"2024-04-28T13:32:33.563814+0000","last_undegraded":"2024-04-28T13:32:33.563814+0000","last_fullsized":"2024-04-28T13:32:33.563814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:08:18.134385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563369+0000","last_change":"2024-04-28T13:32:31.233385+0000","last_active":"2024-04-28T13:32:33.563369+0000","last_peered":"2024-04-28T13:32:33.563369+0000","last_clean":"2024-04-28T13:32:33.563369+0000","last_became_active":"2024-04-28T13:32:31.233075+0000","last_became_peered":"2024-04-28T13:32:31.233075+0000","last_unstale":"2024-04-28T13:32:33.563369+0000","last_undegraded":"2024-04-28T13:32:33.563369+0000","last_fullsized":"2024-04-28T13:32:33.563369+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:20:51.755318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236569+0000","last_change":"2024-04-28T13:32:32.237097+0000","last_active":"2024-04-28T13:32:33.236569+0000","last_peered":"2024-04-28T13:32:33.236569+0000","last_clean":"2024-04-28T13:32:33.236569+0000","last_became_active":"2024-04-28T13:32:32.236693+0000","last_became_peered":"2024-04-28T13:32:32.236693+0000","last_unstale":"2024-04-28T13:32:33.236569+0000","last_undegraded":"2024-04-28T13:32:33.236569+0000","last_fullsized":"2024-04-28T13:32:33.236569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:53:48.013290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.246997+0000","last_change":"2024-04-28T13:32:31.223266+0000","last_active":"2024-04-28T13:32:33.246997+0000","last_peered":"2024-04-28T13:32:33.246997+0000","last_clean":"2024-04-28T13:32:33.246997+0000","last_became_active":"2024-04-28T13:32:31.222951+0000","last_became_peered":"2024-04-28T13:32:31.222951+0000","last_unstale":"2024-04-28T13:32:33.246997+0000","last_undegraded":"2024-04-28T13:32:33.246997+0000","last_fullsized":"2024-04-28T13:32:33.246997+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:24:06.561968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235772+0000","last_change":"2024-04-28T13:32:32.229844+0000","last_active":"2024-04-28T13:32:33.235772+0000","last_peered":"2024-04-28T13:32:33.235772+0000","last_clean":"2024-04-28T13:32:33.235772+0000","last_became_active":"2024-04-28T13:32:32.228009+0000","last_became_peered":"2024-04-28T13:32:32.228009+0000","last_unstale":"2024-04-28T13:32:33.235772+0000","last_undegraded":"2024-04-28T13:32:33.235772+0000","last_fullsized":"2024-04-28T13:32:33.235772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:23:33.274183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236655+0000","last_change":"2024-04-28T13:32:31.226304+0000","last_active":"2024-04-28T13:32:33.236655+0000","last_peered":"2024-04-28T13:32:33.236655+0000","last_clean":"2024-04-28T13:32:33.236655+0000","last_became_active":"2024-04-28T13:32:31.225972+0000","last_became_peered":"2024-04-28T13:32:31.225972+0000","last_unstale":"2024-04-28T13:32:33.236655+0000","last_undegraded":"2024-04-28T13:32:33.236655+0000","last_fullsized":"2024-04-28T13:32:33.236655+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:13:46.879685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.247117+0000","last_change":"2024-04-28T13:32:32.238416+0000","last_active":"2024-04-28T13:32:33.247117+0000","last_peered":"2024-04-28T13:32:33.247117+0000","last_clean":"2024-04-28T13:32:33.247117+0000","last_became_active":"2024-04-28T13:32:32.237247+0000","last_became_peered":"2024-04-28T13:32:32.237247+0000","last_unstale":"2024-04-28T13:32:33.247117+0000","last_undegraded":"2024-04-28T13:32:33.247117+0000","last_fullsized":"2024-04-28T13:32:33.247117+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:14:48.471231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236749+0000","last_change":"2024-04-28T13:32:31.216936+0000","last_active":"2024-04-28T13:32:33.236749+0000","last_peered":"2024-04-28T13:32:33.236749+0000","last_clean":"2024-04-28T13:32:33.236749+0000","last_became_active":"2024-04-28T13:32:31.214638+0000","last_became_peered":"2024-04-28T13:32:31.214638+0000","last_unstale":"2024-04-28T13:32:33.236749+0000","last_undegraded":"2024-04-28T13:32:33.236749+0000","last_fullsized":"2024-04-28T13:32:33.236749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:20:26.159036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.561202+0000","last_change":"2024-04-28T13:32:32.245218+0000","last_active":"2024-04-28T13:32:33.561202+0000","last_peered":"2024-04-28T13:32:33.561202+0000","last_clean":"2024-04-28T13:32:33.561202+0000","last_became_active":"2024-04-28T13:32:32.244867+0000","last_became_peered":"2024-04-28T13:32:32.244867+0000","last_unstale":"2024-04-28T13:32:33.561202+0000","last_undegraded":"2024-04-28T13:32:33.561202+0000","last_fullsized":"2024-04-28T13:32:33.561202+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:12:11.320549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236582+0000","last_change":"2024-04-28T13:32:31.225015+0000","last_active":"2024-04-28T13:32:33.236582+0000","last_peered":"2024-04-28T13:32:33.236582+0000","last_clean":"2024-04-28T13:32:33.236582+0000","last_became_active":"2024-04-28T13:32:31.224704+0000","last_became_peered":"2024-04-28T13:32:31.224704+0000","last_unstale":"2024-04-28T13:32:33.236582+0000","last_undegraded":"2024-04-28T13:32:33.236582+0000","last_fullsized":"2024-04-28T13:32:33.236582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:24:06.561968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.560877+0000","last_change":"2024-04-28T13:32:32.232786+0000","last_active":"2024-04-28T13:32:33.560877+0000","last_peered":"2024-04-28T13:32:33.560877+0000","last_clean":"2024-04-28T13:32:33.560877+0000","last_became_active":"2024-04-28T13:32:32.232492+0000","last_became_peered":"2024-04-28T13:32:32.232492+0000","last_unstale":"2024-04-28T13:32:33.560877+0000","last_undegraded":"2024-04-28T13:32:33.560877+0000","last_fullsized":"2024-04-28T13:32:33.560877+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:11:02.849386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235178+0000","last_change":"2024-04-28T13:32:31.221816+0000","last_active":"2024-04-28T13:32:33.235178+0000","last_peered":"2024-04-28T13:32:33.235178+0000","last_clean":"2024-04-28T13:32:33.235178+0000","last_became_active":"2024-04-28T13:32:31.221471+0000","last_became_peered":"2024-04-28T13:32:31.221471+0000","last_unstale":"2024-04-28T13:32:33.235178+0000","last_undegraded":"2024-04-28T13:32:33.235178+0000","last_fullsized":"2024-04-28T13:32:33.235178+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:12:10.305333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.561108+0000","last_change":"2024-04-28T13:32:32.235301+0000","last_active":"2024-04-28T13:32:33.561108+0000","last_peered":"2024-04-28T13:32:33.561108+0000","last_clean":"2024-04-28T13:32:33.561108+0000","last_became_active":"2024-04-28T13:32:32.235033+0000","last_became_peered":"2024-04-28T13:32:32.235033+0000","last_unstale":"2024-04-28T13:32:33.561108+0000","last_undegraded":"2024-04-28T13:32:33.561108+0000","last_fullsized":"2024-04-28T13:32:33.561108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:32:14.322085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563862+0000","last_change":"2024-04-28T13:32:31.233004+0000","last_active":"2024-04-28T13:32:33.563862+0000","last_peered":"2024-04-28T13:32:33.563862+0000","last_clean":"2024-04-28T13:32:33.563862+0000","last_became_active":"2024-04-28T13:32:31.232714+0000","last_became_peered":"2024-04-28T13:32:31.232714+0000","last_unstale":"2024-04-28T13:32:33.563862+0000","last_undegraded":"2024-04-28T13:32:33.563862+0000","last_fullsized":"2024-04-28T13:32:33.563862+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:58.653633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.561280+0000","last_change":"2024-04-28T13:32:32.231264+0000","last_active":"2024-04-28T13:32:33.561280+0000","last_peered":"2024-04-28T13:32:33.561280+0000","last_clean":"2024-04-28T13:32:33.561280+0000","last_became_active":"2024-04-28T13:32:32.230470+0000","last_became_peered":"2024-04-28T13:32:32.230470+0000","last_unstale":"2024-04-28T13:32:33.561280+0000","last_undegraded":"2024-04-28T13:32:33.561280+0000","last_fullsized":"2024-04-28T13:32:33.561280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:24:07.577184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235903+0000","last_change":"2024-04-28T13:32:31.215630+0000","last_active":"2024-04-28T13:32:33.235903+0000","last_peered":"2024-04-28T13:32:33.235903+0000","last_clean":"2024-04-28T13:32:33.235903+0000","last_became_active":"2024-04-28T13:32:31.215172+0000","last_became_peered":"2024-04-28T13:32:31.215172+0000","last_unstale":"2024-04-28T13:32:33.235903+0000","last_undegraded":"2024-04-28T13:32:33.235903+0000","last_fullsized":"2024-04-28T13:32:33.235903+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:30:48.400845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.565267+0000","last_change":"2024-04-28T13:32:32.253268+0000","last_active":"2024-04-28T13:32:33.565267+0000","last_peered":"2024-04-28T13:32:33.565267+0000","last_clean":"2024-04-28T13:32:33.565267+0000","last_became_active":"2024-04-28T13:32:32.251933+0000","last_became_peered":"2024-04-28T13:32:32.251933+0000","last_unstale":"2024-04-28T13:32:33.565267+0000","last_undegraded":"2024-04-28T13:32:33.565267+0000","last_fullsized":"2024-04-28T13:32:33.565267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:22:58.887460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.233962+0000","last_change":"2024-04-28T13:32:31.225740+0000","last_active":"2024-04-28T13:32:33.233962+0000","last_peered":"2024-04-28T13:32:33.233962+0000","last_clean":"2024-04-28T13:32:33.233962+0000","last_became_active":"2024-04-28T13:32:31.225458+0000","last_became_peered":"2024-04-28T13:32:31.225458+0000","last_unstale":"2024-04-28T13:32:33.233962+0000","last_undegraded":"2024-04-28T13:32:33.233962+0000","last_fullsized":"2024-04-28T13:32:33.233962+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:33:53.691067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.561244+0000","last_change":"2024-04-28T13:32:32.242450+0000","last_active":"2024-04-28T13:32:33.561244+0000","last_peered":"2024-04-28T13:32:33.561244+0000","last_clean":"2024-04-28T13:32:33.561244+0000","last_became_active":"2024-04-28T13:32:32.242148+0000","last_became_peered":"2024-04-28T13:32:32.242148+0000","last_unstale":"2024-04-28T13:32:33.561244+0000","last_undegraded":"2024-04-28T13:32:33.561244+0000","last_fullsized":"2024-04-28T13:32:33.561244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:59.668849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.247125+0000","last_change":"2024-04-28T13:32:31.227326+0000","last_active":"2024-04-28T13:32:33.247125+0000","last_peered":"2024-04-28T13:32:33.247125+0000","last_clean":"2024-04-28T13:32:33.247125+0000","last_became_active":"2024-04-28T13:32:31.226143+0000","last_became_peered":"2024-04-28T13:32:31.226143+0000","last_unstale":"2024-04-28T13:32:33.247125+0000","last_undegraded":"2024-04-28T13:32:33.247125+0000","last_fullsized":"2024-04-28T13:32:33.247125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:30:34.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235619+0000","last_change":"2024-04-28T13:32:32.231308+0000","last_active":"2024-04-28T13:32:33.235619+0000","last_peered":"2024-04-28T13:32:33.235619+0000","last_clean":"2024-04-28T13:32:33.235619+0000","last_became_active":"2024-04-28T13:32:32.230914+0000","last_became_peered":"2024-04-28T13:32:32.230914+0000","last_unstale":"2024-04-28T13:32:33.235619+0000","last_undegraded":"2024-04-28T13:32:33.235619+0000","last_fullsized":"2024-04-28T13:32:33.235619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:59:15.390148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235269+0000","last_change":"2024-04-28T13:32:31.227035+0000","last_active":"2024-04-28T13:32:33.235269+0000","last_peered":"2024-04-28T13:32:33.235269+0000","last_clean":"2024-04-28T13:32:33.235269+0000","last_became_active":"2024-04-28T13:32:31.226312+0000","last_became_peered":"2024-04-28T13:32:31.226312+0000","last_unstale":"2024-04-28T13:32:33.235269+0000","last_undegraded":"2024-04-28T13:32:33.235269+0000","last_fullsized":"2024-04-28T13:32:33.235269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:30:34.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.234727+0000","last_change":"2024-04-28T13:32:32.233696+0000","last_active":"2024-04-28T13:32:33.234727+0000","last_peered":"2024-04-28T13:32:33.234727+0000","last_clean":"2024-04-28T13:32:33.234727+0000","last_became_active":"2024-04-28T13:32:32.233372+0000","last_became_peered":"2024-04-28T13:32:32.233372+0000","last_unstale":"2024-04-28T13:32:33.234727+0000","last_undegraded":"2024-04-28T13:32:33.234727+0000","last_fullsized":"2024-04-28T13:32:33.234727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:41:57.017489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.246994+0000","last_change":"2024-04-28T13:32:31.226997+0000","last_active":"2024-04-28T13:32:33.246994+0000","last_peered":"2024-04-28T13:32:33.246994+0000","last_clean":"2024-04-28T13:32:33.246994+0000","last_became_active":"2024-04-28T13:32:31.226705+0000","last_became_peered":"2024-04-28T13:32:31.226705+0000","last_unstale":"2024-04-28T13:32:33.246994+0000","last_undegraded":"2024-04-28T13:32:33.246994+0000","last_fullsized":"2024-04-28T13:32:33.246994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:12:10.305333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564089+0000","last_change":"2024-04-28T13:32:32.247886+0000","last_active":"2024-04-28T13:32:33.564089+0000","last_peered":"2024-04-28T13:32:33.564089+0000","last_clean":"2024-04-28T13:32:33.564089+0000","last_became_active":"2024-04-28T13:32:32.246316+0000","last_became_peered":"2024-04-28T13:32:32.246316+0000","last_unstale":"2024-04-28T13:32:33.564089+0000","last_undegraded":"2024-04-28T13:32:33.564089+0000","last_fullsized":"2024-04-28T13:32:33.564089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:20:52.770534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235483+0000","last_change":"2024-04-28T13:32:31.216200+0000","last_active":"2024-04-28T13:32:33.235483+0000","last_peered":"2024-04-28T13:32:33.235483+0000","last_clean":"2024-04-28T13:32:33.235483+0000","last_became_active":"2024-04-28T13:32:31.215800+0000","last_became_peered":"2024-04-28T13:32:31.215800+0000","last_unstale":"2024-04-28T13:32:33.235483+0000","last_undegraded":"2024-04-28T13:32:33.235483+0000","last_fullsized":"2024-04-28T13:32:33.235483+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:58.653633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.235371+0000","last_change":"2024-04-28T13:32:32.241745+0000","last_active":"2024-04-28T13:32:33.235371+0000","last_peered":"2024-04-28T13:32:33.235371+0000","last_clean":"2024-04-28T13:32:33.235371+0000","last_became_active":"2024-04-28T13:32:32.241438+0000","last_became_peered":"2024-04-28T13:32:32.241438+0000","last_unstale":"2024-04-28T13:32:33.235371+0000","last_undegraded":"2024-04-28T13:32:33.235371+0000","last_fullsized":"2024-04-28T13:32:33.235371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:01:56.621985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.246042+0000","last_change":"2024-04-28T13:32:31.220706+0000","last_active":"2024-04-28T13:32:33.246042+0000","last_peered":"2024-04-28T13:32:33.246042+0000","last_clean":"2024-04-28T13:32:33.246042+0000","last_became_active":"2024-04-28T13:32:31.220451+0000","last_became_peered":"2024-04-28T13:32:31.220451+0000","last_unstale":"2024-04-28T13:32:33.246042+0000","last_undegraded":"2024-04-28T13:32:33.246042+0000","last_fullsized":"2024-04-28T13:32:33.246042+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:20:26.159036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563517+0000","last_change":"2024-04-28T13:32:32.231453+0000","last_active":"2024-04-28T13:32:33.563517+0000","last_peered":"2024-04-28T13:32:33.563517+0000","last_clean":"2024-04-28T13:32:33.563517+0000","last_became_active":"2024-04-28T13:32:32.231093+0000","last_became_peered":"2024-04-28T13:32:32.231093+0000","last_unstale":"2024-04-28T13:32:33.563517+0000","last_undegraded":"2024-04-28T13:32:33.563517+0000","last_fullsized":"2024-04-28T13:32:33.563517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:22:03.865574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236749+0000","last_change":"2024-04-28T13:32:31.225974+0000","last_active":"2024-04-28T13:32:33.236749+0000","last_peered":"2024-04-28T13:32:33.236749+0000","last_clean":"2024-04-28T13:32:33.236749+0000","last_became_active":"2024-04-28T13:32:31.219380+0000","last_became_peered":"2024-04-28T13:32:31.219380+0000","last_unstale":"2024-04-28T13:32:33.236749+0000","last_undegraded":"2024-04-28T13:32:33.236749+0000","last_fullsized":"2024-04-28T13:32:33.236749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:11:01.834169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.236753+0000","last_change":"2024-04-28T13:32:32.234417+0000","last_active":"2024-04-28T13:32:33.236753+0000","last_peered":"2024-04-28T13:32:33.236753+0000","last_clean":"2024-04-28T13:32:33.236753+0000","last_became_active":"2024-04-28T13:32:32.233680+0000","last_became_peered":"2024-04-28T13:32:32.233680+0000","last_unstale":"2024-04-28T13:32:33.236753+0000","last_undegraded":"2024-04-28T13:32:33.236753+0000","last_fullsized":"2024-04-28T13:32:33.236753+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:34:54.602566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.563203+0000","last_change":"2024-04-28T13:32:31.228304+0000","last_active":"2024-04-28T13:32:33.563203+0000","last_peered":"2024-04-28T13:32:33.563203+0000","last_clean":"2024-04-28T13:32:33.563203+0000","last_became_active":"2024-04-28T13:32:31.227982+0000","last_became_peered":"2024-04-28T13:32:31.227982+0000","last_unstale":"2024-04-28T13:32:33.563203+0000","last_undegraded":"2024-04-28T13:32:33.563203+0000","last_fullsized":"2024-04-28T13:32:33.563203+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:23:30.639925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.566285+0000","last_change":"2024-04-28T13:32:32.242809+0000","last_active":"2024-04-28T13:32:33.566285+0000","last_peered":"2024-04-28T13:32:33.566285+0000","last_clean":"2024-04-28T13:32:33.566285+0000","last_became_active":"2024-04-28T13:32:32.242516+0000","last_became_peered":"2024-04-28T13:32:32.242516+0000","last_unstale":"2024-04-28T13:32:33.566285+0000","last_undegraded":"2024-04-28T13:32:33.566285+0000","last_fullsized":"2024-04-28T13:32:33.566285+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:31.179150+0000","last_clean_scrub_stamp":"2024-04-28T13:32:31.179150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:02:39.028860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564036+0000","last_change":"2024-04-28T13:32:31.228458+0000","last_active":"2024-04-28T13:32:33.564036+0000","last_peered":"2024-04-28T13:32:33.564036+0000","last_clean":"2024-04-28T13:32:33.564036+0000","last_became_active":"2024-04-28T13:32:31.228166+0000","last_became_peered":"2024-04-28T13:32:31.228166+0000","last_unstale":"2024-04-28T13:32:33.564036+0000","last_undegraded":"2024-04-28T13:32:33.564036+0000","last_fullsized":"2024-04-28T13:32:33.564036+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:32:30.163934+0000","last_clean_scrub_stamp":"2024-04-28T13:32:30.163934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:13:46.879685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-28T13:32:33.564733+0000","last_change":"2024-04-28T13:31:11.831538+0000","last_active":"2024-04-28T13:32:33.564733+0000","last_peered":"2024-04-28T13:32:33.564733+0000","last_clean":"2024-04-28T13:32:33.564733+0000","last_became_active":"2024-04-28T13:31:11.828959+0000","last_became_peered":"2024-04-28T13:31:11.828959+0000","last_unstale":"2024-04-28T13:32:33.564733+0000","last_undegraded":"2024-04-28T13:32:33.564733+0000","last_fullsized":"2024-04-28T13:32:33.564733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_clean_scrub_stamp":"2024-04-28T13:31:10.444388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:25:27.078911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":737082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":736992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T13:32:37.164 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-28T13:32:37.165 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-28T13:32:37.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.417+0000 7f401c400640 1 Processor -- start 2024-04-28T13:32:37.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.418+0000 7f401c400640 1 -- start start 2024-04-28T13:32:37.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.419+0000 7f401c400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40140b6300 0x7f40140b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:37.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.419+0000 7f401c400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f401415d030 0x7f40140720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:37.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.419+0000 7f401c400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f401415d4e0 con 0x7f40141459a0 2024-04-28T13:32:37.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.419+0000 7f401c400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f40140b4480 con 0x7f401415d030 2024-04-28T13:32:37.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.419+0000 7f401c400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f40140b4760 con 0x7f40140b6300 2024-04-28T13:32:37.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.420+0000 7f401a400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f401415d030 0x7f40140720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:37.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.420+0000 7f401a400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f401415d030 0x7f40140720f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34336/0 (socket says 172.21.15.196:34336) 2024-04-28T13:32:37.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.420+0000 7f401a400640 1 -- 172.21.15.196:0/3973471531 learned_addr learned my addr 172.21.15.196:0/3973471531 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:37.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.420+0000 7f4019000640 1 --2- 172.21.15.196:0/3973471531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40140b6300 0x7f40140b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:37.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.420+0000 7f401a400640 1 -- 172.21.15.196:0/3973471531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40140b6300 msgr2=0x7f40140b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:37.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.420+0000 7f401a400640 1 --2- 172.21.15.196:0/3973471531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40140b6300 0x7f40140b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.420+0000 7f401a400640 1 -- 172.21.15.196:0/3973471531 >> v1:172.21.15.203:6789/0 conn(0x7f40141459a0 legacy=0x7f40140b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:37.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.420+0000 7f401a400640 1 -- 172.21.15.196:0/3973471531 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40140b4a40 con 0x7f401415d030 2024-04-28T13:32:37.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.420+0000 7f4019000640 1 --2- 172.21.15.196:0/3973471531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40140b6300 0x7f40140b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:37.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.421+0000 7f401a400640 1 --2- 172.21.15.196:0/3973471531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f401415d030 0x7f40140720f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f400c009e50 tx=0x7f400c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5ce346852932d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:37.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.422+0000 7f4013e00640 1 -- 172.21.15.196:0/3973471531 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c036070 con 0x7f401415d030 2024-04-28T13:32:37.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.422+0000 7f4013e00640 1 -- 172.21.15.196:0/3973471531 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f400c031600 con 0x7f401415d030 2024-04-28T13:32:37.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.422+0000 7f4013e00640 1 -- 172.21.15.196:0/3973471531 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c0319d0 con 0x7f401415d030 2024-04-28T13:32:37.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.422+0000 7f401c400640 1 -- 172.21.15.196:0/3973471531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f401415d030 msgr2=0x7f40140720f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:37.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.422+0000 7f401c400640 1 --2- 172.21.15.196:0/3973471531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f401415d030 0x7f40140720f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f400c009e50 tx=0x7f400c02a6d0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.422+0000 7f401c400640 1 -- 172.21.15.196:0/3973471531 shutdown_connections 2024-04-28T13:32:37.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.422+0000 7f401c400640 1 --2- 172.21.15.196:0/3973471531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f401415d030 0x7f40140720f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.422+0000 7f401c400640 1 --2- 172.21.15.196:0/3973471531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40140b6300 0x7f40140b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.422+0000 7f401c400640 1 -- 172.21.15.196:0/3973471531 >> 172.21.15.196:0/3973471531 conn(0x7f40140d0d30 msgr2=0x7f40140d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:37.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.422+0000 7f401c400640 1 -- 172.21.15.196:0/3973471531 shutdown_connections 2024-04-28T13:32:37.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.422+0000 7f401c400640 1 -- 172.21.15.196:0/3973471531 wait complete. 2024-04-28T13:32:37.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.424+0000 7f401c400640 1 Processor -- start 2024-04-28T13:32:37.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.424+0000 7f401c400640 1 -- start start 2024-04-28T13:32:37.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.425+0000 7f401c400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40140b6300 0x7f401415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:37.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.425+0000 7f401c400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40141459a0 0x7f40140b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.425+0000 7f401c400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f401415d030 0x7f40140b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.425+0000 7f401c400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4014072630 con 0x7f401415d030 2024-04-28T13:32:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.425+0000 7f401c400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f40140b5000 con 0x7f40140b6300 2024-04-28T13:32:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.425+0000 7f401c400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f40140727b0 con 0x7f40141459a0 2024-04-28T13:32:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.425+0000 7f401a400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f401415d030 0x7f40140b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.426+0000 7f401a400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f401415d030 0x7f40140b2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34348/0 (socket says 172.21.15.196:34348) 2024-04-28T13:32:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.426+0000 7f401a400640 1 -- 172.21.15.196:0/792860374 learned_addr learned my addr 172.21.15.196:0/792860374 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.426+0000 7f4019a00640 1 --2- 172.21.15.196:0/792860374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40140b6300 0x7f401415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.426+0000 7f401a400640 1 -- 172.21.15.196:0/792860374 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40141459a0 msgr2=0x7f40140b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.426+0000 7f401a400640 1 --2- 172.21.15.196:0/792860374 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40141459a0 0x7f40140b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.426+0000 7f401a400640 1 -- 172.21.15.196:0/792860374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40140b6300 msgr2=0x7f401415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.426+0000 7f401a400640 1 --2- 172.21.15.196:0/792860374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40140b6300 0x7f401415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.426+0000 7f401a400640 1 -- 172.21.15.196:0/792860374 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40140b0140 con 0x7f401415d030 2024-04-28T13:32:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.426+0000 7f401a400640 1 --2- 172.21.15.196:0/792860374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f401415d030 0x7f40140b2ea0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f400c034040 tx=0x7f400c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.427+0000 7f4011600640 1 -- 172.21.15.196:0/792860374 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c036050 con 0x7f401415d030 2024-04-28T13:32:37.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.427+0000 7f4011600640 1 -- 172.21.15.196:0/792860374 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f400c042070 con 0x7f401415d030 2024-04-28T13:32:37.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.427+0000 7f4011600640 1 -- 172.21.15.196:0/792860374 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c009510 con 0x7f401415d030 2024-04-28T13:32:37.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.427+0000 7f401c400640 1 -- 172.21.15.196:0/792860374 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40140b1e80 con 0x7f401415d030 2024-04-28T13:32:37.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.427+0000 7f401c400640 1 -- 172.21.15.196:0/792860374 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40140b1920 con 0x7f401415d030 2024-04-28T13:32:37.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.428+0000 7f401c400640 1 -- 172.21.15.196:0/792860374 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f401406a290 con 0x7f401415d030 2024-04-28T13:32:37.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.429+0000 7f4011600640 1 -- 172.21.15.196:0/792860374 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f400c007bd0 con 0x7f401415d030 2024-04-28T13:32:37.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.430+0000 7f4011600640 1 --2- 172.21.15.196:0/792860374 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3ff406c8f0 0x7f3ff406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:37.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.430+0000 7f4011600640 1 -- 172.21.15.196:0/792860374 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f400c0aca50 con 0x7f401415d030 2024-04-28T13:32:37.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.431+0000 7f4019a00640 1 --2- 172.21.15.196:0/792860374 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3ff406c8f0 0x7f3ff406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:37.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.432+0000 7f4019a00640 1 --2- 172.21.15.196:0/792860374 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3ff406c8f0 0x7f3ff406edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4004004ea0 tx=0x7f4004009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:37.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.435+0000 7f4011600640 1 -- 172.21.15.196:0/792860374 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f400c07aef0 con 0x7f401415d030 2024-04-28T13:32:37.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.712+0000 7f401c400640 1 -- 172.21.15.196:0/792860374 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f40140b1c30 con 0x7f401415d030 2024-04-28T13:32:37.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.717+0000 7f4011600640 1 -- 172.21.15.196:0/792860374 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f400c07a7f0 con 0x7f401415d030 2024-04-28T13:32:37.717 INFO:teuthology.orchestra.run.smithi196.stdout:[client.0] 2024-04-28T13:32:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBlTy5m8movMBAAnKfzHF0pAkX1F/dZiB0qBg== 2024-04-28T13:32:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: caps mds = "allow" 2024-04-28T13:32:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: caps mon = "allow r" 2024-04-28T13:32:37.718 INFO:teuthology.orchestra.run.smithi196.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-28T13:32:37.718 INFO:teuthology.orchestra.run.smithi196.stderr:updated caps for client.0 2024-04-28T13:32:37.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.720+0000 7f401c400640 1 -- 172.21.15.196:0/792860374 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3ff406c8f0 msgr2=0x7f3ff406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:37.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.720+0000 7f401c400640 1 --2- 172.21.15.196:0/792860374 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3ff406c8f0 0x7f3ff406edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4004004ea0 tx=0x7f4004009040 comp rx=0 tx=0).stop 2024-04-28T13:32:37.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.720+0000 7f401c400640 1 -- 172.21.15.196:0/792860374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f401415d030 msgr2=0x7f40140b2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:37.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.720+0000 7f401c400640 1 --2- 172.21.15.196:0/792860374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f401415d030 0x7f40140b2ea0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f400c034040 tx=0x7f400c004a40 comp rx=0 tx=0).stop 2024-04-28T13:32:37.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.720+0000 7f401c400640 1 -- 172.21.15.196:0/792860374 shutdown_connections 2024-04-28T13:32:37.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.720+0000 7f401c400640 1 --2- 172.21.15.196:0/792860374 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3ff406c8f0 0x7f3ff406edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.720+0000 7f401c400640 1 --2- 172.21.15.196:0/792860374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f401415d030 0x7f40140b2ea0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.720+0000 7f401c400640 1 --2- 172.21.15.196:0/792860374 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40141459a0 0x7f40140b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.720+0000 7f401c400640 1 --2- 172.21.15.196:0/792860374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40140b6300 0x7f401415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.720+0000 7f401c400640 1 -- 172.21.15.196:0/792860374 >> 172.21.15.196:0/792860374 conn(0x7f40140d0d30 msgr2=0x7f40140b5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.720+0000 7f401c400640 1 -- 172.21.15.196:0/792860374 shutdown_connections 2024-04-28T13:32:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.720+0000 7f401c400640 1 -- 172.21.15.196:0/792860374 wait complete. 2024-04-28T13:32:37.738 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:32:37.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.939+0000 7f3e67800640 1 Processor -- start 2024-04-28T13:32:37.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.939+0000 7f3e67800640 1 -- start start 2024-04-28T13:32:37.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.940+0000 7f3e67800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e600d0d00 0x7f3e600d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:37.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.941+0000 7f3e67800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3e600b5dd0 0x7f3e600b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:37.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.941+0000 7f3e67800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e60160f40 con 0x7f3e601469f0 2024-04-28T13:32:37.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.941+0000 7f3e67800640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3e600b5960 con 0x7f3e600b5dd0 2024-04-28T13:32:37.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.941+0000 7f3e67800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3e600d16a0 con 0x7f3e600d0d00 2024-04-28T13:32:37.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.941+0000 7f3e65800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3e600b5dd0 0x7f3e600b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:37.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.941+0000 7f3e65800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3e600b5dd0 0x7f3e600b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:56192/0 (socket says 172.21.15.196:56192) 2024-04-28T13:32:37.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.941+0000 7f3e5fe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e600d0d00 0x7f3e600d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:37.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.941+0000 7f3e65800640 1 -- 172.21.15.196:0/2899466855 learned_addr learned my addr 172.21.15.196:0/2899466855 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:37.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.942+0000 7f3e65800640 1 -- 172.21.15.196:0/2899466855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e600d0d00 msgr2=0x7f3e600d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:37.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.942+0000 7f3e65800640 1 --2- 172.21.15.196:0/2899466855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e600d0d00 0x7f3e600d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.942+0000 7f3e65800640 1 -- 172.21.15.196:0/2899466855 >> v1:172.21.15.203:6789/0 conn(0x7f3e601469f0 legacy=0x7f3e600ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:37.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.942+0000 7f3e65800640 1 -- 172.21.15.196:0/2899466855 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e60161220 con 0x7f3e600b5dd0 2024-04-28T13:32:37.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.942+0000 7f3e5fe00640 1 --2- 172.21.15.196:0/2899466855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e600d0d00 0x7f3e600d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:32:37.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.943+0000 7f3e65800640 1 --2- 172.21.15.196:0/2899466855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e600b5dd0 0x7f3e600b6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009e50 tx=0x7f3e5802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e00bc6caca3b5f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:37.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.944+0000 7f3e5f400640 1 -- 172.21.15.196:0/2899466855 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58036070 con 0x7f3e600b5dd0 2024-04-28T13:32:37.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.944+0000 7f3e5f400640 1 -- 172.21.15.196:0/2899466855 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e58031600 con 0x7f3e600b5dd0 2024-04-28T13:32:37.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.944+0000 7f3e5f400640 1 -- 172.21.15.196:0/2899466855 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e580319d0 con 0x7f3e600b5dd0 2024-04-28T13:32:37.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.945+0000 7f3e67800640 1 -- 172.21.15.196:0/2899466855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e600b5dd0 msgr2=0x7f3e600b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:37.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.945+0000 7f3e67800640 1 --2- 172.21.15.196:0/2899466855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e600b5dd0 0x7f3e600b6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009e50 tx=0x7f3e5802a710 comp rx=0 tx=0).stop 2024-04-28T13:32:37.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.945+0000 7f3e67800640 1 -- 172.21.15.196:0/2899466855 shutdown_connections 2024-04-28T13:32:37.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.945+0000 7f3e67800640 1 --2- 172.21.15.196:0/2899466855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e600b5dd0 0x7f3e600b6260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.945+0000 7f3e67800640 1 --2- 172.21.15.196:0/2899466855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e600d0d00 0x7f3e600d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.945+0000 7f3e67800640 1 -- 172.21.15.196:0/2899466855 >> 172.21.15.196:0/2899466855 conn(0x7f3e600cd830 msgr2=0x7f3e600cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:37.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.945+0000 7f3e67800640 1 -- 172.21.15.196:0/2899466855 shutdown_connections 2024-04-28T13:32:37.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.945+0000 7f3e67800640 1 -- 172.21.15.196:0/2899466855 wait complete. 2024-04-28T13:32:37.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.947+0000 7f3e67800640 1 Processor -- start 2024-04-28T13:32:37.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.947+0000 7f3e67800640 1 -- start start 2024-04-28T13:32:37.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e67800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e600b5dd0 0x7f3e601f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:37.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e67800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e600d0d00 0x7f3e6013f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:37.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e67800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e601469f0 0x7f3e6015c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:37.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e67800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e601617e0 con 0x7f3e601469f0 2024-04-28T13:32:37.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e67800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e60161ac0 con 0x7f3e600b5dd0 2024-04-28T13:32:37.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e67800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3e600cd5f0 con 0x7f3e600d0d00 2024-04-28T13:32:37.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e65800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e601469f0 0x7f3e6015c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:37.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e65800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e601469f0 0x7f3e6015c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34360/0 (socket says 172.21.15.196:34360) 2024-04-28T13:32:37.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e65800640 1 -- 172.21.15.196:0/2407870585 learned_addr learned my addr 172.21.15.196:0/2407870585 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:37.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e64e00640 1 --2- 172.21.15.196:0/2407870585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e600b5dd0 0x7f3e601f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:37.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e5fe00640 1 --2- 172.21.15.196:0/2407870585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e600d0d00 0x7f3e6013f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:37.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e65800640 1 -- 172.21.15.196:0/2407870585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e600d0d00 msgr2=0x7f3e6013f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:37.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e65800640 1 --2- 172.21.15.196:0/2407870585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e600d0d00 0x7f3e6013f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e65800640 1 -- 172.21.15.196:0/2407870585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e600b5dd0 msgr2=0x7f3e601f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:37.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e65800640 1 --2- 172.21.15.196:0/2407870585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e600b5dd0 0x7f3e601f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:37.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.948+0000 7f3e65800640 1 -- 172.21.15.196:0/2407870585 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6013fbc0 con 0x7f3e601469f0 2024-04-28T13:32:37.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.949+0000 7f3e65800640 1 --2- 172.21.15.196:0/2407870585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e601469f0 0x7f3e6015c3c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009e50 tx=0x7f3e58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:37.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.949+0000 7f3e5cc00640 1 -- 172.21.15.196:0/2407870585 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58041020 con 0x7f3e601469f0 2024-04-28T13:32:37.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.949+0000 7f3e5cc00640 1 -- 172.21.15.196:0/2407870585 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e58008730 con 0x7f3e601469f0 2024-04-28T13:32:37.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.949+0000 7f3e5cc00640 1 -- 172.21.15.196:0/2407870585 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58036040 con 0x7f3e601469f0 2024-04-28T13:32:37.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.949+0000 7f3e67800640 1 -- 172.21.15.196:0/2407870585 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e601f6410 con 0x7f3e601469f0 2024-04-28T13:32:37.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.949+0000 7f3e67800640 1 -- 172.21.15.196:0/2407870585 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e60140140 con 0x7f3e601469f0 2024-04-28T13:32:37.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.951+0000 7f3e67800640 1 -- 172.21.15.196:0/2407870585 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e6013da60 con 0x7f3e601469f0 2024-04-28T13:32:37.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.951+0000 7f3e5cc00640 1 -- 172.21.15.196:0/2407870585 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e58044020 con 0x7f3e601469f0 2024-04-28T13:32:37.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.952+0000 7f3e5cc00640 1 --2- 172.21.15.196:0/2407870585 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3e4006c920 0x7f3e4006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:37.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.952+0000 7f3e5cc00640 1 -- 172.21.15.196:0/2407870585 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3e580ac600 con 0x7f3e601469f0 2024-04-28T13:32:37.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.953+0000 7f3e64e00640 1 --2- 172.21.15.196:0/2407870585 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3e4006c920 0x7f3e4006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:37.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.954+0000 7f3e64e00640 1 --2- 172.21.15.196:0/2407870585 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3e4006c920 0x7f3e4006ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e54007fd0 tx=0x7f3e54009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:37.957+0000 7f3e5cc00640 1 -- 172.21.15.196:0/2407870585 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3e5804b030 con 0x7f3e601469f0 2024-04-28T13:32:38.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.195+0000 7f3e67800640 1 -- 172.21.15.196:0/2407870585 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e6006a290 con 0x7f3e601469f0 2024-04-28T13:32:38.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.197+0000 7f3e5cc00640 1 -- 172.21.15.196:0/2407870585 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f3e5807aac0 con 0x7f3e601469f0 2024-04-28T13:32:38.197 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:38.197 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/931076983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":931076983},{"type":"v1","addr":"172.21.15.196:6834","nonce":931076983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1114474485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1114474485},{"type":"v1","addr":"172.21.15.196:6835","nonce":1114474485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2070123371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2070123371},{"type":"v1","addr":"172.21.15.203:6837","nonce":2070123371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:32:31.669472+0000","modified":"2024-04-28T13:32:34.725159+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4735":{"gid":4735,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6836/459640695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":459640695},{"type":"v1","addr":"172.21.15.203:6836","nonce":459640695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4735,"qdb_cluster":[4735]},"id":2}]} 2024-04-28T13:32:38.197 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 16 2024-04-28T13:32:38.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.200+0000 7f3e67800640 1 -- 172.21.15.196:0/2407870585 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3e4006c920 msgr2=0x7f3e4006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:38.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.200+0000 7f3e67800640 1 --2- 172.21.15.196:0/2407870585 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3e4006c920 0x7f3e4006ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e54007fd0 tx=0x7f3e54009040 comp rx=0 tx=0).stop 2024-04-28T13:32:38.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.200+0000 7f3e67800640 1 -- 172.21.15.196:0/2407870585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e601469f0 msgr2=0x7f3e6015c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:38.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.200+0000 7f3e67800640 1 --2- 172.21.15.196:0/2407870585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e601469f0 0x7f3e6015c3c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009e50 tx=0x7f3e58004a40 comp rx=0 tx=0).stop 2024-04-28T13:32:38.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.200+0000 7f3e67800640 1 -- 172.21.15.196:0/2407870585 shutdown_connections 2024-04-28T13:32:38.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.200+0000 7f3e67800640 1 --2- 172.21.15.196:0/2407870585 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3e4006c920 0x7f3e4006ede0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.200+0000 7f3e67800640 1 --2- 172.21.15.196:0/2407870585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e601469f0 0x7f3e6015c3c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.200+0000 7f3e67800640 1 --2- 172.21.15.196:0/2407870585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e600d0d00 0x7f3e6013f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.200+0000 7f3e67800640 1 --2- 172.21.15.196:0/2407870585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e600b5dd0 0x7f3e601f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.200+0000 7f3e67800640 1 -- 172.21.15.196:0/2407870585 >> 172.21.15.196:0/2407870585 conn(0x7f3e600cd830 msgr2=0x7f3e600af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:38.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.200+0000 7f3e67800640 1 -- 172.21.15.196:0/2407870585 shutdown_connections 2024-04-28T13:32:38.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.200+0000 7f3e67800640 1 -- 172.21.15.196:0/2407870585 wait complete. 2024-04-28T13:32:38.621 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-28T13:32:31.669472+0000', 'modified': '2024-04-28T13:32:34.725159+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4735}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4735': {'gid': 4735, 'name': 'b', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.203:6836/459640695', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6835', 'nonce': 459640695}, {'type': 'v1', 'addr': '172.21.15.203:6836', 'nonce': 459640695}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4735, 'qdb_cluster': [4735]} 2024-04-28T13:32:38.621 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-28T13:32:38.621 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-28T13:32:38.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.832+0000 7f470de00640 1 Processor -- start 2024-04-28T13:32:38.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.832+0000 7f470de00640 1 -- start start 2024-04-28T13:32:38.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.834+0000 7f470de00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47081469f0 0x7f47080ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.834+0000 7f470de00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4708160f40 con 0x7f47080cd420 2024-04-28T13:32:38.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.834+0000 7f470de00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4708161220 con 0x7f47080d1410 2024-04-28T13:32:38.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.834+0000 7f470de00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f47080b5960 con 0x7f47081469f0 2024-04-28T13:32:38.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.834+0000 7f4706000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f47080cd420 0x7f47080d0d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:48606/0 (socket says 172.21.15.196:48606) 2024-04-28T13:32:38.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.834+0000 7f4706000640 1 -- 172.21.15.196:0/4223890254 learned_addr learned my addr 172.21.15.196:0/4223890254 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:38.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4706a00640 1 --2- 172.21.15.196:0/4223890254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47081469f0 0x7f47080ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4705600640 1 -- 172.21.15.196:0/4223890254 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2878879079 0 0) 0x7f4708160f40 con 0x7f47080cd420 2024-04-28T13:32:38.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4705600640 1 -- 172.21.15.196:0/4223890254 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46ec003520 con 0x7f47080cd420 2024-04-28T13:32:38.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4705600640 1 -- 172.21.15.196:0/4223890254 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3958386407 0 0) 0x7f46ec003520 con 0x7f47080cd420 2024-04-28T13:32:38.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4705600640 1 -- 172.21.15.196:0/4223890254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47081469f0 msgr2=0x7f47080ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4705600640 1 --2- 172.21.15.196:0/4223890254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47081469f0 0x7f47080ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4705600640 1 -- 172.21.15.196:0/4223890254 >> v1:172.21.15.203:6789/0 conn(0x7f47080d1410 legacy=0x7f47080b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4705600640 1 -- 172.21.15.196:0/4223890254 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4708161500 con 0x7f47080cd420 2024-04-28T13:32:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4705600640 1 -- 172.21.15.196:0/4223890254 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f46f4001690 con 0x7f47080cd420 2024-04-28T13:32:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4705600640 1 -- 172.21.15.196:0/4223890254 >> v1:172.21.15.196:6789/0 conn(0x7f47080cd420 legacy=0x7f47080d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4705600640 1 --2- 172.21.15.196:0/4223890254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46ec0039e0 0x7f46ec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4705600640 1 --2- 172.21.15.196:0/4223890254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46ec004800 0x7f46ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4705600640 1 --2- 172.21.15.196:0/4223890254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47081469f0 0x7f46ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4705600640 1 -- 172.21.15.196:0/4223890254 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec00ba70 con 0x7f46ec004800 2024-04-28T13:32:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4705600640 1 -- 172.21.15.196:0/4223890254 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec00bc20 con 0x7f46ec0039e0 2024-04-28T13:32:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.835+0000 7f4705600640 1 -- 172.21.15.196:0/4223890254 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f46ec00be20 con 0x7f47081469f0 2024-04-28T13:32:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.836+0000 7f4706000640 1 --2- 172.21.15.196:0/4223890254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46ec004800 0x7f46ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.836+0000 7f4706a00640 1 --2- 172.21.15.196:0/4223890254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46ec0039e0 0x7f46ec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.836+0000 7f4706000640 1 -- 172.21.15.196:0/4223890254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47081469f0 msgr2=0x7f46ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:38.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.836+0000 7f4706000640 1 --2- 172.21.15.196:0/4223890254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47081469f0 0x7f46ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.836+0000 7f4706000640 1 -- 172.21.15.196:0/4223890254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46ec0039e0 msgr2=0x7f46ec003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:38.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.836+0000 7f4706000640 1 --2- 172.21.15.196:0/4223890254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46ec0039e0 0x7f46ec003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.836+0000 7f4706000640 1 -- 172.21.15.196:0/4223890254 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec003520 con 0x7f46ec004800 2024-04-28T13:32:38.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.837+0000 7f4706000640 1 --2- 172.21.15.196:0/4223890254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46ec004800 0x7f46ec004c80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f47080cd800 tx=0x7f46ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef552ed0c598430c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:38.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.838+0000 7f4705600640 1 -- 172.21.15.196:0/4223890254 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4007c40 con 0x7f46ec004800 2024-04-28T13:32:38.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.838+0000 7f4705600640 1 -- 172.21.15.196:0/4223890254 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f4005ad0 con 0x7f46ec004800 2024-04-28T13:32:38.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.838+0000 7f4705600640 1 -- 172.21.15.196:0/4223890254 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4007070 con 0x7f46ec004800 2024-04-28T13:32:38.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.838+0000 7f470de00640 1 -- 172.21.15.196:0/4223890254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46ec004800 msgr2=0x7f46ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:38.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.838+0000 7f470de00640 1 --2- 172.21.15.196:0/4223890254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46ec004800 0x7f46ec004c80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f47080cd800 tx=0x7f46ec005830 comp rx=0 tx=0).stop 2024-04-28T13:32:38.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.838+0000 7f4707400640 1 -- 172.21.15.196:0/4223890254 reap_dead start 2024-04-28T13:32:38.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.838+0000 7f470de00640 1 -- 172.21.15.196:0/4223890254 shutdown_connections 2024-04-28T13:32:38.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.838+0000 7f470de00640 1 -- 172.21.15.196:0/4223890254 >> 172.21.15.196:0/4223890254 conn(0x7f47080cd830 msgr2=0x7f47080cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:38.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.838+0000 7f470de00640 1 -- 172.21.15.196:0/4223890254 shutdown_connections 2024-04-28T13:32:38.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.838+0000 7f470de00640 1 -- 172.21.15.196:0/4223890254 wait complete. 2024-04-28T13:32:38.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.840+0000 7f470de00640 1 Processor -- start 2024-04-28T13:32:38.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.840+0000 7f470de00640 1 -- start start 2024-04-28T13:32:38.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.841+0000 7f470de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f470815c8a0 0x7f470815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.841+0000 7f470de00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47080aa9b0 0x7f47081353d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.841+0000 7f470de00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4708135910 0x7f47081f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.841+0000 7f470de00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4708072020 con 0x7f47080aa9b0 2024-04-28T13:32:38.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.841+0000 7f470de00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4708161ac0 con 0x7f470815c8a0 2024-04-28T13:32:38.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.841+0000 7f470de00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4708160b80 con 0x7f4708135910 2024-04-28T13:32:38.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.841+0000 7f4706000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47080aa9b0 0x7f47081353d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.842+0000 7f4706000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47080aa9b0 0x7f47081353d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34380/0 (socket says 172.21.15.196:34380) 2024-04-28T13:32:38.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.842+0000 7f4706000640 1 -- 172.21.15.196:0/478602967 learned_addr learned my addr 172.21.15.196:0/478602967 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:38.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.842+0000 7f4706a00640 1 --2- 172.21.15.196:0/478602967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f470815c8a0 0x7f470815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.842+0000 7f4707400640 1 --2- 172.21.15.196:0/478602967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4708135910 0x7f47081f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.842+0000 7f4706000640 1 -- 172.21.15.196:0/478602967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4708135910 msgr2=0x7f47081f67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:38.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.842+0000 7f4706000640 1 --2- 172.21.15.196:0/478602967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4708135910 0x7f47081f67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.842+0000 7f4706000640 1 -- 172.21.15.196:0/478602967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f470815c8a0 msgr2=0x7f470815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:38.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.842+0000 7f4706000640 1 --2- 172.21.15.196:0/478602967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f470815c8a0 0x7f470815efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.842+0000 7f4706000640 1 -- 172.21.15.196:0/478602967 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4708135d90 con 0x7f47080aa9b0 2024-04-28T13:32:38.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.842+0000 7f4707400640 1 --2- 172.21.15.196:0/478602967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4708135910 0x7f47081f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:38.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.842+0000 7f4706000640 1 --2- 172.21.15.196:0/478602967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47080aa9b0 0x7f47081353d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f46f400c1a0 tx=0x7f47081613f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:38.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.843+0000 7f46ff400640 1 -- 172.21.15.196:0/478602967 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4041020 con 0x7f47080aa9b0 2024-04-28T13:32:38.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.843+0000 7f46ff400640 1 -- 172.21.15.196:0/478602967 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f4038040 con 0x7f47080aa9b0 2024-04-28T13:32:38.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.843+0000 7f470de00640 1 -- 172.21.15.196:0/478602967 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470815f8a0 con 0x7f47080aa9b0 2024-04-28T13:32:38.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.843+0000 7f46ff400640 1 -- 172.21.15.196:0/478602967 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4007290 con 0x7f47080aa9b0 2024-04-28T13:32:38.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.843+0000 7f470de00640 1 -- 172.21.15.196:0/478602967 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470815fe60 con 0x7f47080aa9b0 2024-04-28T13:32:38.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.845+0000 7f46ff400640 1 -- 172.21.15.196:0/478602967 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46f4044020 con 0x7f47080aa9b0 2024-04-28T13:32:38.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.846+0000 7f46ff400640 1 --2- 172.21.15.196:0/478602967 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f46e806c920 0x7f46e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.846+0000 7f46ff400640 1 -- 172.21.15.196:0/478602967 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f46f40abe70 con 0x7f47080aa9b0 2024-04-28T13:32:38.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.847+0000 7f4706a00640 1 --2- 172.21.15.196:0/478602967 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f46e806c920 0x7f46e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.847+0000 7f4710c0a740 1 Processor -- start 2024-04-28T13:32:38.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.847+0000 7f4710c0a740 1 -- start start 2024-04-28T13:32:38.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.848+0000 7f4710c0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed3d55e0 0x55fbed3d7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.848+0000 7f4710c0a740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed411cb0 0x55fbed3d59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.848+0000 7f4710c0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed4000a0 0x55fbed400520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.848+0000 7f4710c0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fbed339600 con 0x55fbed3d55e0 2024-04-28T13:32:38.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.848+0000 7f4710c0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fbed2d6100 con 0x55fbed4000a0 2024-04-28T13:32:38.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.848+0000 7f4710c0a740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55fbed3355c0 con 0x55fbed411cb0 2024-04-28T13:32:38.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.848+0000 7f4706a00640 1 --2- 172.21.15.196:0/478602967 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f46e806c920 0x7f46e806ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009ab0 tx=0x7f46f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:38.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.848+0000 7f4706a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed3d55e0 0x55fbed3d7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.849+0000 7f4706a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed3d55e0 0x55fbed3d7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34390/0 (socket says 172.21.15.196:34390) 2024-04-28T13:32:38.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.849+0000 7f4706a00640 1 -- 172.21.15.196:0/1341216390 learned_addr learned my addr 172.21.15.196:0/1341216390 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:38.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.849+0000 7f4707400640 1 --2- 172.21.15.196:0/1341216390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed4000a0 0x55fbed400520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.849+0000 7f4706000640 1 --2- 172.21.15.196:0/1341216390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed411cb0 0x55fbed3d59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.849+0000 7f4706a00640 1 -- 172.21.15.196:0/1341216390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed411cb0 msgr2=0x55fbed3d59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:38.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.849+0000 7f4706a00640 1 --2- 172.21.15.196:0/1341216390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed411cb0 0x55fbed3d59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.849+0000 7f4706a00640 1 -- 172.21.15.196:0/1341216390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed4000a0 msgr2=0x55fbed400520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:38.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.849+0000 7f4706a00640 1 --2- 172.21.15.196:0/1341216390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed4000a0 0x55fbed400520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.849+0000 7f4706a00640 1 -- 172.21.15.196:0/1341216390 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbed04b2d0 con 0x55fbed3d55e0 2024-04-28T13:32:38.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.849+0000 7f4707400640 1 --2- 172.21.15.196:0/1341216390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed4000a0 0x55fbed400520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:32:38.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.849+0000 7f4706a00640 1 --2- 172.21.15.196:0/1341216390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed3d55e0 0x55fbed3d7740 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f46f8013d90 tx=0x7f46f8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c448d7bc0376ea2c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:38.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.850+0000 7f46f3400640 1 -- 172.21.15.196:0/1341216390 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8013dc0 con 0x55fbed3d55e0 2024-04-28T13:32:38.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.850+0000 7f46f3400640 1 -- 172.21.15.196:0/1341216390 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f801c030 con 0x55fbed3d55e0 2024-04-28T13:32:38.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.850+0000 7f46f3400640 1 -- 172.21.15.196:0/1341216390 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8004220 con 0x55fbed3d55e0 2024-04-28T13:32:38.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.850+0000 7f4710c0a740 1 -- 172.21.15.196:0/1341216390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed3d55e0 msgr2=0x55fbed3d7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:38.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.850+0000 7f4710c0a740 1 --2- 172.21.15.196:0/1341216390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed3d55e0 0x55fbed3d7740 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f46f8013d90 tx=0x7f46f8016000 comp rx=0 tx=0).stop 2024-04-28T13:32:38.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.850+0000 7f4710c0a740 1 -- 172.21.15.196:0/1341216390 shutdown_connections 2024-04-28T13:32:38.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.850+0000 7f4710c0a740 1 --2- 172.21.15.196:0/1341216390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed4000a0 0x55fbed400520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.850+0000 7f4710c0a740 1 --2- 172.21.15.196:0/1341216390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed411cb0 0x55fbed3d59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.850+0000 7f4710c0a740 1 --2- 172.21.15.196:0/1341216390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed3d55e0 0x55fbed3d7740 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.850+0000 7f4710c0a740 1 -- 172.21.15.196:0/1341216390 >> 172.21.15.196:0/1341216390 conn(0x55fbed369d10 msgr2=0x55fbed41b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:38.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.851+0000 7f4710c0a740 1 -- 172.21.15.196:0/1341216390 shutdown_connections 2024-04-28T13:32:38.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.851+0000 7f4710c0a740 1 -- 172.21.15.196:0/1341216390 wait complete. 2024-04-28T13:32:38.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.851+0000 7f4710c0a740 1 -- start start 2024-04-28T13:32:38.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.852+0000 7f4710c0a740 1 Processor -- start 2024-04-28T13:32:38.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.852+0000 7f4710c0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed3d55e0 0x55fbed403a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.852+0000 7f4710c0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed4000a0 0x55fbed3efac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.852+0000 7f4710c0a740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed411cb0 0x55fbed3fc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.852+0000 7f4710c0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fbed2c0f70 con 0x55fbed4000a0 2024-04-28T13:32:38.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.852+0000 7f4710c0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fbed3fdbf0 con 0x55fbed3d55e0 2024-04-28T13:32:38.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.853+0000 7f4710c0a740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55fbed403f50 con 0x55fbed411cb0 2024-04-28T13:32:38.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.853+0000 7f4710c0a740 10 client.0 _ll_register_callbacks cb 0x55fbed32f480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T13:32:38.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.853+0000 7f4706000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed4000a0 0x55fbed3efac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.853+0000 7f4706000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed4000a0 0x55fbed3efac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34392/0 (socket says 172.21.15.196:34392) 2024-04-28T13:32:38.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.853+0000 7f4706000640 1 -- 172.21.15.196:0/588240508 learned_addr learned my addr 172.21.15.196:0/588240508 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:38.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.853+0000 7f4706a00640 1 --2- 172.21.15.196:0/588240508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed3d55e0 0x55fbed403a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.853+0000 7f4706000640 1 -- 172.21.15.196:0/588240508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed411cb0 msgr2=0x55fbed3fc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:38.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.853+0000 7f4706000640 1 --2- 172.21.15.196:0/588240508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed411cb0 0x55fbed3fc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.853+0000 7f4706000640 1 -- 172.21.15.196:0/588240508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed3d55e0 msgr2=0x55fbed403a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:38.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.853+0000 7f4706000640 1 --2- 172.21.15.196:0/588240508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed3d55e0 0x55fbed403a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.853+0000 7f4706000640 1 -- 172.21.15.196:0/588240508 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbed1156b0 con 0x55fbed4000a0 2024-04-28T13:32:38.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.853+0000 7f4706a00640 1 --2- 172.21.15.196:0/588240508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed3d55e0 0x55fbed403a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:38.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f4706000640 1 --2- 172.21.15.196:0/588240508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed4000a0 0x55fbed3efac0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f46f40aa270 tx=0x7f46f407a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e14133117a051ba server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:38.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f46e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-04-28T13:32:38.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f46e7e00640 1 -- 172.21.15.196:0/588240508 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f40826e0 con 0x55fbed4000a0 2024-04-28T13:32:38.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f46e7e00640 1 -- 172.21.15.196:0/588240508 <== mon.0 v2:172.21.15.196:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f46f4092030 con 0x55fbed4000a0 2024-04-28T13:32:38.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f4710c0a740 1 -- 172.21.15.196:0/588240508 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46f801c030 con 0x55fbed4000a0 2024-04-28T13:32:38.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f46e7e00640 1 -- 172.21.15.196:0/588240508 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f46f4099050 con 0x55fbed4000a0 2024-04-28T13:32:38.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f4710c0a740 10 client.5217 fetch_fsmap learned FSMap version 16 2024-04-28T13:32:38.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f4710c0a740 1 -- 172.21.15.196:0/588240508 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55fbed4019b0 con 0x55fbed4000a0 2024-04-28T13:32:38.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f46e7e00640 1 -- 172.21.15.196:0/588240508 <== mon.0 v2:172.21.15.196:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f801c030 con 0x55fbed4000a0 2024-04-28T13:32:38.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f46e7e00640 1 -- 172.21.15.196:0/588240508 <== mon.0 v2:172.21.15.196:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f46f404e060 con 0x55fbed4000a0 2024-04-28T13:32:38.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f4710c0a740 10 client.5217 fetch_fsmap finished waiting for FSMap version 16 2024-04-28T13:32:38.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f4710c0a740 10 client.5217 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T13:32:38.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f4710c0a740 20 client.5217 populate_metadata read hostname 'smithi196' 2024-04-28T13:32:38.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f4710c0a740 1 --2- 172.21.15.196:0/588240508 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55fbed415690 0x55fbed4021b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f4710c0a740 4 client.5217 mds_command: new command op to 4735 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T13:32:38.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.854+0000 7f4710c0a740 1 -- 172.21.15.196:0/588240508 --> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbed347210 con 0x55fbed415690 2024-04-28T13:32:38.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.855+0000 7f4706a00640 1 --2- 172.21.15.196:0/588240508 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55fbed415690 0x55fbed4021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.856+0000 7f4706a00640 1 --2- 172.21.15.196:0/588240508 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55fbed415690 0x55fbed4021b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41fbd99bf328c3ba server_cookie=b475a5a1ddc68d20 in_seq=0 out_seq=0 2024-04-28T13:32:38.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.856+0000 7f46e7e00640 10 client.5217 ms_handle_connect on v2:172.21.15.203:6835/459640695 2024-04-28T13:32:38.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.858+0000 7f46e7e00640 1 -- 172.21.15.196:0/588240508 <== mds.0 v2:172.21.15.203:6835/459640695 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fbed347210 con 0x55fbed415690 2024-04-28T13:32:38.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.858+0000 7f46e7e00640 10 client.5217 handle_command_reply: tid=0 2024-04-28T13:32:38.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.859+0000 7f4710c0a740 1 client.5217 shutdown 2024-04-28T13:32:38.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.859+0000 7f4710c0a740 1 -- 172.21.15.196:0/588240508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed4000a0 msgr2=0x55fbed3efac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:38.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.859+0000 7f4710c0a740 1 --2- 172.21.15.196:0/588240508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed4000a0 0x55fbed3efac0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f46f40aa270 tx=0x7f46f407a200 comp rx=0 tx=0).stop 2024-04-28T13:32:38.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.859+0000 7f4710c0a740 1 -- 172.21.15.196:0/588240508 shutdown_connections 2024-04-28T13:32:38.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.859+0000 7f4710c0a740 1 --2- 172.21.15.196:0/588240508 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55fbed415690 0x55fbed4021b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.859+0000 7f4710c0a740 1 --2- 172.21.15.196:0/588240508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed411cb0 0x55fbed3fc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.859+0000 7f4710c0a740 1 --2- 172.21.15.196:0/588240508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed4000a0 0x55fbed3efac0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.859+0000 7f4710c0a740 1 --2- 172.21.15.196:0/588240508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed3d55e0 0x55fbed403a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.859+0000 7f4710c0a740 1 -- 172.21.15.196:0/588240508 >> 172.21.15.196:0/588240508 conn(0x55fbed369d10 msgr2=0x55fbed3fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:38.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.859+0000 7f46e7e00640 0 client.5217 ms_handle_reset on v2:172.21.15.203:6835/459640695 2024-04-28T13:32:38.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.859+0000 7f4710c0a740 1 -- 172.21.15.196:0/588240508 shutdown_connections 2024-04-28T13:32:38.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.859+0000 7f4710c0a740 1 -- 172.21.15.196:0/588240508 wait complete. 2024-04-28T13:32:38.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.859+0000 7f4710c0a740 20 client.5217 trim_cache size 0 max 16384 2024-04-28T13:32:38.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.879+0000 7f4710c0a740 1 Processor -- start 2024-04-28T13:32:38.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.879+0000 7f4710c0a740 1 -- start start 2024-04-28T13:32:38.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.880+0000 7f4710c0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed3d55e0 0x55fbed434810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.880+0000 7f4710c0a740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed4000a0 0x55fbed415030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.880+0000 7f4710c0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed411cb0 0x55fbed415570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.880+0000 7f4710c0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47081ff370 con 0x55fbed411cb0 2024-04-28T13:32:38.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.880+0000 7f4710c0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4708134c30 con 0x55fbed3d55e0 2024-04-28T13:32:38.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.880+0000 7f4710c0a740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f47081fed20 con 0x55fbed4000a0 2024-04-28T13:32:38.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.880+0000 7f4707400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed411cb0 0x55fbed415570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.880+0000 7f4707400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed411cb0 0x55fbed415570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34394/0 (socket says 172.21.15.196:34394) 2024-04-28T13:32:38.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.880+0000 7f4707400640 1 -- 172.21.15.196:0/3908215594 learned_addr learned my addr 172.21.15.196:0/3908215594 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:38.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.881+0000 7f4707400640 1 -- 172.21.15.196:0/3908215594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed4000a0 msgr2=0x55fbed415030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:38.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.881+0000 7f4707400640 1 --2- 172.21.15.196:0/3908215594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed4000a0 0x55fbed415030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.881+0000 7f4707400640 1 -- 172.21.15.196:0/3908215594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed3d55e0 msgr2=0x55fbed434810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:38.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.881+0000 7f4707400640 1 --2- 172.21.15.196:0/3908215594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed3d55e0 0x55fbed434810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.881+0000 7f4707400640 1 -- 172.21.15.196:0/3908215594 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbed3fc800 con 0x55fbed411cb0 2024-04-28T13:32:38.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.881+0000 7f4707400640 1 --2- 172.21.15.196:0/3908215594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed411cb0 0x55fbed415570 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4700001b00 tx=0x7f4700001fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4ea2adc9fa7210 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:38.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.881+0000 7f46f2a00640 1 -- 172.21.15.196:0/3908215594 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470000e7a0 con 0x55fbed411cb0 2024-04-28T13:32:38.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.881+0000 7f46f2a00640 1 -- 172.21.15.196:0/3908215594 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f470000e940 con 0x55fbed411cb0 2024-04-28T13:32:38.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.881+0000 7f46f2a00640 1 -- 172.21.15.196:0/3908215594 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470000ed30 con 0x55fbed411cb0 2024-04-28T13:32:38.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.882+0000 7f4710c0a740 1 -- 172.21.15.196:0/3908215594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed411cb0 msgr2=0x55fbed415570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:38.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.882+0000 7f4710c0a740 1 --2- 172.21.15.196:0/3908215594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed411cb0 0x55fbed415570 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4700001b00 tx=0x7f4700001fc0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.882+0000 7f4710c0a740 1 -- 172.21.15.196:0/3908215594 shutdown_connections 2024-04-28T13:32:38.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.882+0000 7f4710c0a740 1 --2- 172.21.15.196:0/3908215594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed411cb0 0x55fbed415570 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.882+0000 7f4710c0a740 1 --2- 172.21.15.196:0/3908215594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed4000a0 0x55fbed415030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.882+0000 7f4710c0a740 1 --2- 172.21.15.196:0/3908215594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed3d55e0 0x55fbed434810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.882+0000 7f4710c0a740 1 -- 172.21.15.196:0/3908215594 >> 172.21.15.196:0/3908215594 conn(0x55fbed369d10 msgr2=0x55fbed415fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:38.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.882+0000 7f4710c0a740 1 -- 172.21.15.196:0/3908215594 shutdown_connections 2024-04-28T13:32:38.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.882+0000 7f4710c0a740 1 -- 172.21.15.196:0/3908215594 wait complete. 2024-04-28T13:32:38.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.883+0000 7f4710c0a740 1 -- start start 2024-04-28T13:32:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.883+0000 7f4710c0a740 1 Processor -- start 2024-04-28T13:32:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4710c0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed3d55e0 0x55fbed3d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4710c0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed4000a0 0x55fbed3d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4710c0a740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed411cb0 0x55fbed3fb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4710c0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fbed2c0f70 con 0x55fbed4000a0 2024-04-28T13:32:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4710c0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fbed3fdbf0 con 0x55fbed3d55e0 2024-04-28T13:32:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4710c0a740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55fbed42e520 con 0x55fbed411cb0 2024-04-28T13:32:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4710c0a740 10 client.0 _ll_register_callbacks cb 0x7f46f4001d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T13:32:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4706000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed4000a0 0x55fbed3d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4706000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed4000a0 0x55fbed3d4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34406/0 (socket says 172.21.15.196:34406) 2024-04-28T13:32:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4706000640 1 -- 172.21.15.196:0/3818104249 learned_addr learned my addr 172.21.15.196:0/3818104249 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:38.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4707400640 1 --2- 172.21.15.196:0/3818104249 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed411cb0 0x55fbed3fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4706000640 1 -- 172.21.15.196:0/3818104249 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed411cb0 msgr2=0x55fbed3fb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:38.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4706a00640 1 --2- 172.21.15.196:0/3818104249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed3d55e0 0x55fbed3d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4706000640 1 --2- 172.21.15.196:0/3818104249 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed411cb0 0x55fbed3fb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4706000640 1 -- 172.21.15.196:0/3818104249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed3d55e0 msgr2=0x55fbed3d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:38.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4706000640 1 --2- 172.21.15.196:0/3818104249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed3d55e0 0x55fbed3d3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.884+0000 7f4706000640 1 -- 172.21.15.196:0/3818104249 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbed402c60 con 0x55fbed4000a0 2024-04-28T13:32:38.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.885+0000 7f4706000640 1 --2- 172.21.15.196:0/3818104249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed4000a0 0x55fbed3d4150 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f46f4079cc0 tx=0x7f46f40aa1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4a6ef36e4c02c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:38.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.885+0000 7f46e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-04-28T13:32:38.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.885+0000 7f46e7e00640 1 -- 172.21.15.196:0/3818104249 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4079640 con 0x55fbed4000a0 2024-04-28T13:32:38.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.885+0000 7f46e7e00640 1 -- 172.21.15.196:0/3818104249 <== mon.0 v2:172.21.15.196:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f46f4092030 con 0x55fbed4000a0 2024-04-28T13:32:38.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.885+0000 7f4710c0a740 1 -- 172.21.15.196:0/3818104249 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f470000e940 con 0x55fbed4000a0 2024-04-28T13:32:38.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.885+0000 7f46e7e00640 1 -- 172.21.15.196:0/3818104249 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f46f4075da0 con 0x55fbed4000a0 2024-04-28T13:32:38.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.885+0000 7f4710c0a740 10 client.5223 fetch_fsmap learned FSMap version 16 2024-04-28T13:32:38.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.885+0000 7f4710c0a740 1 -- 172.21.15.196:0/3818104249 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55fbed402c60 con 0x55fbed4000a0 2024-04-28T13:32:38.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.885+0000 7f46e7e00640 1 -- 172.21.15.196:0/3818104249 <== mon.0 v2:172.21.15.196:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f470000e940 con 0x55fbed4000a0 2024-04-28T13:32:38.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.885+0000 7f46e7e00640 1 -- 172.21.15.196:0/3818104249 <== mon.0 v2:172.21.15.196:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f46f4007430 con 0x55fbed4000a0 2024-04-28T13:32:38.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.885+0000 7f4710c0a740 10 client.5223 fetch_fsmap finished waiting for FSMap version 16 2024-04-28T13:32:38.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.885+0000 7f4710c0a740 10 client.5223 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T13:32:38.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.885+0000 7f4710c0a740 20 client.5223 populate_metadata read hostname 'smithi196' 2024-04-28T13:32:38.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.886+0000 7f4710c0a740 1 --2- 172.21.15.196:0/3818104249 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55fbed3fbb20 0x55fbed42cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:38.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.886+0000 7f4710c0a740 4 client.5223 mds_command: new command op to 4735 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T13:32:38.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.886+0000 7f4710c0a740 1 -- 172.21.15.196:0/3818104249 --> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] -- command(tid 0: {"prefix": "status"}) -- 0x55fbed416bf0 con 0x55fbed3fbb20 2024-04-28T13:32:38.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.886+0000 7f4706a00640 1 --2- 172.21.15.196:0/3818104249 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55fbed3fbb20 0x55fbed42cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:38.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.887+0000 7f4706a00640 1 --2- 172.21.15.196:0/3818104249 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55fbed3fbb20 0x55fbed42cfd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ac1051821677017 server_cookie=ea1260069076d79d in_seq=0 out_seq=0 2024-04-28T13:32:38.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.887+0000 7f46e7e00640 10 client.5223 ms_handle_connect on v2:172.21.15.203:6835/459640695 2024-04-28T13:32:38.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.887+0000 7f46e7e00640 1 -- 172.21.15.196:0/3818104249 <== mds.0 v2:172.21.15.203:6835/459640695 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55fbed416bf0 con 0x55fbed3fbb20 2024-04-28T13:32:38.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.887+0000 7f46e7e00640 10 client.5223 handle_command_reply: tid=0 2024-04-28T13:32:38.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.888+0000 7f4710c0a740 1 client.5223 shutdown 2024-04-28T13:32:38.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.888+0000 7f4710c0a740 1 -- 172.21.15.196:0/3818104249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed4000a0 msgr2=0x55fbed3d4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:38.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.888+0000 7f4710c0a740 1 --2- 172.21.15.196:0/3818104249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed4000a0 0x55fbed3d4150 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f46f4079cc0 tx=0x7f46f40aa1d0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.888+0000 7f4710c0a740 1 -- 172.21.15.196:0/3818104249 shutdown_connections 2024-04-28T13:32:38.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.888+0000 7f4710c0a740 1 --2- 172.21.15.196:0/3818104249 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55fbed3fbb20 0x55fbed42cfd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.888+0000 7f4710c0a740 1 --2- 172.21.15.196:0/3818104249 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fbed411cb0 0x55fbed3fb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.888+0000 7f46e7e00640 0 client.5223 ms_handle_reset on v2:172.21.15.203:6835/459640695 2024-04-28T13:32:38.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.888+0000 7f4710c0a740 1 --2- 172.21.15.196:0/3818104249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fbed4000a0 0x55fbed3d4150 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.888+0000 7f4710c0a740 1 --2- 172.21.15.196:0/3818104249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fbed3d55e0 0x55fbed3d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.888+0000 7f4710c0a740 1 -- 172.21.15.196:0/3818104249 >> 172.21.15.196:0/3818104249 conn(0x55fbed369d10 msgr2=0x55fbed400a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:38.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.888+0000 7f4710c0a740 1 -- 172.21.15.196:0/3818104249 shutdown_connections 2024-04-28T13:32:38.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.888+0000 7f4710c0a740 1 -- 172.21.15.196:0/3818104249 wait complete. 2024-04-28T13:32:38.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.888+0000 7f4710c0a740 20 client.5223 trim_cache size 0 max 16384 2024-04-28T13:32:38.892 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:32:38.892 INFO:teuthology.orchestra.run.smithi196.stdout: "cluster_fsid": "270aea32-2e16-4a97-85f7-aeae39bc8c76", 2024-04-28T13:32:38.892 INFO:teuthology.orchestra.run.smithi196.stdout: "whoami": 0, 2024-04-28T13:32:38.892 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 4735, 2024-04-28T13:32:38.893 INFO:teuthology.orchestra.run.smithi196.stdout: "want_state": "up:active", 2024-04-28T13:32:38.893 INFO:teuthology.orchestra.run.smithi196.stdout: "state": "up:active", 2024-04-28T13:32:38.893 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs", 2024-04-28T13:32:38.893 INFO:teuthology.orchestra.run.smithi196.stdout: "rank_uptime": 6.6628125569999996, 2024-04-28T13:32:38.893 INFO:teuthology.orchestra.run.smithi196.stdout: "mdsmap_epoch": 16, 2024-04-28T13:32:38.893 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_epoch": 33, 2024-04-28T13:32:38.893 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_epoch_barrier": 33, 2024-04-28T13:32:38.893 INFO:teuthology.orchestra.run.smithi196.stdout: "uptime": 15.543719982000001 2024-04-28T13:32:38.893 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:32:38.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.889+0000 7f470de00640 1 -- 172.21.15.196:0/478602967 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f46e806c920 msgr2=0x7f46e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:38.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.889+0000 7f470de00640 1 --2- 172.21.15.196:0/478602967 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f46e806c920 0x7f46e806ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009ab0 tx=0x7f46f8008040 comp rx=0 tx=0).stop 2024-04-28T13:32:38.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.889+0000 7f470de00640 1 -- 172.21.15.196:0/478602967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47080aa9b0 msgr2=0x7f47081353d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:38.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.889+0000 7f470de00640 1 --2- 172.21.15.196:0/478602967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47080aa9b0 0x7f47081353d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f46f400c1a0 tx=0x7f47081613f0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.889+0000 7f470de00640 1 -- 172.21.15.196:0/478602967 shutdown_connections 2024-04-28T13:32:38.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.889+0000 7f470de00640 1 --2- 172.21.15.196:0/478602967 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f46e806c920 0x7f46e806ede0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.889+0000 7f470de00640 1 --2- 172.21.15.196:0/478602967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4708135910 0x7f47081f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.889+0000 7f470de00640 1 --2- 172.21.15.196:0/478602967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47080aa9b0 0x7f47081353d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.889+0000 7f470de00640 1 --2- 172.21.15.196:0/478602967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f470815c8a0 0x7f470815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:38.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.889+0000 7f470de00640 1 -- 172.21.15.196:0/478602967 >> 172.21.15.196:0/478602967 conn(0x7f47080cd830 msgr2=0x7f47080b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:38.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.889+0000 7f470de00640 1 -- 172.21.15.196:0/478602967 shutdown_connections 2024-04-28T13:32:38.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:38.889+0000 7f470de00640 1 -- 172.21.15.196:0/478602967 wait complete. 2024-04-28T13:32:38.906 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T13:32:39.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.103+0000 7f900b600640 1 Processor -- start 2024-04-28T13:32:39.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.104+0000 7f900b600640 1 -- start start 2024-04-28T13:32:39.105 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.105+0000 7f900b600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f90040d3ee0 0x7f90040cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:39.105 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.105+0000 7f900b600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90040d03a0 con 0x7f900415f750 2024-04-28T13:32:39.105 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.105+0000 7f900b600640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90040d0680 con 0x7f900415f370 2024-04-28T13:32:39.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.105+0000 7f900b600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f90040d0960 con 0x7f90040d3ee0 2024-04-28T13:32:39.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.105+0000 7f9003e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f90040d3ee0 0x7f90040cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:39.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.105+0000 7f9003e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f90040d3ee0 0x7f90040cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34412/0 (socket says 172.21.15.196:34412) 2024-04-28T13:32:39.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.105+0000 7f9003e00640 1 -- 172.21.15.196:0/2431232223 learned_addr learned my addr 172.21.15.196:0/2431232223 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:39.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.105+0000 7f9009600640 1 --1- 172.21.15.196:0/2431232223 >> v1:172.21.15.196:6789/0 conn(0x7f900415f750 0x7f90040b51c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:48610/0 (socket says 172.21.15.196:48610) 2024-04-28T13:32:39.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.106+0000 7f9003e00640 1 -- 172.21.15.196:0/2431232223 >> v1:172.21.15.203:6789/0 conn(0x7f900415f370 legacy=0x7f90040d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:39.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.106+0000 7f9003e00640 1 -- 172.21.15.196:0/2431232223 >> v1:172.21.15.196:6789/0 conn(0x7f900415f750 legacy=0x7f90040b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:39.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.106+0000 7f9003e00640 1 -- 172.21.15.196:0/2431232223 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90040d0c40 con 0x7f90040d3ee0 2024-04-28T13:32:39.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.106+0000 7f9003400640 1 -- 172.21.15.196:0/2431232223 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2826301706 0 0) 0x7f90040d03a0 con 0x7f900415f750 2024-04-28T13:32:39.107 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.107+0000 7f9003e00640 1 --2- 172.21.15.196:0/2431232223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90040d3ee0 0x7f90040cfd70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0098a0 tx=0x7f8fec02a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb62c78836c3f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:39.107 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.107+0000 7f9003400640 1 -- 172.21.15.196:0/2431232223 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec037070 con 0x7f90040d3ee0 2024-04-28T13:32:39.107 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.107+0000 7f9003400640 1 -- 172.21.15.196:0/2431232223 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fec032440 con 0x7f90040d3ee0 2024-04-28T13:32:39.107 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.107+0000 7f9003400640 1 -- 172.21.15.196:0/2431232223 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec032810 con 0x7f90040d3ee0 2024-04-28T13:32:39.108 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.108+0000 7f900b600640 1 -- 172.21.15.196:0/2431232223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90040d3ee0 msgr2=0x7f90040cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:39.108 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.108+0000 7f900b600640 1 --2- 172.21.15.196:0/2431232223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90040d3ee0 0x7f90040cfd70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0098a0 tx=0x7f8fec02a2e0 comp rx=0 tx=0).stop 2024-04-28T13:32:39.108 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.108+0000 7f900b600640 1 -- 172.21.15.196:0/2431232223 shutdown_connections 2024-04-28T13:32:39.108 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.108+0000 7f900b600640 1 --2- 172.21.15.196:0/2431232223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90040d3ee0 0x7f90040cfd70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:39.108 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.108+0000 7f900b600640 1 -- 172.21.15.196:0/2431232223 >> 172.21.15.196:0/2431232223 conn(0x7f900408ec10 msgr2=0x7f900409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:39.108 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.108+0000 7f900b600640 1 -- 172.21.15.196:0/2431232223 shutdown_connections 2024-04-28T13:32:39.108 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.108+0000 7f900b600640 1 -- 172.21.15.196:0/2431232223 wait complete. 2024-04-28T13:32:39.109 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.110+0000 7f900b600640 1 Processor -- start 2024-04-28T13:32:39.109 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.110+0000 7f900b600640 1 -- start start 2024-04-28T13:32:39.112 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f900b600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90040d3ee0 0x7f90040cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:39.112 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f900b600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900415f370 0x7f90040c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:39.112 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f900b600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f900415f750 0x7f90040c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:39.112 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f900b600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f90040b4950 con 0x7f900415f750 2024-04-28T13:32:39.112 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f900b600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f90040b47d0 con 0x7f90040d3ee0 2024-04-28T13:32:39.112 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f900b600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f90040b4ad0 con 0x7f900415f370 2024-04-28T13:32:39.112 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f9009600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f900415f750 0x7f90040c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:39.112 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f9009600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f900415f750 0x7f90040c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34418/0 (socket says 172.21.15.196:34418) 2024-04-28T13:32:39.112 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f9009600640 1 -- 172.21.15.196:0/1839925433 learned_addr learned my addr 172.21.15.196:0/1839925433 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:39.112 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f9008c00640 1 --2- 172.21.15.196:0/1839925433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90040d3ee0 0x7f90040cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:39.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f9009600640 1 -- 172.21.15.196:0/1839925433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900415f370 msgr2=0x7f90040c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:39.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f9003e00640 1 --2- 172.21.15.196:0/1839925433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900415f370 0x7f90040c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:39.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f9009600640 1 --2- 172.21.15.196:0/1839925433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900415f370 0x7f90040c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:39.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f9009600640 1 -- 172.21.15.196:0/1839925433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90040d3ee0 msgr2=0x7f90040cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:39.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f9009600640 1 --2- 172.21.15.196:0/1839925433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90040d3ee0 0x7f90040cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:39.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f9009600640 1 -- 172.21.15.196:0/1839925433 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90040cecb0 con 0x7f900415f750 2024-04-28T13:32:39.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.111+0000 7f9008c00640 1 --2- 172.21.15.196:0/1839925433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90040d3ee0 0x7f90040cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:39.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.112+0000 7f9009600640 1 --2- 172.21.15.196:0/1839925433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f900415f750 0x7f90040c9230 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00c700 tx=0x7f8ffc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:39.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.112+0000 7f9000c00640 1 -- 172.21.15.196:0/1839925433 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc018070 con 0x7f900415f750 2024-04-28T13:32:39.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.112+0000 7f9000c00640 1 -- 172.21.15.196:0/1839925433 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ffc00ce10 con 0x7f900415f750 2024-04-28T13:32:39.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.112+0000 7f9000c00640 1 -- 172.21.15.196:0/1839925433 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00a860 con 0x7f900415f750 2024-04-28T13:32:39.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.112+0000 7f900b600640 1 -- 172.21.15.196:0/1839925433 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90040cdc10 con 0x7f900415f750 2024-04-28T13:32:39.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.112+0000 7f900b600640 1 -- 172.21.15.196:0/1839925433 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90040ce1d0 con 0x7f900415f750 2024-04-28T13:32:39.114 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.113+0000 7f900b600640 1 -- 172.21.15.196:0/1839925433 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900406a290 con 0x7f900415f750 2024-04-28T13:32:39.114 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.114+0000 7f9000c00640 1 -- 172.21.15.196:0/1839925433 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ffc00aa00 con 0x7f900415f750 2024-04-28T13:32:39.115 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.115+0000 7f9000c00640 1 --2- 172.21.15.196:0/1839925433 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8fe006c8f0 0x7f8fe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:39.115 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.115+0000 7f9000c00640 1 -- 172.21.15.196:0/1839925433 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8ffc08b1f0 con 0x7f900415f750 2024-04-28T13:32:39.115 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.116+0000 7f9008c00640 1 --2- 172.21.15.196:0/1839925433 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8fe006c8f0 0x7f8fe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:39.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.117+0000 7f9008c00640 1 --2- 172.21.15.196:0/1839925433 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8fe006c8f0 0x7f8fe006edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4003eb0 tx=0x7f8ff4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:39.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.120+0000 7f9000c00640 1 -- 172.21.15.196:0/1839925433 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8ffc059580 con 0x7f900415f750 2024-04-28T13:32:39.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.363+0000 7f900b600640 1 -- 172.21.15.196:0/1839925433 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f90040b6700 con 0x7f900415f750 2024-04-28T13:32:39.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.364+0000 7f9000c00640 1 -- 172.21.15.196:0/1839925433 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f8ffc058e80 con 0x7f900415f750 2024-04-28T13:32:39.364 INFO:teuthology.orchestra.run.smithi196.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-28T13:32:39.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.367+0000 7f900b600640 1 -- 172.21.15.196:0/1839925433 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8fe006c8f0 msgr2=0x7f8fe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:39.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.367+0000 7f900b600640 1 --2- 172.21.15.196:0/1839925433 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8fe006c8f0 0x7f8fe006edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4003eb0 tx=0x7f8ff4009040 comp rx=0 tx=0).stop 2024-04-28T13:32:39.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.367+0000 7f900b600640 1 -- 172.21.15.196:0/1839925433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f900415f750 msgr2=0x7f90040c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:39.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.367+0000 7f900b600640 1 --2- 172.21.15.196:0/1839925433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f900415f750 0x7f90040c9230 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00c700 tx=0x7f8ffc00cbc0 comp rx=0 tx=0).stop 2024-04-28T13:32:39.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.367+0000 7f900b600640 1 -- 172.21.15.196:0/1839925433 shutdown_connections 2024-04-28T13:32:39.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.367+0000 7f900b600640 1 --2- 172.21.15.196:0/1839925433 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8fe006c8f0 0x7f8fe006edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:39.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.367+0000 7f900b600640 1 --2- 172.21.15.196:0/1839925433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f900415f750 0x7f90040c9230 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:39.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.367+0000 7f900b600640 1 --2- 172.21.15.196:0/1839925433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900415f370 0x7f90040c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:39.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.367+0000 7f900b600640 1 --2- 172.21.15.196:0/1839925433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90040d3ee0 0x7f90040cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:39.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.367+0000 7f900b600640 1 -- 172.21.15.196:0/1839925433 >> 172.21.15.196:0/1839925433 conn(0x7f900408ec10 msgr2=0x7f900409e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:39.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.367+0000 7f900b600640 1 -- 172.21.15.196:0/1839925433 shutdown_connections 2024-04-28T13:32:39.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39.367+0000 7f900b600640 1 -- 172.21.15.196:0/1839925433 wait complete. 2024-04-28T13:32:39.385 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-28T13:32:39.385 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T13:32:39.385 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-04-28T13:32:39.385 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-28T13:32:39.385 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-28T13:32:39.386 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-28T13:32:39.386 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-28T13:32:39.386 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T13:32:39.386 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-28T13:32:39.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:39.386 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-04-28T13:32:39.438 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:32:39.438 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:32:39.438 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:32:39.438 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:39.438 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-28T13:32:39.438 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:39.438 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:39.438 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42591sec preferred_lft 42591sec 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 1e:4d:24:54:f5:c3 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-28T13:32:39.439 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:39.440 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::3cb7:a8ff:fe55:18c0/64 scope link 2024-04-28T13:32:39.440 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:39.440 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:32:39.440 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether c6:b9:8c:18:7a:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T13:32:39.440 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::c4b9:8cff:fe18:7aec/64 scope link 2024-04-28T13:32:39.440 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:39.440 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:32:39.440 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 1e:4d:24:54:f5:c3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T13:32:39.440 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::1c4d:24ff:fe54:f5c3/64 scope link 2024-04-28T13:32:39.440 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:39.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:39.441 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T13:32:39.516 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-04-28T13:32:39.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:39.521 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-04-28T13:32:39.570 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-04-28T13:32:39.570 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T13:32:39.570 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T13:32:39.570 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T13:32:39.570 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T13:32:39.571 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:32:39.571 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-28T13:32:39.571 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T13:32:39.571 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T13:32:39.571 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T13:32:39.571 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-28T13:32:39.671 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:32:39.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:32:39.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:39.777 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-28T13:32:39.818 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:32:39.827 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:32:39.828 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-28T13:32:39.828 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-28T13:32:39.829 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-28T13:32:39.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:39.829 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-04-28T13:32:39.937 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:32:39.937 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-28T13:32:40.034 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-28T13:32:40.034 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:32:40.035 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-28T13:32:40.085 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-28T13:32:40.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:40.085 DEBUG:teuthology.orchestra.run.smithi196:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:32:40.216 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:32:40.216 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-28T13:32:40.216 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:32:40.242 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:32:40.242 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-28T13:32:40.242 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:32:40.242 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:32:40.242 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:32:40.271 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:32:40.271 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-28T13:32:40.271 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:32:40.271 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:32:40.271 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:32:40.271 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-28T13:32:40.271 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-28T13:32:40.271 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=270aea32-2e16-4a97-85f7-aeae39bc8c76" will pass to kernel 2024-04-28T13:32:40.273 INFO:teuthology.orchestra.run.smithi196.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-28T13:32:40.273 INFO:teuthology.orchestra.run.smithi196.stdout: You just mounted a file system that supports labels which does not 2024-04-28T13:32:40.273 INFO:teuthology.orchestra.run.smithi196.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-28T13:32:40.273 INFO:teuthology.orchestra.run.smithi196.stdout: applications will generate AVC messages and not be allowed access to 2024-04-28T13:32:40.273 INFO:teuthology.orchestra.run.smithi196.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-28T13:32:40.278 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-28T13:32:40.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:40.279 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-28T13:32:40.350 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:32:40.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:32:40.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:32:40.488 INFO:teuthology.orchestra.run.smithi196.stdout:client4828 2024-04-28T13:32:40.488 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:32:40.488 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:40.537 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T13:32:40.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.786+0000 7f2505400640 1 Processor -- start 2024-04-28T13:32:40.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.786+0000 7f2505400640 1 -- start start 2024-04-28T13:32:40.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.788+0000 7f2505400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2500140350 0x7f25000cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:40.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.788+0000 7f2505400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25000d05b0 con 0x7f2500146790 2024-04-28T13:32:40.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.788+0000 7f2505400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25000d0890 con 0x7f250013ff70 2024-04-28T13:32:40.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.788+0000 7f2505400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f250006e450 con 0x7f2500140350 2024-04-28T13:32:40.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.788+0000 7f24ff200640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f2500146790 0x7f25000cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:48622/0 (socket says 172.21.15.196:48622) 2024-04-28T13:32:40.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.788+0000 7f24ff200640 1 -- 172.21.15.196:0/2993824637 learned_addr learned my addr 172.21.15.196:0/2993824637 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:40.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.788+0000 7f24f7e00640 1 --2- 172.21.15.196:0/2993824637 >> v2:172.21.15.203:3300/0 conn(0x7f2500140350 0x7f25000cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:40.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.788+0000 7f24fde00640 1 -- 172.21.15.196:0/2993824637 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3100167594 0 0) 0x7f25000d05b0 con 0x7f2500146790 2024-04-28T13:32:40.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.789+0000 7f24fde00640 1 -- 172.21.15.196:0/2993824637 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24e4003520 con 0x7f2500146790 2024-04-28T13:32:40.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.789+0000 7f24fde00640 1 -- 172.21.15.196:0/2993824637 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3327309775 0 0) 0x7f24e4003520 con 0x7f2500146790 2024-04-28T13:32:40.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.789+0000 7f24fde00640 1 -- 172.21.15.196:0/2993824637 >> v2:172.21.15.203:3300/0 conn(0x7f2500140350 msgr2=0x7f25000cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:40.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.789+0000 7f24fde00640 1 --2- 172.21.15.196:0/2993824637 >> v2:172.21.15.203:3300/0 conn(0x7f2500140350 0x7f25000cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:40.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.789+0000 7f24fde00640 1 -- 172.21.15.196:0/2993824637 >> v1:172.21.15.203:6789/0 conn(0x7f250013ff70 legacy=0x7f2500146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.789+0000 7f24fde00640 1 -- 172.21.15.196:0/2993824637 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250006e730 con 0x7f2500146790 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.789+0000 7f24fde00640 1 -- 172.21.15.196:0/2993824637 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f24f80019f0 con 0x7f2500146790 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.789+0000 7f24fde00640 1 -- 172.21.15.196:0/2993824637 >> v1:172.21.15.196:6789/0 conn(0x7f2500146790 legacy=0x7f25000cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.789+0000 7f24fde00640 1 --2- 172.21.15.196:0/2993824637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24e40039b0 0x7f24e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.789+0000 7f24fde00640 1 --2- 172.21.15.196:0/2993824637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24e4004800 0x7f24e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.789+0000 7f24fde00640 1 --2- 172.21.15.196:0/2993824637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24e40051c0 0x7f24e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.789+0000 7f24fde00640 1 -- 172.21.15.196:0/2993824637 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f250006e450 con 0x7f24e40039b0 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.789+0000 7f24fde00640 1 -- 172.21.15.196:0/2993824637 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f24e400c160 con 0x7f24e40051c0 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.789+0000 7f24fde00640 1 -- 172.21.15.196:0/2993824637 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f24e400c360 con 0x7f24e4004800 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.789+0000 7f24fe800640 1 --2- 172.21.15.196:0/2993824637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24e40039b0 0x7f24e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.790+0000 7f24ff200640 1 --2- 172.21.15.196:0/2993824637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24e40051c0 0x7f24e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.790+0000 7f24fe800640 1 -- 172.21.15.196:0/2993824637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24e4004800 msgr2=0x7f24e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.790+0000 7f24fe800640 1 --2- 172.21.15.196:0/2993824637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24e4004800 0x7f24e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.790+0000 7f24fe800640 1 -- 172.21.15.196:0/2993824637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24e40051c0 msgr2=0x7f24e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.790+0000 7f24fe800640 1 --2- 172.21.15.196:0/2993824637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24e40051c0 0x7f24e400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:40.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.790+0000 7f24fe800640 1 -- 172.21.15.196:0/2993824637 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e4003520 con 0x7f24e40039b0 2024-04-28T13:32:40.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.790+0000 7f24ff200640 1 -- 172.21.15.196:0/2993824637 reap_dead start 2024-04-28T13:32:40.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.791+0000 7f24fe800640 1 --2- 172.21.15.196:0/2993824637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24e40039b0 0x7f24e4003dd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f24f0001750 tx=0x7f24e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98eecb966737456b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:40.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.791+0000 7f24fde00640 1 -- 172.21.15.196:0/2993824637 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0001370 con 0x7f24e40039b0 2024-04-28T13:32:40.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.791+0000 7f24fde00640 1 -- 172.21.15.196:0/2993824637 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f00052d0 con 0x7f24e40039b0 2024-04-28T13:32:40.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.791+0000 7f24fde00640 1 -- 172.21.15.196:0/2993824637 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f002abc0 con 0x7f24e40039b0 2024-04-28T13:32:40.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.792+0000 7f2505400640 1 -- 172.21.15.196:0/2993824637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24e40039b0 msgr2=0x7f24e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:40.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.792+0000 7f2505400640 1 --2- 172.21.15.196:0/2993824637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24e40039b0 0x7f24e4003dd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f24f0001750 tx=0x7f24e4005830 comp rx=0 tx=0).stop 2024-04-28T13:32:40.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.792+0000 7f2505400640 1 -- 172.21.15.196:0/2993824637 shutdown_connections 2024-04-28T13:32:40.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.792+0000 7f2505400640 1 --2- 172.21.15.196:0/2993824637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24e40039b0 0x7f24e4003dd0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:40.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.792+0000 7f2505400640 1 -- 172.21.15.196:0/2993824637 >> 172.21.15.196:0/2993824637 conn(0x7f25000d0d30 msgr2=0x7f25000d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:40.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.792+0000 7f2505400640 1 -- 172.21.15.196:0/2993824637 shutdown_connections 2024-04-28T13:32:40.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.792+0000 7f2505400640 1 -- 172.21.15.196:0/2993824637 wait complete. 2024-04-28T13:32:40.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.794+0000 7f2505400640 1 Processor -- start 2024-04-28T13:32:40.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.794+0000 7f2505400640 1 -- start start 2024-04-28T13:32:40.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f2505400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24e40039b0 0x7f250013b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:40.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f2505400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f250013d5c0 0x7f250013eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:40.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f2505400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f250015adb0 0x7f25000b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:40.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f2505400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2500146410 con 0x7f250013d5c0 2024-04-28T13:32:40.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f2505400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2500146290 con 0x7f250015adb0 2024-04-28T13:32:40.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f2505400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2500146590 con 0x7f24e40039b0 2024-04-28T13:32:40.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f24f7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f250013d5c0 0x7f250013eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:40.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f24f7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f250013d5c0 0x7f250013eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34434/0 (socket says 172.21.15.196:34434) 2024-04-28T13:32:40.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f24f7e00640 1 -- 172.21.15.196:0/3988604891 learned_addr learned my addr 172.21.15.196:0/3988604891 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:40.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f24fe800640 1 --2- 172.21.15.196:0/3988604891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24e40039b0 0x7f250013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:40.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f24ff200640 1 --2- 172.21.15.196:0/3988604891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f250015adb0 0x7f25000b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:40.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f24f7e00640 1 -- 172.21.15.196:0/3988604891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24e40039b0 msgr2=0x7f250013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:40.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f24f7e00640 1 --2- 172.21.15.196:0/3988604891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24e40039b0 0x7f250013b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:40.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f24f7e00640 1 -- 172.21.15.196:0/3988604891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f250015adb0 msgr2=0x7f25000b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:40.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f24f7e00640 1 --2- 172.21.15.196:0/3988604891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f250015adb0 0x7f25000b5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:40.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.795+0000 7f24f7e00640 1 -- 172.21.15.196:0/3988604891 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2500159d10 con 0x7f250013d5c0 2024-04-28T13:32:40.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.796+0000 7f24f7e00640 1 --2- 172.21.15.196:0/3988604891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f250013d5c0 0x7f250013eca0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f24e800e770 tx=0x7f24e800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:40.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.796+0000 7f24f6a00640 1 -- 172.21.15.196:0/3988604891 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8017070 con 0x7f250013d5c0 2024-04-28T13:32:40.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.796+0000 7f24f6a00640 1 -- 172.21.15.196:0/3988604891 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e800c730 con 0x7f250013d5c0 2024-04-28T13:32:40.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.796+0000 7f24f6a00640 1 -- 172.21.15.196:0/3988604891 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e800cb20 con 0x7f250013d5c0 2024-04-28T13:32:40.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.796+0000 7f2505400640 1 -- 172.21.15.196:0/3988604891 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2500159ff0 con 0x7f250013d5c0 2024-04-28T13:32:40.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.796+0000 7f2505400640 1 -- 172.21.15.196:0/3988604891 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f250015a5b0 con 0x7f250013d5c0 2024-04-28T13:32:40.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.797+0000 7f2505400640 1 -- 172.21.15.196:0/3988604891 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f250006a290 con 0x7f250013d5c0 2024-04-28T13:32:40.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.798+0000 7f24f6a00640 1 -- 172.21.15.196:0/3988604891 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24e801a070 con 0x7f250013d5c0 2024-04-28T13:32:40.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.799+0000 7f24f6a00640 1 --2- 172.21.15.196:0/3988604891 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f24dc06c920 0x7f24dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:40.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.799+0000 7f24f6a00640 1 -- 172.21.15.196:0/3988604891 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f24e808beb0 con 0x7f250013d5c0 2024-04-28T13:32:40.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.800+0000 7f24fe800640 1 --2- 172.21.15.196:0/3988604891 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f24dc06c920 0x7f24dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:40.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.801+0000 7f24fe800640 1 --2- 172.21.15.196:0/3988604891 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f24dc06c920 0x7f24dc06ede0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f24f0032000 tx=0x7f24f002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:40.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:40.804+0000 7f24f6a00640 1 -- 172.21.15.196:0/3988604891 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f24e805a240 con 0x7f250013d5c0 2024-04-28T13:32:41.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:41.048+0000 7f2505400640 1 -- 172.21.15.196:0/3988604891 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f25000b45e0 con 0x7f250013d5c0 2024-04-28T13:32:41.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:41.049+0000 7f24f6a00640 1 -- 172.21.15.196:0/3988604891 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f24e8059b40 con 0x7f250013d5c0 2024-04-28T13:32:41.049 INFO:teuthology.orchestra.run.smithi196.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-28T13:32:41.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:41.051+0000 7f2505400640 1 -- 172.21.15.196:0/3988604891 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f24dc06c920 msgr2=0x7f24dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:41.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:41.051+0000 7f2505400640 1 --2- 172.21.15.196:0/3988604891 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f24dc06c920 0x7f24dc06ede0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f24f0032000 tx=0x7f24f002e000 comp rx=0 tx=0).stop 2024-04-28T13:32:41.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:41.051+0000 7f2505400640 1 -- 172.21.15.196:0/3988604891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f250013d5c0 msgr2=0x7f250013eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:41.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:41.051+0000 7f2505400640 1 --2- 172.21.15.196:0/3988604891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f250013d5c0 0x7f250013eca0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f24e800e770 tx=0x7f24e800ec30 comp rx=0 tx=0).stop 2024-04-28T13:32:41.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:41.052+0000 7f2505400640 1 -- 172.21.15.196:0/3988604891 shutdown_connections 2024-04-28T13:32:41.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:41.052+0000 7f2505400640 1 --2- 172.21.15.196:0/3988604891 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f24dc06c920 0x7f24dc06ede0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:41.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:41.052+0000 7f2505400640 1 --2- 172.21.15.196:0/3988604891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f250015adb0 0x7f25000b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:41.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:41.052+0000 7f2505400640 1 --2- 172.21.15.196:0/3988604891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f250013d5c0 0x7f250013eca0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:41.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:41.052+0000 7f2505400640 1 --2- 172.21.15.196:0/3988604891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24e40039b0 0x7f250013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:41.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:41.052+0000 7f2505400640 1 -- 172.21.15.196:0/3988604891 >> 172.21.15.196:0/3988604891 conn(0x7f25000d0d30 msgr2=0x7f25001412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:41.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:41.052+0000 7f2505400640 1 -- 172.21.15.196:0/3988604891 shutdown_connections 2024-04-28T13:32:41.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:41.052+0000 7f2505400640 1 -- 172.21.15.196:0/3988604891 wait complete. 2024-04-28T13:32:41.072 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-28T13:32:41.072 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T13:32:41.072 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2024-04-28T13:32:41.072 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-28T13:32:41.072 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-28T13:32:41.072 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-28T13:32:41.072 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-28T13:32:41.072 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T13:32:41.072 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-28T13:32:41.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:41.073 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2024-04-28T13:32:41.130 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:32:41.130 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:32:41.130 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:32:41.130 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:41.130 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-28T13:32:41.130 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:41.130 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:41.130 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:41.130 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f0 2024-04-28T13:32:41.131 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:41.131 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:41.131 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f1 2024-04-28T13:32:41.131 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:41.131 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:41.131 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T13:32:41.131 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:41.131 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T13:32:41.131 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42558sec preferred_lft 42558sec 2024-04-28T13:32:41.131 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ec4:7aff:fe8f:4a23/64 scope link 2024-04-28T13:32:41.131 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:41.131 INFO:teuthology.orchestra.run.smithi203.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T13:32:41.131 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 1e:04:67:22:4e:82 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:41.131 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-28T13:32:41.132 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:41.132 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::4e3:c8ff:fea0:8dce/64 scope link 2024-04-28T13:32:41.132 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:41.132 INFO:teuthology.orchestra.run.smithi203.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:32:41.132 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 1e:04:67:22:4e:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T13:32:41.132 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::1c04:67ff:fe22:4e82/64 scope link 2024-04-28T13:32:41.132 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:41.132 INFO:teuthology.orchestra.run.smithi203.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:32:41.132 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ae:7c:61:1f:29:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-28T13:32:41.132 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ac7c:61ff:fe1f:297b/64 scope link 2024-04-28T13:32:41.132 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:41.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:41.133 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T13:32:41.214 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-28T13:32:41.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:41.218 DEBUG:teuthology.orchestra.run.smithi203:> route 2024-04-28T13:32:41.270 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2024-04-28T13:32:41.270 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T13:32:41.270 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T13:32:41.270 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T13:32:41.270 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T13:32:41.271 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:32:41.271 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-28T13:32:41.271 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T13:32:41.271 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T13:32:41.271 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T13:32:41.271 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-28T13:32:41.374 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:32:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:32:41.470 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:32:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:32:41.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:41.480 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-28T13:32:41.529 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-28T13:32:41.529 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-28T13:32:41.529 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-28T13:32:41.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:41.529 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2024-04-28T13:32:41.612 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:32:41.612 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-28T13:32:41.664 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-28T13:32:41.665 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:32:41.665 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-28T13:32:41.715 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-28T13:32:41.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:41.716 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:32:41.850 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:32:41.850 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-28T13:32:41.850 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:32:41.871 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:32:41.871 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-28T13:32:41.871 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:32:41.871 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:32:41.871 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:32:41.898 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:32:41.898 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-28T13:32:41.898 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:32:41.898 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:32:41.899 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:32:41.899 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-28T13:32:41.899 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-28T13:32:41.899 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=270aea32-2e16-4a97-85f7-aeae39bc8c76" will pass to kernel 2024-04-28T13:32:41.903 INFO:teuthology.orchestra.run.smithi203.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-28T13:32:41.903 INFO:teuthology.orchestra.run.smithi203.stdout: You just mounted a file system that supports labels which does not 2024-04-28T13:32:41.903 INFO:teuthology.orchestra.run.smithi203.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-28T13:32:41.903 INFO:teuthology.orchestra.run.smithi203.stdout: applications will generate AVC messages and not be allowed access to 2024-04-28T13:32:41.903 INFO:teuthology.orchestra.run.smithi203.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-28T13:32:41.907 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-28T13:32:41.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:41.907 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-28T13:32:41.980 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:32:42.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:32:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:32:42.084 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:32:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:32:42.093 INFO:teuthology.orchestra.run.smithi203.stdout:client4837 2024-04-28T13:32:42.094 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:32:42.094 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:42.146 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds debug_mds 25 2024-04-28T13:32:42.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.361+0000 7f6eb6a00640 1 Processor -- start 2024-04-28T13:32:42.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.361+0000 7f6eb6a00640 1 -- start start 2024-04-28T13:32:42.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.363+0000 7f6eb6a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6eb00b4eb0 0x7f6eb00b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:42.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.363+0000 7f6eb6a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eb015dca0 con 0x7f6eb0143840 2024-04-28T13:32:42.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.363+0000 7f6eb6a00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eb015df80 con 0x7f6eb00ceff0 2024-04-28T13:32:42.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.363+0000 7f6eb6a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb0143c20 con 0x7f6eb00b4eb0 2024-04-28T13:32:42.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.364+0000 7f6eafe00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f6eb0143840 0x7f6eb00b3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:40428/0 (socket says 172.21.15.196:40428) 2024-04-28T13:32:42.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.364+0000 7f6eafe00640 1 -- 172.21.15.196:0/1793606253 learned_addr learned my addr 172.21.15.196:0/1793606253 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:42.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.364+0000 7f6eaea00640 1 -- 172.21.15.196:0/1793606253 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2138226999 0 0) 0x7f6eb015dca0 con 0x7f6eb0143840 2024-04-28T13:32:42.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.364+0000 7f6eaf400640 1 --2- 172.21.15.196:0/1793606253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6eb00b4eb0 0x7f6eb00b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:42.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.364+0000 7f6eaea00640 1 -- 172.21.15.196:0/1793606253 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e94003520 con 0x7f6eb0143840 2024-04-28T13:32:42.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.364+0000 7f6eaea00640 1 -- 172.21.15.196:0/1793606253 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 787885534 0 0) 0x7f6e94003520 con 0x7f6eb0143840 2024-04-28T13:32:42.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.364+0000 7f6eaea00640 1 -- 172.21.15.196:0/1793606253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6eb00b4eb0 msgr2=0x7f6eb00b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:42.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.364+0000 7f6eaea00640 1 --2- 172.21.15.196:0/1793606253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6eb00b4eb0 0x7f6eb00b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:42.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.364+0000 7f6eaea00640 1 -- 172.21.15.196:0/1793606253 >> v1:172.21.15.203:6789/0 conn(0x7f6eb00ceff0 legacy=0x7f6eb00cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:42.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.364+0000 7f6eaea00640 1 -- 172.21.15.196:0/1793606253 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb015e260 con 0x7f6eb0143840 2024-04-28T13:32:42.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eaea00640 1 -- 172.21.15.196:0/1793606253 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f6ea0001670 con 0x7f6eb0143840 2024-04-28T13:32:42.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eaea00640 1 -- 172.21.15.196:0/1793606253 >> v1:172.21.15.196:6789/0 conn(0x7f6eb0143840 legacy=0x7f6eb00b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:42.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eaea00640 1 --2- 172.21.15.196:0/1793606253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6eb00b4eb0 0x7f6e940039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:42.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eaea00640 1 --2- 172.21.15.196:0/1793606253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e94004040 0x7f6e94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:42.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eaea00640 1 --2- 172.21.15.196:0/1793606253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e94004d40 0x7f6e940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:42.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eaea00640 1 -- 172.21.15.196:0/1793606253 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb0143c20 con 0x7f6e94004d40 2024-04-28T13:32:42.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eaea00640 1 -- 172.21.15.196:0/1793606253 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9400bbb0 con 0x7f6e94004040 2024-04-28T13:32:42.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eaea00640 1 -- 172.21.15.196:0/1793606253 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6e9400bdb0 con 0x7f6eb00b4eb0 2024-04-28T13:32:42.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eb4a00640 1 --2- 172.21.15.196:0/1793606253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e94004d40 0x7f6e940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:42.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eafe00640 1 --2- 172.21.15.196:0/1793606253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6eb00b4eb0 0x7f6e940039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:42.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eaf400640 1 --2- 172.21.15.196:0/1793606253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e94004040 0x7f6e94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:42.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eb4a00640 1 -- 172.21.15.196:0/1793606253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6eb00b4eb0 msgr2=0x7f6e940039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:42.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eb4a00640 1 --2- 172.21.15.196:0/1793606253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6eb00b4eb0 0x7f6e940039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:42.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eb4a00640 1 -- 172.21.15.196:0/1793606253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e94004040 msgr2=0x7f6e94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:42.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eb4a00640 1 --2- 172.21.15.196:0/1793606253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e94004040 0x7f6e94004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:42.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eb4a00640 1 -- 172.21.15.196:0/1793606253 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e94003520 con 0x7f6e94004d40 2024-04-28T13:32:42.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.365+0000 7f6eafe00640 1 --2- 172.21.15.196:0/1793606253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6eb00b4eb0 0x7f6e940039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:42.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.366+0000 7f6eb4a00640 1 --2- 172.21.15.196:0/1793606253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e94004d40 0x7f6e940051c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80044b0 tx=0x7f6e94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1753f510b4e27c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:42.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.367+0000 7f6eaea00640 1 -- 172.21.15.196:0/1793606253 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8037070 con 0x7f6e94004d40 2024-04-28T13:32:42.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.367+0000 7f6eaea00640 1 -- 172.21.15.196:0/1793606253 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea802ad80 con 0x7f6e94004d40 2024-04-28T13:32:42.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.367+0000 7f6eaea00640 1 -- 172.21.15.196:0/1793606253 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8004080 con 0x7f6e94004d40 2024-04-28T13:32:42.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.367+0000 7f6eb6a00640 1 -- 172.21.15.196:0/1793606253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e94004d40 msgr2=0x7f6e940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:42.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.367+0000 7f6eb6a00640 1 --2- 172.21.15.196:0/1793606253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e94004d40 0x7f6e940051c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80044b0 tx=0x7f6e94005830 comp rx=0 tx=0).stop 2024-04-28T13:32:42.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.367+0000 7f6eb4a00640 1 -- 172.21.15.196:0/1793606253 reap_dead start 2024-04-28T13:32:42.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.367+0000 7f6eb6a00640 1 -- 172.21.15.196:0/1793606253 shutdown_connections 2024-04-28T13:32:42.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.367+0000 7f6eb6a00640 1 -- 172.21.15.196:0/1793606253 >> 172.21.15.196:0/1793606253 conn(0x7f6eb00cd8a0 msgr2=0x7f6eb00cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:42.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.367+0000 7f6eb6a00640 1 -- 172.21.15.196:0/1793606253 shutdown_connections 2024-04-28T13:32:42.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.367+0000 7f6eb6a00640 1 -- 172.21.15.196:0/1793606253 wait complete. 2024-04-28T13:32:42.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.369+0000 7f6eb6a00640 1 Processor -- start 2024-04-28T13:32:42.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.369+0000 7f6eb6a00640 1 -- start start 2024-04-28T13:32:42.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.370+0000 7f6eb6a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e94004d40 0x7f6eb01f2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:42.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.370+0000 7f6eb6a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6eb01f3610 0x7f6eb01f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:42.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.370+0000 7f6eb6a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eb00aa4a0 0x7f6eb00aa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:42.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.370+0000 7f6eb6a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb015d8c0 con 0x7f6eb00aa4a0 2024-04-28T13:32:42.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.370+0000 7f6eb6a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb00b46f0 con 0x7f6eb01f3610 2024-04-28T13:32:42.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.370+0000 7f6eb6a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb015da40 con 0x7f6e94004d40 2024-04-28T13:32:42.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.370+0000 7f6eb4a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eb00aa4a0 0x7f6eb00aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:42.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.371+0000 7f6eb4a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eb00aa4a0 0x7f6eb00aa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43138/0 (socket says 172.21.15.196:43138) 2024-04-28T13:32:42.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.371+0000 7f6eb4a00640 1 -- 172.21.15.196:0/2433180715 learned_addr learned my addr 172.21.15.196:0/2433180715 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:42.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.371+0000 7f6eafe00640 1 --2- 172.21.15.196:0/2433180715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e94004d40 0x7f6eb01f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:42.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.371+0000 7f6eaf400640 1 --2- 172.21.15.196:0/2433180715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6eb01f3610 0x7f6eb01f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:42.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.371+0000 7f6eb4a00640 1 -- 172.21.15.196:0/2433180715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e94004d40 msgr2=0x7f6eb01f2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:42.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.371+0000 7f6eb4a00640 1 --2- 172.21.15.196:0/2433180715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e94004d40 0x7f6eb01f2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:42.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.371+0000 7f6eb4a00640 1 -- 172.21.15.196:0/2433180715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6eb01f3610 msgr2=0x7f6eb01f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:42.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.371+0000 7f6eb4a00640 1 --2- 172.21.15.196:0/2433180715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6eb01f3610 0x7f6eb01f5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:42.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.371+0000 7f6eb4a00640 1 -- 172.21.15.196:0/2433180715 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb01f68e0 con 0x7f6eb00aa4a0 2024-04-28T13:32:42.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.371+0000 7f6eb4a00640 1 --2- 172.21.15.196:0/2433180715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eb00aa4a0 0x7f6eb00aa920 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80045d0 tx=0x7f6ea8003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:42.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.371+0000 7f6ea7e00640 1 -- 172.21.15.196:0/2433180715 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8032e60 con 0x7f6eb00aa4a0 2024-04-28T13:32:42.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.371+0000 7f6ea7e00640 1 -- 172.21.15.196:0/2433180715 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea8006b50 con 0x7f6eb00aa4a0 2024-04-28T13:32:42.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.372+0000 7f6ea7e00640 1 -- 172.21.15.196:0/2433180715 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8037040 con 0x7f6eb00aa4a0 2024-04-28T13:32:42.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.372+0000 7f6eb6a00640 1 -- 172.21.15.196:0/2433180715 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb01f6bc0 con 0x7f6eb00aa4a0 2024-04-28T13:32:42.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.372+0000 7f6eb6a00640 1 -- 172.21.15.196:0/2433180715 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb01f27c0 con 0x7f6eb00aa4a0 2024-04-28T13:32:42.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.373+0000 7f6eb6a00640 1 -- 172.21.15.196:0/2433180715 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb006a290 con 0x7f6eb00aa4a0 2024-04-28T13:32:42.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.374+0000 7f6ea7e00640 1 -- 172.21.15.196:0/2433180715 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ea8007030 con 0x7f6eb00aa4a0 2024-04-28T13:32:42.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.375+0000 7f6ea7e00640 1 --2- 172.21.15.196:0/2433180715 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6e9006c8f0 0x7f6e9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:42.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.375+0000 7f6ea7e00640 1 -- 172.21.15.196:0/2433180715 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6ea80ab940 con 0x7f6eb00aa4a0 2024-04-28T13:32:42.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.375+0000 7f6eafe00640 1 --2- 172.21.15.196:0/2433180715 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6e9006c8f0 0x7f6e9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:42.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.377+0000 7f6eafe00640 1 --2- 172.21.15.196:0/2433180715 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6e9006c8f0 0x7f6e9006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00079d0 tx=0x7f6ea0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:42.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.379+0000 7f6ea7e00640 1 -- 172.21.15.196:0/2433180715 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ea8079d60 con 0x7f6eb00aa4a0 2024-04-28T13:32:42.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.580+0000 7f6eb6a00640 1 -- 172.21.15.196:0/2433180715 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f6eb015d3c0 con 0x7f6eb00aa4a0 2024-04-28T13:32:42.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.585+0000 7f6ea7e00640 1 -- 172.21.15.196:0/2433180715 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v11)=0 v11) ==== 108+0+0 (secure 0 0 0) 0x7f6ea8079660 con 0x7f6eb00aa4a0 2024-04-28T13:32:42.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.588+0000 7f6eb6a00640 1 -- 172.21.15.196:0/2433180715 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6e9006c8f0 msgr2=0x7f6e9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:42.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.588+0000 7f6eb6a00640 1 --2- 172.21.15.196:0/2433180715 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6e9006c8f0 0x7f6e9006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00079d0 tx=0x7f6ea0009040 comp rx=0 tx=0).stop 2024-04-28T13:32:42.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.588+0000 7f6eb6a00640 1 -- 172.21.15.196:0/2433180715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eb00aa4a0 msgr2=0x7f6eb00aa920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:42.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.588+0000 7f6eb6a00640 1 --2- 172.21.15.196:0/2433180715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eb00aa4a0 0x7f6eb00aa920 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80045d0 tx=0x7f6ea8003a70 comp rx=0 tx=0).stop 2024-04-28T13:32:42.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.588+0000 7f6eb6a00640 1 -- 172.21.15.196:0/2433180715 shutdown_connections 2024-04-28T13:32:42.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.588+0000 7f6eb6a00640 1 --2- 172.21.15.196:0/2433180715 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f6e9006c8f0 0x7f6e9006edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:42.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.588+0000 7f6eb6a00640 1 --2- 172.21.15.196:0/2433180715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eb00aa4a0 0x7f6eb00aa920 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:42.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.588+0000 7f6eb6a00640 1 --2- 172.21.15.196:0/2433180715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6eb01f3610 0x7f6eb01f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:42.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.588+0000 7f6eb6a00640 1 --2- 172.21.15.196:0/2433180715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e94004d40 0x7f6eb01f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:42.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.588+0000 7f6eb6a00640 1 -- 172.21.15.196:0/2433180715 >> 172.21.15.196:0/2433180715 conn(0x7f6eb00cd8a0 msgr2=0x7f6eb00d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:42.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.588+0000 7f6eb6a00640 1 -- 172.21.15.196:0/2433180715 shutdown_connections 2024-04-28T13:32:42.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.588+0000 7f6eb6a00640 1 -- 172.21.15.196:0/2433180715 wait complete. 2024-04-28T13:32:42.608 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_quiesce_splitauth true 2024-04-28T13:32:42.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.815+0000 7f3d97e00640 1 Processor -- start 2024-04-28T13:32:42.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.816+0000 7f3d97e00640 1 -- start start 2024-04-28T13:32:42.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.817+0000 7f3d97e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3d980d3c10 0x7f3d980b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:42.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.818+0000 7f3d97e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d980b8370 0x7f3d980d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:42.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.818+0000 7f3d97e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d980cfb80 con 0x7f3d980d3830 2024-04-28T13:32:42.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.818+0000 7f3d97e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3d980d2ee0 con 0x7f3d980d3c10 2024-04-28T13:32:42.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.818+0000 7f3d97e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3d980d31c0 con 0x7f3d980b8370 2024-04-28T13:32:42.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.818+0000 7f3d8e000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3d980d3c10 0x7f3d980b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:42.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.818+0000 7f3d8e000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3d980d3c10 0x7f3d980b7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43140/0 (socket says 172.21.15.196:43140) 2024-04-28T13:32:42.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.818+0000 7f3d8e000640 1 -- 172.21.15.196:0/1074689444 learned_addr learned my addr 172.21.15.196:0/1074689444 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:42.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.818+0000 7f3d97400640 1 --2- 172.21.15.196:0/1074689444 >> v2:172.21.15.203:3300/0 conn(0x7f3d980b8370 0x7f3d980d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:42.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.818+0000 7f3d8e000640 1 -- 172.21.15.196:0/1074689444 >> v2:172.21.15.203:3300/0 conn(0x7f3d980b8370 msgr2=0x7f3d980d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:42.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.818+0000 7f3d8e000640 1 --2- 172.21.15.196:0/1074689444 >> v2:172.21.15.203:3300/0 conn(0x7f3d980b8370 0x7f3d980d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:42.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.818+0000 7f3d8e000640 1 -- 172.21.15.196:0/1074689444 >> v1:172.21.15.203:6789/0 conn(0x7f3d980d3830 legacy=0x7f3d980d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:42.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.818+0000 7f3d8e000640 1 -- 172.21.15.196:0/1074689444 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d980cfe00 con 0x7f3d980d3c10 2024-04-28T13:32:42.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.820+0000 7f3d8e000640 1 --2- 172.21.15.196:0/1074689444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d980d3c10 0x7f3d980b7e30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3d9808e7e0 tx=0x7f3d8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afcb4ab3d8069d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:42.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.820+0000 7f3d96000640 1 -- 172.21.15.196:0/1074689444 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80037070 con 0x7f3d980d3c10 2024-04-28T13:32:42.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.820+0000 7f3d96000640 1 -- 172.21.15.196:0/1074689444 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d80032440 con 0x7f3d980d3c10 2024-04-28T13:32:42.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.820+0000 7f3d96000640 1 -- 172.21.15.196:0/1074689444 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80032810 con 0x7f3d980d3c10 2024-04-28T13:32:42.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.820+0000 7f3d97e00640 1 -- 172.21.15.196:0/1074689444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d980d3c10 msgr2=0x7f3d980b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:42.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.820+0000 7f3d97e00640 1 --2- 172.21.15.196:0/1074689444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d980d3c10 0x7f3d980b7e30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3d9808e7e0 tx=0x7f3d8002a230 comp rx=0 tx=0).stop 2024-04-28T13:32:42.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.821+0000 7f3d97e00640 1 -- 172.21.15.196:0/1074689444 shutdown_connections 2024-04-28T13:32:42.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.821+0000 7f3d97e00640 1 --2- 172.21.15.196:0/1074689444 >> v2:172.21.15.203:3300/0 conn(0x7f3d980b8370 0x7f3d980d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:42.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.821+0000 7f3d97e00640 1 --2- 172.21.15.196:0/1074689444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d980d3c10 0x7f3d980b7e30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:42.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.821+0000 7f3d97e00640 1 -- 172.21.15.196:0/1074689444 >> 172.21.15.196:0/1074689444 conn(0x7f3d980d4fe0 msgr2=0x7f3d980d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:42.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.821+0000 7f3d97e00640 1 -- 172.21.15.196:0/1074689444 shutdown_connections 2024-04-28T13:32:42.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.821+0000 7f3d97e00640 1 -- 172.21.15.196:0/1074689444 wait complete. 2024-04-28T13:32:42.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.822+0000 7f3d97e00640 1 Processor -- start 2024-04-28T13:32:42.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.823+0000 7f3d97e00640 1 -- start start 2024-04-28T13:32:42.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.823+0000 7f3d97e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d980b8370 0x7f3d980c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:42.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.824+0000 7f3d97e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d980d3830 0x7f3d9814c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:42.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.824+0000 7f3d97e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d980d3c10 0x7f3d980ce730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:42.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.824+0000 7f3d97e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d980d2bb0 con 0x7f3d980d3c10 2024-04-28T13:32:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.824+0000 7f3d97e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3d980d5d90 con 0x7f3d980b8370 2024-04-28T13:32:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.824+0000 7f3d97e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3d980d0b40 con 0x7f3d980d3830 2024-04-28T13:32:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.824+0000 7f3d97400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d980d3c10 0x7f3d980ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.824+0000 7f3d97400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d980d3c10 0x7f3d980ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43152/0 (socket says 172.21.15.196:43152) 2024-04-28T13:32:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.824+0000 7f3d97400640 1 -- 172.21.15.196:0/3898100535 learned_addr learned my addr 172.21.15.196:0/3898100535 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.824+0000 7f3d96a00640 1 --2- 172.21.15.196:0/3898100535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d980b8370 0x7f3d980c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.824+0000 7f3d97400640 1 -- 172.21.15.196:0/3898100535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d980d3830 msgr2=0x7f3d9814c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.824+0000 7f3d97400640 1 --2- 172.21.15.196:0/3898100535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d980d3830 0x7f3d9814c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.824+0000 7f3d97400640 1 -- 172.21.15.196:0/3898100535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d980b8370 msgr2=0x7f3d980c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.824+0000 7f3d97400640 1 --2- 172.21.15.196:0/3898100535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d980b8370 0x7f3d980c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.824+0000 7f3d97400640 1 -- 172.21.15.196:0/3898100535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d981566a0 con 0x7f3d980d3c10 2024-04-28T13:32:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.824+0000 7f3d96a00640 1 --2- 172.21.15.196:0/3898100535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d980b8370 0x7f3d980c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.825+0000 7f3d97400640 1 --2- 172.21.15.196:0/3898100535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d980d3c10 0x7f3d980ce730 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3d90007e30 tx=0x7f3d9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:42.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.825+0000 7f3d8fe00640 1 -- 172.21.15.196:0/3898100535 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90014620 con 0x7f3d980d3c10 2024-04-28T13:32:42.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.825+0000 7f3d8fe00640 1 -- 172.21.15.196:0/3898100535 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d90019070 con 0x7f3d980d3c10 2024-04-28T13:32:42.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.825+0000 7f3d97e00640 1 -- 172.21.15.196:0/3898100535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9814c910 con 0x7f3d980d3c10 2024-04-28T13:32:42.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.825+0000 7f3d8fe00640 1 -- 172.21.15.196:0/3898100535 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90014a00 con 0x7f3d980d3c10 2024-04-28T13:32:42.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.825+0000 7f3d97e00640 1 -- 172.21.15.196:0/3898100535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d980b8eb0 con 0x7f3d980d3c10 2024-04-28T13:32:42.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.826+0000 7f3d97e00640 1 -- 172.21.15.196:0/3898100535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d98050730 con 0x7f3d980d3c10 2024-04-28T13:32:42.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.827+0000 7f3d8fe00640 1 -- 172.21.15.196:0/3898100535 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d90014ba0 con 0x7f3d980d3c10 2024-04-28T13:32:42.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.828+0000 7f3d8fe00640 1 --2- 172.21.15.196:0/3898100535 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3d7406c920 0x7f3d7406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:42.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.828+0000 7f3d8fe00640 1 -- 172.21.15.196:0/3898100535 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3d9008c4b0 con 0x7f3d980d3c10 2024-04-28T13:32:42.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.829+0000 7f3d96a00640 1 --2- 172.21.15.196:0/3898100535 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3d7406c920 0x7f3d7406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:42.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.830+0000 7f3d96a00640 1 --2- 172.21.15.196:0/3898100535 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3d7406c920 0x7f3d7406ede0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3d88003eb0 tx=0x7f3d88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:42.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:42.833+0000 7f3d8fe00640 1 -- 172.21.15.196:0/3898100535 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d981566a0 con 0x7f3d980d3c10 2024-04-28T13:32:43.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.034+0000 7f3d97e00640 1 -- 172.21.15.196:0/3898100535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command([{prefix=config set, name=mds_cache_quiesce_splitauth}] v 0) -- 0x7f3d980d0f00 con 0x7f3d980d3c10 2024-04-28T13:32:43.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.039+0000 7f3d8fe00640 1 -- 172.21.15.196:0/3898100535 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_quiesce_splitauth}]=0 v12)=0 v12) ==== 128+0+0 (secure 0 0 0) 0x7f3d9005a840 con 0x7f3d980d3c10 2024-04-28T13:32:43.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.042+0000 7f3d97e00640 1 -- 172.21.15.196:0/3898100535 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3d7406c920 msgr2=0x7f3d7406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:43.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.042+0000 7f3d97e00640 1 --2- 172.21.15.196:0/3898100535 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3d7406c920 0x7f3d7406ede0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3d88003eb0 tx=0x7f3d88009040 comp rx=0 tx=0).stop 2024-04-28T13:32:43.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.042+0000 7f3d97e00640 1 -- 172.21.15.196:0/3898100535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d980d3c10 msgr2=0x7f3d980ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:43.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.042+0000 7f3d97e00640 1 --2- 172.21.15.196:0/3898100535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d980d3c10 0x7f3d980ce730 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3d90007e30 tx=0x7f3d9000e4b0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.042+0000 7f3d97e00640 1 -- 172.21.15.196:0/3898100535 shutdown_connections 2024-04-28T13:32:43.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.042+0000 7f3d97e00640 1 --2- 172.21.15.196:0/3898100535 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3d7406c920 0x7f3d7406ede0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.042+0000 7f3d97e00640 1 --2- 172.21.15.196:0/3898100535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d980d3c10 0x7f3d980ce730 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.042+0000 7f3d97e00640 1 --2- 172.21.15.196:0/3898100535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d980d3830 0x7f3d9814c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.042+0000 7f3d97e00640 1 --2- 172.21.15.196:0/3898100535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d980b8370 0x7f3d980c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.042+0000 7f3d97e00640 1 -- 172.21.15.196:0/3898100535 >> 172.21.15.196:0/3898100535 conn(0x7f3d980d4fe0 msgr2=0x7f3d981637d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:43.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.042+0000 7f3d97e00640 1 -- 172.21.15.196:0/3898100535 shutdown_connections 2024-04-28T13:32:43.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.042+0000 7f3d97e00640 1 -- 172.21.15.196:0/3898100535 wait complete. 2024-04-28T13:32:43.062 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol_quiesce --mode=777 2024-04-28T13:32:43.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.268+0000 7fececc00640 1 Processor -- start 2024-04-28T13:32:43.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.269+0000 7fececc00640 1 -- start start 2024-04-28T13:32:43.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.270+0000 7fececc00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fece81437f0 0x7fece806f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:43.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.270+0000 7fececc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fece8160e70 0x7fece8144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:43.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.270+0000 7fececc00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece81612e0 con 0x7fece80d1190 2024-04-28T13:32:43.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.270+0000 7fececc00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fece8073440 con 0x7fece8160e70 2024-04-28T13:32:43.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.270+0000 7fececc00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fece8161570 con 0x7fece81437f0 2024-04-28T13:32:43.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.271+0000 7fecdfe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fece8160e70 0x7fece8144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:43.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.271+0000 7fecdfe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fece8160e70 0x7fece8144120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43154/0 (socket says 172.21.15.196:43154) 2024-04-28T13:32:43.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.271+0000 7fecdfe00640 1 -- 172.21.15.196:0/869996821 learned_addr learned my addr 172.21.15.196:0/869996821 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:43.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.271+0000 7fece5e00640 1 --2- 172.21.15.196:0/869996821 >> v2:172.21.15.203:3300/0 conn(0x7fece81437f0 0x7fece806f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:43.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.271+0000 7fecdfe00640 1 -- 172.21.15.196:0/869996821 >> v2:172.21.15.203:3300/0 conn(0x7fece81437f0 msgr2=0x7fece806f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:43.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.271+0000 7fecdfe00640 1 --2- 172.21.15.196:0/869996821 >> v2:172.21.15.203:3300/0 conn(0x7fece81437f0 0x7fece806f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.271+0000 7fecdfe00640 1 -- 172.21.15.196:0/869996821 >> v1:172.21.15.203:6789/0 conn(0x7fece80d1190 legacy=0x7fece8073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:43.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.271+0000 7fecdfe00640 1 -- 172.21.15.196:0/869996821 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece8161850 con 0x7fece8160e70 2024-04-28T13:32:43.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.272+0000 7fecdfe00640 1 --2- 172.21.15.196:0/869996821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece8160e70 0x7fece8144120 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fece80ceb40 tx=0x7fecd002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9f44fbc6584a73 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:43.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.273+0000 7fece5400640 1 -- 172.21.15.196:0/869996821 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd0037070 con 0x7fece8160e70 2024-04-28T13:32:43.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.273+0000 7fece5400640 1 -- 172.21.15.196:0/869996821 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd0032440 con 0x7fece8160e70 2024-04-28T13:32:43.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.273+0000 7fece5400640 1 -- 172.21.15.196:0/869996821 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd0032810 con 0x7fece8160e70 2024-04-28T13:32:43.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.274+0000 7fececc00640 1 -- 172.21.15.196:0/869996821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece8160e70 msgr2=0x7fece8144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:43.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.274+0000 7fececc00640 1 --2- 172.21.15.196:0/869996821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece8160e70 0x7fece8144120 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fece80ceb40 tx=0x7fecd002a230 comp rx=0 tx=0).stop 2024-04-28T13:32:43.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.274+0000 7fececc00640 1 -- 172.21.15.196:0/869996821 shutdown_connections 2024-04-28T13:32:43.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.274+0000 7fececc00640 1 --2- 172.21.15.196:0/869996821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece8160e70 0x7fece8144120 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.274+0000 7fececc00640 1 --2- 172.21.15.196:0/869996821 >> v2:172.21.15.203:3300/0 conn(0x7fece81437f0 0x7fece806f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.274+0000 7fececc00640 1 -- 172.21.15.196:0/869996821 >> 172.21.15.196:0/869996821 conn(0x7fece806c690 msgr2=0x7fece806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:43.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.274+0000 7fececc00640 1 -- 172.21.15.196:0/869996821 shutdown_connections 2024-04-28T13:32:43.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.274+0000 7fececc00640 1 -- 172.21.15.196:0/869996821 wait complete. 2024-04-28T13:32:43.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.276+0000 7fececc00640 1 Processor -- start 2024-04-28T13:32:43.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.276+0000 7fececc00640 1 -- start start 2024-04-28T13:32:43.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.277+0000 7fececc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece80d1190 0x7fece815c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:43.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.277+0000 7fececc00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fece81437f0 0x7fece815dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:43.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.277+0000 7fececc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fece8160e70 0x7fece81601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:43.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.277+0000 7fececc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fece8144660 con 0x7fece80d1190 2024-04-28T13:32:43.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.277+0000 7fececc00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fece80736a0 con 0x7fece8160e70 2024-04-28T13:32:43.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.277+0000 7fececc00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fece81447e0 con 0x7fece81437f0 2024-04-28T13:32:43.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.277+0000 7fece5e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece80d1190 0x7fece815c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:43.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.277+0000 7fece5e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece80d1190 0x7fece815c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43170/0 (socket says 172.21.15.196:43170) 2024-04-28T13:32:43.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.277+0000 7fece5e00640 1 -- 172.21.15.196:0/3899841085 learned_addr learned my addr 172.21.15.196:0/3899841085 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:43.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.278+0000 7fece5e00640 1 -- 172.21.15.196:0/3899841085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fece81437f0 msgr2=0x7fece815dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:43.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.278+0000 7fecdfe00640 1 --2- 172.21.15.196:0/3899841085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fece81437f0 0x7fece815dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:43.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.278+0000 7fece5e00640 1 --2- 172.21.15.196:0/3899841085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fece81437f0 0x7fece815dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.278+0000 7fece5e00640 1 -- 172.21.15.196:0/3899841085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fece8160e70 msgr2=0x7fece81601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:43.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.278+0000 7fece5e00640 1 --2- 172.21.15.196:0/3899841085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fece8160e70 0x7fece81601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.278+0000 7fece5e00640 1 -- 172.21.15.196:0/3899841085 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece815e640 con 0x7fece80d1190 2024-04-28T13:32:43.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.278+0000 7fece5e00640 1 --2- 172.21.15.196:0/3899841085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece80d1190 0x7fece815c4d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fecd800d830 tx=0x7fecd800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:43.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.278+0000 7fecde000640 1 -- 172.21.15.196:0/3899841085 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd800b6c0 con 0x7fece80d1190 2024-04-28T13:32:43.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.278+0000 7fecde000640 1 -- 172.21.15.196:0/3899841085 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd8019070 con 0x7fece80d1190 2024-04-28T13:32:43.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.278+0000 7fecde000640 1 -- 172.21.15.196:0/3899841085 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd800ba80 con 0x7fece80d1190 2024-04-28T13:32:43.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.278+0000 7fececc00640 1 -- 172.21.15.196:0/3899841085 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece815e920 con 0x7fece80d1190 2024-04-28T13:32:43.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.278+0000 7fececc00640 1 -- 172.21.15.196:0/3899841085 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece815eee0 con 0x7fece80d1190 2024-04-28T13:32:43.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.280+0000 7fececc00640 1 -- 172.21.15.196:0/3899841085 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece806a290 con 0x7fece80d1190 2024-04-28T13:32:43.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.280+0000 7fecde000640 1 -- 172.21.15.196:0/3899841085 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecd801e460 con 0x7fece80d1190 2024-04-28T13:32:43.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.281+0000 7fecde000640 1 --2- 172.21.15.196:0/3899841085 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fecc406c8f0 0x7fecc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:43.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.281+0000 7fecde000640 1 -- 172.21.15.196:0/3899841085 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fecd808c360 con 0x7fece80d1190 2024-04-28T13:32:43.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.282+0000 7fecdfe00640 1 --2- 172.21.15.196:0/3899841085 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fecc406c8f0 0x7fecc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:43.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.283+0000 7fecdfe00640 1 --2- 172.21.15.196:0/3899841085 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fecc406c8f0 0x7fecc406edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fecd0034000 tx=0x7fecd002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:43.286 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.286+0000 7fecde000640 1 -- 172.21.15.196:0/3899841085 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fecd805a660 con 0x7fece80d1190 2024-04-28T13:32:43.523 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.523+0000 7fececc00640 1 -- 172.21.15.196:0/3899841085 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol_quiesce", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fece806eb80 con 0x7fecc406c8f0 2024-04-28T13:32:43.534 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:32:43.534+0000 7f3ba0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-28T13:32:43.534 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:32:43.534+0000 7f3ba0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-28T13:32:43.534 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:32:43.534+0000 7f3ba0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-28T13:32:43.534 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:32:43.534+0000 7f3ba0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-28T13:32:43.534 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-28T13:32:43.534+0000 7f3ba0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-28T13:32:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.605+0000 7fecde000640 1 -- 172.21.15.196:0/3899841085 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fece806eb80 con 0x7fecc406c8f0 2024-04-28T13:32:43.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.608+0000 7fececc00640 1 -- 172.21.15.196:0/3899841085 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fecc406c8f0 msgr2=0x7fecc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:43.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.608+0000 7fececc00640 1 --2- 172.21.15.196:0/3899841085 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fecc406c8f0 0x7fecc406edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fecd0034000 tx=0x7fecd002e000 comp rx=0 tx=0).stop 2024-04-28T13:32:43.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.608+0000 7fececc00640 1 -- 172.21.15.196:0/3899841085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece80d1190 msgr2=0x7fece815c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:43.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.608+0000 7fececc00640 1 --2- 172.21.15.196:0/3899841085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece80d1190 0x7fece815c4d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fecd800d830 tx=0x7fecd800dcf0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.608+0000 7fececc00640 1 -- 172.21.15.196:0/3899841085 shutdown_connections 2024-04-28T13:32:43.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.608+0000 7fececc00640 1 --2- 172.21.15.196:0/3899841085 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fecc406c8f0 0x7fecc406edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.608+0000 7fececc00640 1 --2- 172.21.15.196:0/3899841085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fece8160e70 0x7fece81601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.608+0000 7fececc00640 1 --2- 172.21.15.196:0/3899841085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fece81437f0 0x7fece815dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.608+0000 7fececc00640 1 --2- 172.21.15.196:0/3899841085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fece80d1190 0x7fece815c4d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.608+0000 7fececc00640 1 -- 172.21.15.196:0/3899841085 >> 172.21.15.196:0/3899841085 conn(0x7fece806c690 msgr2=0x7fece80ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:43.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.608+0000 7fececc00640 1 -- 172.21.15.196:0/3899841085 shutdown_connections 2024-04-28T13:32:43.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.608+0000 7fececc00640 1 -- 172.21.15.196:0/3899841085 wait complete. 2024-04-28T13:32:43.628 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_quiesce 2024-04-28T13:32:43.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.829+0000 7fc782c00640 1 Processor -- start 2024-04-28T13:32:43.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.829+0000 7fc782c00640 1 -- start start 2024-04-28T13:32:43.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.831+0000 7fc782c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc77c0b6b70 0x7fc77c0720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:43.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.831+0000 7fc782c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc77c0b46a0 con 0x7fc77c06f340 2024-04-28T13:32:43.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.831+0000 7fc782c00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc77c0b5db0 con 0x7fc77c0d0300 2024-04-28T13:32:43.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.831+0000 7fc782c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc77c0b6090 con 0x7fc77c0b6b70 2024-04-28T13:32:43.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.831+0000 7fc77be00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc77c06f340 0x7fc77c06f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:40444/0 (socket says 172.21.15.196:40444) 2024-04-28T13:32:43.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.831+0000 7fc77be00640 1 -- 172.21.15.196:0/337091108 learned_addr learned my addr 172.21.15.196:0/337091108 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:43.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.831+0000 7fc780c00640 1 --2- 172.21.15.196:0/337091108 >> v2:172.21.15.196:3300/0 conn(0x7fc77c0b6b70 0x7fc77c0720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:43.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.832+0000 7fc77aa00640 1 -- 172.21.15.196:0/337091108 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2165738406 0 0) 0x7fc77c0b46a0 con 0x7fc77c06f340 2024-04-28T13:32:43.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.832+0000 7fc780c00640 1 -- 172.21.15.196:0/337091108 >> v1:172.21.15.203:6789/0 conn(0x7fc77c0d0300 legacy=0x7fc77c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:43.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.832+0000 7fc780c00640 1 -- 172.21.15.196:0/337091108 >> v1:172.21.15.196:6789/0 conn(0x7fc77c06f340 legacy=0x7fc77c06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:43.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.832+0000 7fc780c00640 1 -- 172.21.15.196:0/337091108 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c0b6370 con 0x7fc77c0b6b70 2024-04-28T13:32:43.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.833+0000 7fc780c00640 1 --2- 172.21.15.196:0/337091108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c0b6b70 0x7fc77c0720f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc7740097a0 tx=0x7fc77402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1cc25403c4faf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:43.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.833+0000 7fc77aa00640 1 -- 172.21.15.196:0/337091108 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc774037070 con 0x7fc77c0b6b70 2024-04-28T13:32:43.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.833+0000 7fc77aa00640 1 -- 172.21.15.196:0/337091108 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77402ae50 con 0x7fc77c0b6b70 2024-04-28T13:32:43.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.833+0000 7fc77aa00640 1 -- 172.21.15.196:0/337091108 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc774032660 con 0x7fc77c0b6b70 2024-04-28T13:32:43.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.834+0000 7fc782c00640 1 -- 172.21.15.196:0/337091108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c0b6b70 msgr2=0x7fc77c0720f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:43.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.834+0000 7fc782c00640 1 --2- 172.21.15.196:0/337091108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c0b6b70 0x7fc77c0720f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc7740097a0 tx=0x7fc77402a140 comp rx=0 tx=0).stop 2024-04-28T13:32:43.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.834+0000 7fc782c00640 1 -- 172.21.15.196:0/337091108 shutdown_connections 2024-04-28T13:32:43.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.834+0000 7fc782c00640 1 --2- 172.21.15.196:0/337091108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c0b6b70 0x7fc77c0720f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.834+0000 7fc782c00640 1 -- 172.21.15.196:0/337091108 >> 172.21.15.196:0/337091108 conn(0x7fc77c0cd8a0 msgr2=0x7fc77c0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:43.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.834+0000 7fc782c00640 1 -- 172.21.15.196:0/337091108 shutdown_connections 2024-04-28T13:32:43.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.834+0000 7fc782c00640 1 -- 172.21.15.196:0/337091108 wait complete. 2024-04-28T13:32:43.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.836+0000 7fc782c00640 1 Processor -- start 2024-04-28T13:32:43.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.836+0000 7fc782c00640 1 -- start start 2024-04-28T13:32:43.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.837+0000 7fc782c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc77c06f340 0x7fc77c159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:43.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.837+0000 7fc782c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c0b6b70 0x7fc77c0ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:43.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.837+0000 7fc782c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc77c0d0300 0x7fc77c0cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:43.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.837+0000 7fc782c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c142bb0 con 0x7fc77c0b6b70 2024-04-28T13:32:43.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.837+0000 7fc782c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c142a30 con 0x7fc77c0d0300 2024-04-28T13:32:43.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.837+0000 7fc782c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc77c072630 con 0x7fc77c06f340 2024-04-28T13:32:43.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.837+0000 7fc77b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c0b6b70 0x7fc77c0ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:43.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.837+0000 7fc77b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c0b6b70 0x7fc77c0ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43202/0 (socket says 172.21.15.196:43202) 2024-04-28T13:32:43.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.837+0000 7fc77b400640 1 -- 172.21.15.196:0/3654009979 learned_addr learned my addr 172.21.15.196:0/3654009979 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:43.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.837+0000 7fc780c00640 1 --2- 172.21.15.196:0/3654009979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc77c0d0300 0x7fc77c0cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:43.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.837+0000 7fc77b400640 1 -- 172.21.15.196:0/3654009979 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc77c06f340 msgr2=0x7fc77c159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:43.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.837+0000 7fc77b400640 1 --2- 172.21.15.196:0/3654009979 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc77c06f340 0x7fc77c159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.837+0000 7fc77b400640 1 -- 172.21.15.196:0/3654009979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc77c0d0300 msgr2=0x7fc77c0cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:43.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.837+0000 7fc77b400640 1 --2- 172.21.15.196:0/3654009979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc77c0d0300 0x7fc77c0cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:43.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.837+0000 7fc77b400640 1 -- 172.21.15.196:0/3654009979 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c0cac90 con 0x7fc77c0b6b70 2024-04-28T13:32:43.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.838+0000 7fc77b400640 1 --2- 172.21.15.196:0/3654009979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c0b6b70 0x7fc77c0ca1d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc76400ba50 tx=0x7fc76400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:43.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.838+0000 7fc773e00640 1 -- 172.21.15.196:0/3654009979 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764019070 con 0x7fc77c0b6b70 2024-04-28T13:32:43.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.838+0000 7fc773e00640 1 -- 172.21.15.196:0/3654009979 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc76400cc20 con 0x7fc77c0b6b70 2024-04-28T13:32:43.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.838+0000 7fc773e00640 1 -- 172.21.15.196:0/3654009979 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764014680 con 0x7fc77c0b6b70 2024-04-28T13:32:43.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.838+0000 7fc782c00640 1 -- 172.21.15.196:0/3654009979 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77c0caf70 con 0x7fc77c0b6b70 2024-04-28T13:32:43.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.838+0000 7fc782c00640 1 -- 172.21.15.196:0/3654009979 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc77c0cb530 con 0x7fc77c0b6b70 2024-04-28T13:32:43.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.839+0000 7fc782c00640 1 -- 172.21.15.196:0/3654009979 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc77c06a290 con 0x7fc77c0b6b70 2024-04-28T13:32:43.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.840+0000 7fc773e00640 1 -- 172.21.15.196:0/3654009979 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc76401c070 con 0x7fc77c0b6b70 2024-04-28T13:32:43.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.842+0000 7fc773e00640 1 --2- 172.21.15.196:0/3654009979 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc75806c8f0 0x7fc75806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:43.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.842+0000 7fc773e00640 1 -- 172.21.15.196:0/3654009979 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc76408b9a0 con 0x7fc77c0b6b70 2024-04-28T13:32:43.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.842+0000 7fc77be00640 1 --2- 172.21.15.196:0/3654009979 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc75806c8f0 0x7fc75806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:43.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.844+0000 7fc77be00640 1 --2- 172.21.15.196:0/3654009979 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc75806c8f0 0x7fc75806edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc76c007fd0 tx=0x7fc76c007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:43.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:43.846+0000 7fc773e00640 1 -- 172.21.15.196:0/3654009979 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc764059650 con 0x7fc77c0b6b70 2024-04-28T13:32:44.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.072+0000 7fc782c00640 1 -- 172.21.15.196:0/3654009979 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_quiesce", "target": ["mon-mgr", ""]}) -- 0x7fc77c074450 con 0x7fc75806c8f0 2024-04-28T13:32:44.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.084+0000 7fc773e00640 1 -- 172.21.15.196:0/3654009979 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+70 (secure 0 0 0) 0x7fc77c074450 con 0x7fc75806c8f0 2024-04-28T13:32:44.084 INFO:teuthology.orchestra.run.smithi196.stdout:/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 2024-04-28T13:32:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.086+0000 7fc782c00640 1 -- 172.21.15.196:0/3654009979 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc75806c8f0 msgr2=0x7fc75806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.086+0000 7fc782c00640 1 --2- 172.21.15.196:0/3654009979 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc75806c8f0 0x7fc75806edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc76c007fd0 tx=0x7fc76c007920 comp rx=0 tx=0).stop 2024-04-28T13:32:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.086+0000 7fc782c00640 1 -- 172.21.15.196:0/3654009979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c0b6b70 msgr2=0x7fc77c0ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.086+0000 7fc782c00640 1 --2- 172.21.15.196:0/3654009979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c0b6b70 0x7fc77c0ca1d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc76400ba50 tx=0x7fc76400bf10 comp rx=0 tx=0).stop 2024-04-28T13:32:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.087+0000 7fc782c00640 1 -- 172.21.15.196:0/3654009979 shutdown_connections 2024-04-28T13:32:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.087+0000 7fc782c00640 1 --2- 172.21.15.196:0/3654009979 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc75806c8f0 0x7fc75806edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.087+0000 7fc782c00640 1 --2- 172.21.15.196:0/3654009979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc77c0d0300 0x7fc77c0cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.087+0000 7fc782c00640 1 --2- 172.21.15.196:0/3654009979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc77c0b6b70 0x7fc77c0ca1d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.087+0000 7fc782c00640 1 --2- 172.21.15.196:0/3654009979 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc77c06f340 0x7fc77c159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.087+0000 7fc782c00640 1 -- 172.21.15.196:0/3654009979 >> 172.21.15.196:0/3654009979 conn(0x7fc77c0cd8a0 msgr2=0x7fc77c0ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.087+0000 7fc782c00640 1 -- 172.21.15.196:0/3654009979 shutdown_connections 2024-04-28T13:32:44.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.087+0000 7fc782c00640 1 -- 172.21.15.196:0/3654009979 wait complete. 2024-04-28T13:32:44.103 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:32:44.103 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:44.151 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:32:44.151 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:44.244 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:32:44.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.492+0000 7f7556e00640 1 Processor -- start 2024-04-28T13:32:44.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.492+0000 7f7556e00640 1 -- start start 2024-04-28T13:32:44.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.494+0000 7f7556e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f75500d0d00 0x7f75500d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:44.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.494+0000 7f7556e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f75500b5dd0 0x7f75500b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:44.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.494+0000 7f7556e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7550160f40 con 0x7f75501469f0 2024-04-28T13:32:44.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.494+0000 7f7556e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f75500b5960 con 0x7f75500b5dd0 2024-04-28T13:32:44.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.494+0000 7f7556e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f7550161ac0 con 0x7f75500d0d00 2024-04-28T13:32:44.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.495+0000 7f7554e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f75500b5dd0 0x7f75500b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:44.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.495+0000 7f7554e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f75500b5dd0 0x7f75500b6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43214/0 (socket says 172.21.15.196:43214) 2024-04-28T13:32:44.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.495+0000 7f7554e00640 1 -- 172.21.15.196:0/2797597955 learned_addr learned my addr 172.21.15.196:0/2797597955 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:44.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.495+0000 7f7554e00640 1 -- 172.21.15.196:0/2797597955 >> v2:172.21.15.203:3300/0 conn(0x7f75500d0d00 msgr2=0x7f75500d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:44.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.495+0000 7f7554e00640 1 --2- 172.21.15.196:0/2797597955 >> v2:172.21.15.203:3300/0 conn(0x7f75500d0d00 0x7f75500d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.495+0000 7f7554e00640 1 -- 172.21.15.196:0/2797597955 >> v1:172.21.15.203:6789/0 conn(0x7f75501469f0 legacy=0x7f75500ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:44.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.495+0000 7f7554e00640 1 -- 172.21.15.196:0/2797597955 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7550161220 con 0x7f75500b5dd0 2024-04-28T13:32:44.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.496+0000 7f7554e00640 1 --2- 172.21.15.196:0/2797597955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75500b5dd0 0x7f75500b6200 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f75500d1de0 tx=0x7f754802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de62046470982726 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:44.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.497+0000 7f754f400640 1 -- 172.21.15.196:0/2797597955 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548037070 con 0x7f75500b5dd0 2024-04-28T13:32:44.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.497+0000 7f754f400640 1 -- 172.21.15.196:0/2797597955 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7548032440 con 0x7f75500b5dd0 2024-04-28T13:32:44.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.497+0000 7f754f400640 1 -- 172.21.15.196:0/2797597955 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548032810 con 0x7f75500b5dd0 2024-04-28T13:32:44.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.497+0000 7f7556e00640 1 -- 172.21.15.196:0/2797597955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75500b5dd0 msgr2=0x7f75500b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:44.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.497+0000 7f7556e00640 1 --2- 172.21.15.196:0/2797597955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75500b5dd0 0x7f75500b6200 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f75500d1de0 tx=0x7f754802a230 comp rx=0 tx=0).stop 2024-04-28T13:32:44.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.497+0000 7f7556e00640 1 -- 172.21.15.196:0/2797597955 shutdown_connections 2024-04-28T13:32:44.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.497+0000 7f7556e00640 1 --2- 172.21.15.196:0/2797597955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75500b5dd0 0x7f75500b6200 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.497+0000 7f7556e00640 1 --2- 172.21.15.196:0/2797597955 >> v2:172.21.15.203:3300/0 conn(0x7f75500d0d00 0x7f75500d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.497+0000 7f7556e00640 1 -- 172.21.15.196:0/2797597955 >> 172.21.15.196:0/2797597955 conn(0x7f75500cd830 msgr2=0x7f75500cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:44.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.497+0000 7f7556e00640 1 -- 172.21.15.196:0/2797597955 shutdown_connections 2024-04-28T13:32:44.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.498+0000 7f7556e00640 1 -- 172.21.15.196:0/2797597955 wait complete. 2024-04-28T13:32:44.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.499+0000 7f7556e00640 1 Processor -- start 2024-04-28T13:32:44.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.499+0000 7f7556e00640 1 -- start start 2024-04-28T13:32:44.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.500+0000 7f7556e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75500b5dd0 0x7f7550157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:44.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.500+0000 7f7556e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75500d0d00 0x7f755013f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:44.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.500+0000 7f7556e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75501469f0 0x7f755013fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:44.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.500+0000 7f7556e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75500cd5f0 con 0x7f75500d0d00 2024-04-28T13:32:44.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.500+0000 7f7556e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75501617e0 con 0x7f75501469f0 2024-04-28T13:32:44.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.500+0000 7f7556e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7550072020 con 0x7f75500b5dd0 2024-04-28T13:32:44.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.500+0000 7f7547400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75500d0d00 0x7f755013f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:44.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.501+0000 7f7547400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75500d0d00 0x7f755013f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43222/0 (socket says 172.21.15.196:43222) 2024-04-28T13:32:44.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.501+0000 7f7547400640 1 -- 172.21.15.196:0/3042002466 learned_addr learned my addr 172.21.15.196:0/3042002466 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:44.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.501+0000 7f7554e00640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75501469f0 0x7f755013fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.501+0000 7f754fe00640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75500b5dd0 0x7f7550157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.501+0000 7f7547400640 1 -- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75500b5dd0 msgr2=0x7f7550157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.501+0000 7f7547400640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75500b5dd0 0x7f7550157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.501+0000 7f7547400640 1 -- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75501469f0 msgr2=0x7f755013fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.501+0000 7f7547400640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75501469f0 0x7f755013fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.501+0000 7f7547400640 1 -- 172.21.15.196:0/3042002466 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755015cbc0 con 0x7f75500d0d00 2024-04-28T13:32:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.501+0000 7f7554e00640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75501469f0 0x7f755013fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.501+0000 7f754fe00640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75500b5dd0 0x7f7550157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.501+0000 7f7547400640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75500d0d00 0x7f755013f6b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f753800ba50 tx=0x7f753800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.502+0000 7f754cc00640 1 -- 172.21.15.196:0/3042002466 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538019070 con 0x7f75500d0d00 2024-04-28T13:32:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.502+0000 7f754cc00640 1 -- 172.21.15.196:0/3042002466 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f753800cc00 con 0x7f75500d0d00 2024-04-28T13:32:44.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.502+0000 7f754cc00640 1 -- 172.21.15.196:0/3042002466 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538014440 con 0x7f75500d0d00 2024-04-28T13:32:44.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.502+0000 7f7556e00640 1 -- 172.21.15.196:0/3042002466 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755015c600 con 0x7f75500d0d00 2024-04-28T13:32:44.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.502+0000 7f7556e00640 1 -- 172.21.15.196:0/3042002466 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75500b0980 con 0x7f75500d0d00 2024-04-28T13:32:44.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.503+0000 7f7556e00640 1 -- 172.21.15.196:0/3042002466 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f755006a290 con 0x7f75500d0d00 2024-04-28T13:32:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.504+0000 7f754cc00640 1 -- 172.21.15.196:0/3042002466 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f753801c070 con 0x7f75500d0d00 2024-04-28T13:32:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.505+0000 7f754cc00640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f752c06c8f0 0x7f752c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.505+0000 7f754cc00640 1 -- 172.21.15.196:0/3042002466 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f753808b8e0 con 0x7f75500d0d00 2024-04-28T13:32:44.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.505+0000 7f754fe00640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f752c06c8f0 0x7f752c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:44.506 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.506+0000 7f754fe00640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f752c06c8f0 0x7f752c06edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f754000a810 tx=0x7f7540008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:44.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.509+0000 7f754cc00640 1 -- 172.21.15.196:0/3042002466 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7538059d10 con 0x7f75500d0d00 2024-04-28T13:32:44.606 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.606+0000 7f754cc00640 1 -- 172.21.15.196:0/3042002466 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7538055c80 con 0x7f75500d0d00 2024-04-28T13:32:44.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.720+0000 7f7556e00640 1 -- 172.21.15.196:0/3042002466 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f755006fd90 con 0x7f75500d0d00 2024-04-28T13:32:44.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.722+0000 7f754cc00640 1 -- 172.21.15.196:0/3042002466 <== mon.0 v2:172.21.15.196:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16705 (secure 0 0 0) 0x7f7538007370 con 0x7f75500d0d00 2024-04-28T13:32:44.723 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:44.723 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":33,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:33.217952+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:32:29.352348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-28T13:32:30.637498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:32:44.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.725+0000 7f7556e00640 1 -- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f752c06c8f0 msgr2=0x7f752c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:44.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.725+0000 7f7556e00640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f752c06c8f0 0x7f752c06edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f754000a810 tx=0x7f7540008040 comp rx=0 tx=0).stop 2024-04-28T13:32:44.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.725+0000 7f7556e00640 1 -- 172.21.15.196:0/3042002466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75500d0d00 msgr2=0x7f755013f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:44.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.725+0000 7f7556e00640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75500d0d00 0x7f755013f6b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f753800ba50 tx=0x7f753800bf10 comp rx=0 tx=0).stop 2024-04-28T13:32:44.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.725+0000 7f7556e00640 1 -- 172.21.15.196:0/3042002466 shutdown_connections 2024-04-28T13:32:44.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.725+0000 7f7556e00640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f752c06c8f0 0x7f752c06edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.725+0000 7f7556e00640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75501469f0 0x7f755013fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.725+0000 7f7556e00640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75500d0d00 0x7f755013f6b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.725+0000 7f7556e00640 1 --2- 172.21.15.196:0/3042002466 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75500b5dd0 0x7f7550157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.725+0000 7f7556e00640 1 -- 172.21.15.196:0/3042002466 >> 172.21.15.196:0/3042002466 conn(0x7f75500cd830 msgr2=0x7f75500d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:44.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.725+0000 7f7556e00640 1 -- 172.21.15.196:0/3042002466 shutdown_connections 2024-04-28T13:32:44.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.725+0000 7f7556e00640 1 -- 172.21.15.196:0/3042002466 wait complete. 2024-04-28T13:32:44.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:32:29.352348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:32:30.637498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:32:44.745 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:32:44.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.951+0000 7f515b400640 1 Processor -- start 2024-04-28T13:32:44.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.952+0000 7f515b400640 1 -- start start 2024-04-28T13:32:44.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.953+0000 7f515b400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5154145960 0x7f51540af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:44.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.953+0000 7f515b400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f51540afdb0 0x7f51540b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:44.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.953+0000 7f515b400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51541607d0 con 0x7f515415fa10 2024-04-28T13:32:44.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.953+0000 7f515b400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f51541605b0 con 0x7f51540afdb0 2024-04-28T13:32:44.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.953+0000 7f515b400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f51540aea50 con 0x7f5154145960 2024-04-28T13:32:44.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.954+0000 7f5159400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f515415fa10 0x7f515415fea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:40456/0 (socket says 172.21.15.196:40456) 2024-04-28T13:32:44.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.954+0000 7f5159400640 1 -- 172.21.15.196:0/3284985257 learned_addr learned my addr 172.21.15.196:0/3284985257 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:44.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.954+0000 7f5158a00640 1 --2- 172.21.15.196:0/3284985257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5154145960 0x7f51540af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:44.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.954+0000 7f5153e00640 1 --2- 172.21.15.196:0/3284985257 >> v2:172.21.15.203:3300/0 conn(0x7f51540afdb0 0x7f51540b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:44.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.954+0000 7f5153400640 1 -- 172.21.15.196:0/3284985257 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 379847050 0 0) 0x7f51541607d0 con 0x7f515415fa10 2024-04-28T13:32:44.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.954+0000 7f5153400640 1 -- 172.21.15.196:0/3284985257 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5138003520 con 0x7f515415fa10 2024-04-28T13:32:44.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.954+0000 7f5153400640 1 -- 172.21.15.196:0/3284985257 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2340142708 0 0) 0x7f5138003520 con 0x7f515415fa10 2024-04-28T13:32:44.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.954+0000 7f5153400640 1 -- 172.21.15.196:0/3284985257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5154145960 msgr2=0x7f51540af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:44.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.954+0000 7f5153400640 1 --2- 172.21.15.196:0/3284985257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5154145960 0x7f51540af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5153400640 1 -- 172.21.15.196:0/3284985257 >> v2:172.21.15.203:3300/0 conn(0x7f51540afdb0 msgr2=0x7f51540b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:44.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5153400640 1 --2- 172.21.15.196:0/3284985257 >> v2:172.21.15.203:3300/0 conn(0x7f51540afdb0 0x7f51540b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5153400640 1 -- 172.21.15.196:0/3284985257 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51540aed30 con 0x7f515415fa10 2024-04-28T13:32:44.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5153400640 1 -- 172.21.15.196:0/3284985257 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f514c0019f0 con 0x7f515415fa10 2024-04-28T13:32:44.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5153400640 1 -- 172.21.15.196:0/3284985257 >> v1:172.21.15.196:6789/0 conn(0x7f515415fa10 legacy=0x7f515415fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:44.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5153400640 1 --2- 172.21.15.196:0/3284985257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5154145960 0x7f51380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:44.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5153400640 1 --2- 172.21.15.196:0/3284985257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5138003f30 0x7f5138004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:44.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5153400640 1 --2- 172.21.15.196:0/3284985257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138004d40 0x7f51380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:44.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5153400640 1 -- 172.21.15.196:0/3284985257 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51541605b0 con 0x7f5138004d40 2024-04-28T13:32:44.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5153400640 1 -- 172.21.15.196:0/3284985257 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f51540aea50 con 0x7f5138003f30 2024-04-28T13:32:44.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5153400640 1 -- 172.21.15.196:0/3284985257 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f513800b9c0 con 0x7f5154145960 2024-04-28T13:32:44.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5159400640 1 --2- 172.21.15.196:0/3284985257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138004d40 0x7f51380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:44.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5153e00640 1 --2- 172.21.15.196:0/3284985257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5138003f30 0x7f5138004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:44.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5159400640 1 -- 172.21.15.196:0/3284985257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5154145960 msgr2=0x7f51380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:44.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5159400640 1 --2- 172.21.15.196:0/3284985257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5154145960 0x7f51380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5159400640 1 -- 172.21.15.196:0/3284985257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5138003f30 msgr2=0x7f5138004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:44.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5159400640 1 --2- 172.21.15.196:0/3284985257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5138003f30 0x7f5138004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5159400640 1 -- 172.21.15.196:0/3284985257 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5138003520 con 0x7f5138004d40 2024-04-28T13:32:44.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.955+0000 7f5153e00640 1 --2- 172.21.15.196:0/3284985257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5138003f30 0x7f5138004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:44.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.956+0000 7f5159400640 1 --2- 172.21.15.196:0/3284985257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138004d40 0x7f51380051c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f514c00a870 tx=0x7f5138005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb2f332b22e295b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:44.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.957+0000 7f5153400640 1 -- 172.21.15.196:0/3284985257 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c00a4c0 con 0x7f5138004d40 2024-04-28T13:32:44.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.957+0000 7f5153400640 1 -- 172.21.15.196:0/3284985257 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f514c02f070 con 0x7f5138004d40 2024-04-28T13:32:44.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.957+0000 7f5153400640 1 -- 172.21.15.196:0/3284985257 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c02bcb0 con 0x7f5138004d40 2024-04-28T13:32:44.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.957+0000 7f515b400640 1 -- 172.21.15.196:0/3284985257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138004d40 msgr2=0x7f51380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:44.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.957+0000 7f515b400640 1 --2- 172.21.15.196:0/3284985257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5138004d40 0x7f51380051c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f514c00a870 tx=0x7f5138005830 comp rx=0 tx=0).stop 2024-04-28T13:32:44.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.957+0000 7f5159400640 1 -- 172.21.15.196:0/3284985257 reap_dead start 2024-04-28T13:32:44.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.957+0000 7f515b400640 1 -- 172.21.15.196:0/3284985257 shutdown_connections 2024-04-28T13:32:44.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.957+0000 7f515b400640 1 -- 172.21.15.196:0/3284985257 >> 172.21.15.196:0/3284985257 conn(0x7f51540cd8a0 msgr2=0x7f51540cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:44.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.958+0000 7f515b400640 1 -- 172.21.15.196:0/3284985257 shutdown_connections 2024-04-28T13:32:44.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.958+0000 7f515b400640 1 -- 172.21.15.196:0/3284985257 wait complete. 2024-04-28T13:32:44.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.959+0000 7f515b400640 1 Processor -- start 2024-04-28T13:32:44.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.960+0000 7f515b400640 1 -- start start 2024-04-28T13:32:44.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.961+0000 7f515b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5138004d40 0x7f51540c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:44.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.961+0000 7f515b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f515415e9b0 0x7f51541353f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:44.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.961+0000 7f515b400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5154136c70 0x7f51540cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:44.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.961+0000 7f515b400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51540af530 con 0x7f515415e9b0 2024-04-28T13:32:44.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.961+0000 7f515b400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f51540b0750 con 0x7f5138004d40 2024-04-28T13:32:44.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.961+0000 7f515b400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f51540af6b0 con 0x7f5154136c70 2024-04-28T13:32:44.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.961+0000 7f5153e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f515415e9b0 0x7f51541353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:44.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.961+0000 7f5153e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f515415e9b0 0x7f51541353f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43234/0 (socket says 172.21.15.196:43234) 2024-04-28T13:32:44.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.961+0000 7f5153e00640 1 -- 172.21.15.196:0/4220590940 learned_addr learned my addr 172.21.15.196:0/4220590940 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:44.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.961+0000 7f5153e00640 1 -- 172.21.15.196:0/4220590940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5154136c70 msgr2=0x7f51540cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:44.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.961+0000 7f5153e00640 1 --2- 172.21.15.196:0/4220590940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5154136c70 0x7f51540cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.961+0000 7f5158a00640 1 --2- 172.21.15.196:0/4220590940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5138004d40 0x7f51540c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:44.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.961+0000 7f5153e00640 1 -- 172.21.15.196:0/4220590940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5138004d40 msgr2=0x7f51540c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:44.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.961+0000 7f5153e00640 1 --2- 172.21.15.196:0/4220590940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5138004d40 0x7f51540c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:44.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.961+0000 7f5153e00640 1 -- 172.21.15.196:0/4220590940 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5154135930 con 0x7f515415e9b0 2024-04-28T13:32:44.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.962+0000 7f5153e00640 1 --2- 172.21.15.196:0/4220590940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f515415e9b0 0x7f51541353f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f513c010950 tx=0x7f513c010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:44.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.962+0000 7f5150c00640 1 -- 172.21.15.196:0/4220590940 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c00f040 con 0x7f515415e9b0 2024-04-28T13:32:44.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.962+0000 7f5150c00640 1 -- 172.21.15.196:0/4220590940 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f513c00eb60 con 0x7f515415e9b0 2024-04-28T13:32:44.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.962+0000 7f5150c00640 1 -- 172.21.15.196:0/4220590940 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c01c400 con 0x7f515415e9b0 2024-04-28T13:32:44.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.962+0000 7f515b400640 1 -- 172.21.15.196:0/4220590940 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515415ee30 con 0x7f515415e9b0 2024-04-28T13:32:44.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.962+0000 7f515b400640 1 -- 172.21.15.196:0/4220590940 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5154135eb0 con 0x7f515415e9b0 2024-04-28T13:32:44.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.963+0000 7f515b400640 1 -- 172.21.15.196:0/4220590940 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515406a290 con 0x7f515415e9b0 2024-04-28T13:32:44.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.964+0000 7f5150c00640 1 -- 172.21.15.196:0/4220590940 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f513c01c5a0 con 0x7f515415e9b0 2024-04-28T13:32:44.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.966+0000 7f5150c00640 1 --2- 172.21.15.196:0/4220590940 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f513006c9b0 0x7f513006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:44.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.966+0000 7f5150c00640 1 -- 172.21.15.196:0/4220590940 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f513c08c340 con 0x7f515415e9b0 2024-04-28T13:32:44.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.966+0000 7f5158a00640 1 --2- 172.21.15.196:0/4220590940 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f513006c9b0 0x7f513006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:44.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.967+0000 7f5158a00640 1 --2- 172.21.15.196:0/4220590940 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f513006c9b0 0x7f513006ee70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5144004770 tx=0x7f5144008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:44.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:44.971+0000 7f5150c00640 1 -- 172.21.15.196:0/4220590940 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f513c018030 con 0x7f515415e9b0 2024-04-28T13:32:45.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.173+0000 7f515b400640 1 -- 172.21.15.196:0/4220590940 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5154072240 con 0x7f515415e9b0 2024-04-28T13:32:45.174 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.174+0000 7f5150c00640 1 -- 172.21.15.196:0/4220590940 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f513c05a710 con 0x7f515415e9b0 2024-04-28T13:32:45.174 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:32:45.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.177+0000 7f515b400640 1 -- 172.21.15.196:0/4220590940 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f513006c9b0 msgr2=0x7f513006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:45.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.177+0000 7f515b400640 1 --2- 172.21.15.196:0/4220590940 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f513006c9b0 0x7f513006ee70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5144004770 tx=0x7f5144008040 comp rx=0 tx=0).stop 2024-04-28T13:32:45.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.177+0000 7f515b400640 1 -- 172.21.15.196:0/4220590940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f515415e9b0 msgr2=0x7f51541353f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:45.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.177+0000 7f515b400640 1 --2- 172.21.15.196:0/4220590940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f515415e9b0 0x7f51541353f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f513c010950 tx=0x7f513c010e10 comp rx=0 tx=0).stop 2024-04-28T13:32:45.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.177+0000 7f515b400640 1 -- 172.21.15.196:0/4220590940 shutdown_connections 2024-04-28T13:32:45.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.177+0000 7f515b400640 1 --2- 172.21.15.196:0/4220590940 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f513006c9b0 0x7f513006ee70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.177+0000 7f515b400640 1 --2- 172.21.15.196:0/4220590940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5154136c70 0x7f51540cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.177+0000 7f515b400640 1 --2- 172.21.15.196:0/4220590940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f515415e9b0 0x7f51541353f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.177+0000 7f515b400640 1 --2- 172.21.15.196:0/4220590940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5138004d40 0x7f51540c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.177+0000 7f515b400640 1 -- 172.21.15.196:0/4220590940 >> 172.21.15.196:0/4220590940 conn(0x7f51540cd8a0 msgr2=0x7f5154161570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:45.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.177+0000 7f515b400640 1 -- 172.21.15.196:0/4220590940 shutdown_connections 2024-04-28T13:32:45.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.177+0000 7f515b400640 1 -- 172.21.15.196:0/4220590940 wait complete. 2024-04-28T13:32:45.197 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:32:45.400 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.400+0000 7feb11e00640 1 Processor -- start 2024-04-28T13:32:45.400 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.401+0000 7feb11e00640 1 -- start start 2024-04-28T13:32:45.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.402+0000 7feb11e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7feb0c15f5b0 0x7feb0c0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:45.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.402+0000 7feb11e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb0c0b6aa0 con 0x7feb0c15f990 2024-04-28T13:32:45.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.402+0000 7feb11e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb0c0d3eb0 con 0x7feb0c0b55b0 2024-04-28T13:32:45.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.402+0000 7feb11e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7feb0c0d4190 con 0x7feb0c15f5b0 2024-04-28T13:32:45.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.402+0000 7feb0aa00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7feb0c15f5b0 0x7feb0c0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:45.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.402+0000 7feb0aa00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7feb0c15f5b0 0x7feb0c0b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43240/0 (socket says 172.21.15.196:43240) 2024-04-28T13:32:45.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.402+0000 7feb0aa00640 1 -- 172.21.15.196:0/4157484605 learned_addr learned my addr 172.21.15.196:0/4157484605 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:45.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.402+0000 7feb0a000640 1 --1- 172.21.15.196:0/4157484605 >> v1:172.21.15.196:6789/0 conn(0x7feb0c15f990 0x7feb0c0d0a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:40464/0 (socket says 172.21.15.196:40464) 2024-04-28T13:32:45.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.403+0000 7feb0aa00640 1 -- 172.21.15.196:0/4157484605 >> v1:172.21.15.203:6789/0 conn(0x7feb0c0b55b0 legacy=0x7feb0c0d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:45.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.403+0000 7feb0aa00640 1 -- 172.21.15.196:0/4157484605 >> v1:172.21.15.196:6789/0 conn(0x7feb0c15f990 legacy=0x7feb0c0d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:45.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.403+0000 7feb0aa00640 1 -- 172.21.15.196:0/4157484605 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c0d0200 con 0x7feb0c15f5b0 2024-04-28T13:32:45.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.403+0000 7feb09600640 1 -- 172.21.15.196:0/4157484605 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1698895459 0 0) 0x7feb0c0b6aa0 con 0x7feb0c15f990 2024-04-28T13:32:45.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.404+0000 7feb0aa00640 1 --2- 172.21.15.196:0/4157484605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb0c15f5b0 0x7feb0c0b5070 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feafc009870 tx=0x7feafc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2364c86fcdb3a655 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:45.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.404+0000 7feb09600640 1 -- 172.21.15.196:0/4157484605 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc037070 con 0x7feb0c15f5b0 2024-04-28T13:32:45.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.404+0000 7feb09600640 1 -- 172.21.15.196:0/4157484605 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feafc032440 con 0x7feb0c15f5b0 2024-04-28T13:32:45.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.404+0000 7feb09600640 1 -- 172.21.15.196:0/4157484605 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc032810 con 0x7feb0c15f5b0 2024-04-28T13:32:45.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.405+0000 7feb11e00640 1 -- 172.21.15.196:0/4157484605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb0c15f5b0 msgr2=0x7feb0c0b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:45.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.405+0000 7feb11e00640 1 --2- 172.21.15.196:0/4157484605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb0c15f5b0 0x7feb0c0b5070 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feafc009870 tx=0x7feafc02a210 comp rx=0 tx=0).stop 2024-04-28T13:32:45.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.405+0000 7feb11e00640 1 -- 172.21.15.196:0/4157484605 shutdown_connections 2024-04-28T13:32:45.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.405+0000 7feb11e00640 1 --2- 172.21.15.196:0/4157484605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb0c15f5b0 0x7feb0c0b5070 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.405+0000 7feb11e00640 1 -- 172.21.15.196:0/4157484605 >> 172.21.15.196:0/4157484605 conn(0x7feb0c08ec40 msgr2=0x7feb0c09e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:45.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.405+0000 7feb11e00640 1 -- 172.21.15.196:0/4157484605 shutdown_connections 2024-04-28T13:32:45.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.405+0000 7feb11e00640 1 -- 172.21.15.196:0/4157484605 wait complete. 2024-04-28T13:32:45.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.407+0000 7feb11e00640 1 Processor -- start 2024-04-28T13:32:45.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.407+0000 7feb11e00640 1 -- start start 2024-04-28T13:32:45.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.408+0000 7feb11e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb0c0b55b0 0x7feb0c1f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:45.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.408+0000 7feb11e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb0c15f5b0 0x7feb0c0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:45.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.408+0000 7feb11e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb0c15f990 0x7feb0c153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:45.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.408+0000 7feb11e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c0d1290 con 0x7feb0c15f5b0 2024-04-28T13:32:45.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.408+0000 7feb11e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c0d1110 con 0x7feb0c0b55b0 2024-04-28T13:32:45.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.408+0000 7feb11e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feb0c0d3a50 con 0x7feb0c15f990 2024-04-28T13:32:45.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.408+0000 7feb0a000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb0c15f5b0 0x7feb0c0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:45.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.408+0000 7feb0a000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb0c15f5b0 0x7feb0c0cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43250/0 (socket says 172.21.15.196:43250) 2024-04-28T13:32:45.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.408+0000 7feb0a000640 1 -- 172.21.15.196:0/461922623 learned_addr learned my addr 172.21.15.196:0/461922623 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:45.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.409+0000 7feb0aa00640 1 --2- 172.21.15.196:0/461922623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb0c0b55b0 0x7feb0c1f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:45.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.409+0000 7feb0a000640 1 -- 172.21.15.196:0/461922623 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb0c15f990 msgr2=0x7feb0c153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:45.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.409+0000 7feb0a000640 1 --2- 172.21.15.196:0/461922623 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb0c15f990 0x7feb0c153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.409+0000 7feb0a000640 1 -- 172.21.15.196:0/461922623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb0c0b55b0 msgr2=0x7feb0c1f89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:45.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.409+0000 7feb0a000640 1 --2- 172.21.15.196:0/461922623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb0c0b55b0 0x7feb0c1f89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.409+0000 7feb0a000640 1 -- 172.21.15.196:0/461922623 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c160b00 con 0x7feb0c15f5b0 2024-04-28T13:32:45.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.409+0000 7feb0a000640 1 --2- 172.21.15.196:0/461922623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb0c15f5b0 0x7feb0c0cc300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7feaf400c7f0 tx=0x7feaf400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:45.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.409+0000 7feb03400640 1 -- 172.21.15.196:0/461922623 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4018070 con 0x7feb0c15f5b0 2024-04-28T13:32:45.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.409+0000 7feb03400640 1 -- 172.21.15.196:0/461922623 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf400a640 con 0x7feb0c15f5b0 2024-04-28T13:32:45.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.409+0000 7feb03400640 1 -- 172.21.15.196:0/461922623 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf400aa30 con 0x7feb0c15f5b0 2024-04-28T13:32:45.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.409+0000 7feb11e00640 1 -- 172.21.15.196:0/461922623 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0c160de0 con 0x7feb0c15f5b0 2024-04-28T13:32:45.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.410+0000 7feb11e00640 1 -- 172.21.15.196:0/461922623 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0c1613a0 con 0x7feb0c15f5b0 2024-04-28T13:32:45.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.411+0000 7feb11e00640 1 -- 172.21.15.196:0/461922623 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb0c06a290 con 0x7feb0c15f5b0 2024-04-28T13:32:45.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.412+0000 7feb03400640 1 -- 172.21.15.196:0/461922623 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7feaf400abd0 con 0x7feb0c15f5b0 2024-04-28T13:32:45.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.413+0000 7feb03400640 1 --2- 172.21.15.196:0/461922623 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feae806c980 0x7feae806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:45.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.413+0000 7feb03400640 1 -- 172.21.15.196:0/461922623 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7feaf408b1b0 con 0x7feb0c15f5b0 2024-04-28T13:32:45.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.414+0000 7feb0aa00640 1 --2- 172.21.15.196:0/461922623 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feae806c980 0x7feae806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:45.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.415+0000 7feb0aa00640 1 --2- 172.21.15.196:0/461922623 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feae806c980 0x7feae806ee40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feafc034000 tx=0x7feafc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:45.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.418+0000 7feb03400640 1 -- 172.21.15.196:0/461922623 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feaf4059580 con 0x7feb0c15f5b0 2024-04-28T13:32:45.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.622+0000 7feb11e00640 1 -- 172.21.15.196:0/461922623 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feb0c1635c0 con 0x7feb0c15f5b0 2024-04-28T13:32:45.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.622+0000 7feb03400640 1 -- 172.21.15.196:0/461922623 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7feaf4058e80 con 0x7feb0c15f5b0 2024-04-28T13:32:45.622 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:32:45.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.625+0000 7feb11e00640 1 -- 172.21.15.196:0/461922623 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feae806c980 msgr2=0x7feae806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:45.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.625+0000 7feb11e00640 1 --2- 172.21.15.196:0/461922623 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feae806c980 0x7feae806ee40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feafc034000 tx=0x7feafc02e000 comp rx=0 tx=0).stop 2024-04-28T13:32:45.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.625+0000 7feb11e00640 1 -- 172.21.15.196:0/461922623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb0c15f5b0 msgr2=0x7feb0c0cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:45.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.625+0000 7feb11e00640 1 --2- 172.21.15.196:0/461922623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb0c15f5b0 0x7feb0c0cc300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7feaf400c7f0 tx=0x7feaf400ccb0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.625+0000 7feb11e00640 1 -- 172.21.15.196:0/461922623 shutdown_connections 2024-04-28T13:32:45.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.625+0000 7feb11e00640 1 --2- 172.21.15.196:0/461922623 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feae806c980 0x7feae806ee40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.625+0000 7feb11e00640 1 --2- 172.21.15.196:0/461922623 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb0c15f990 0x7feb0c153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.625+0000 7feb11e00640 1 --2- 172.21.15.196:0/461922623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb0c15f5b0 0x7feb0c0cc300 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.625+0000 7feb11e00640 1 --2- 172.21.15.196:0/461922623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb0c0b55b0 0x7feb0c1f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.625+0000 7feb11e00640 1 -- 172.21.15.196:0/461922623 >> 172.21.15.196:0/461922623 conn(0x7feb0c08ec40 msgr2=0x7feb0c06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:45.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.625+0000 7feb11e00640 1 -- 172.21.15.196:0/461922623 shutdown_connections 2024-04-28T13:32:45.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.625+0000 7feb11e00640 1 -- 172.21.15.196:0/461922623 wait complete. 2024-04-28T13:32:45.644 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:32:45.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.847+0000 7fa0fe200640 1 Processor -- start 2024-04-28T13:32:45.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.847+0000 7fa0fe200640 1 -- start start 2024-04-28T13:32:45.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.849+0000 7fa0fe200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa0f80add70 0x7fa0f8073af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:45.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.849+0000 7fa0fe200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa0f80cff00 0x7fa0f80d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:45.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.849+0000 7fa0fe200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0f80b05b0 con 0x7fa0f815cff0 2024-04-28T13:32:45.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.849+0000 7fa0fe200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa0f80d0870 con 0x7fa0f80add70 2024-04-28T13:32:45.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.849+0000 7fa0fe200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa0f8074180 con 0x7fa0f80cff00 2024-04-28T13:32:45.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.849+0000 7fa0f7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa0f80add70 0x7fa0f8073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:45.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.849+0000 7fa0f7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa0f80add70 0x7fa0f8073af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43252/0 (socket says 172.21.15.196:43252) 2024-04-28T13:32:45.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.849+0000 7fa0f7e00640 1 -- 172.21.15.196:0/891057068 learned_addr learned my addr 172.21.15.196:0/891057068 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:45.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.849+0000 7fa0fd800640 1 --2- 172.21.15.196:0/891057068 >> v2:172.21.15.203:3300/0 conn(0x7fa0f80cff00 0x7fa0f80d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:45.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.850+0000 7fa0f7e00640 1 -- 172.21.15.196:0/891057068 >> v2:172.21.15.203:3300/0 conn(0x7fa0f80cff00 msgr2=0x7fa0f80d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:45.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.850+0000 7fa0f7e00640 1 --2- 172.21.15.196:0/891057068 >> v2:172.21.15.203:3300/0 conn(0x7fa0f80cff00 0x7fa0f80d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.850+0000 7fa0f7e00640 1 -- 172.21.15.196:0/891057068 >> v1:172.21.15.203:6789/0 conn(0x7fa0f815cff0 legacy=0x7fa0f8072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:45.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.850+0000 7fa0f7e00640 1 -- 172.21.15.196:0/891057068 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f8071ef0 con 0x7fa0f80add70 2024-04-28T13:32:45.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.851+0000 7fa0f7e00640 1 --2- 172.21.15.196:0/891057068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0f80add70 0x7fa0f8073af0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8161cb0 tx=0x7fa0e402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a2361a2e965b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:45.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.851+0000 7fa0f7400640 1 -- 172.21.15.196:0/891057068 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4037070 con 0x7fa0f80add70 2024-04-28T13:32:45.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.851+0000 7fa0f7400640 1 -- 172.21.15.196:0/891057068 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e4032440 con 0x7fa0f80add70 2024-04-28T13:32:45.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.851+0000 7fa0f7400640 1 -- 172.21.15.196:0/891057068 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4032810 con 0x7fa0f80add70 2024-04-28T13:32:45.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.852+0000 7fa0fe200640 1 -- 172.21.15.196:0/891057068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0f80add70 msgr2=0x7fa0f8073af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:45.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.852+0000 7fa0fe200640 1 --2- 172.21.15.196:0/891057068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0f80add70 0x7fa0f8073af0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8161cb0 tx=0x7fa0e402a1a0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.852+0000 7fa0fe200640 1 -- 172.21.15.196:0/891057068 shutdown_connections 2024-04-28T13:32:45.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.852+0000 7fa0fe200640 1 --2- 172.21.15.196:0/891057068 >> v2:172.21.15.203:3300/0 conn(0x7fa0f80cff00 0x7fa0f80d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.852+0000 7fa0fe200640 1 --2- 172.21.15.196:0/891057068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0f80add70 0x7fa0f8073af0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.852+0000 7fa0fe200640 1 -- 172.21.15.196:0/891057068 >> 172.21.15.196:0/891057068 conn(0x7fa0f80b2990 msgr2=0x7fa0f80b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:45.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.852+0000 7fa0fe200640 1 -- 172.21.15.196:0/891057068 shutdown_connections 2024-04-28T13:32:45.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.852+0000 7fa0fe200640 1 -- 172.21.15.196:0/891057068 wait complete. 2024-04-28T13:32:45.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.854+0000 7fa0fe200640 1 Processor -- start 2024-04-28T13:32:45.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.854+0000 7fa0fe200640 1 -- start start 2024-04-28T13:32:45.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.854+0000 7fa0fe200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0f80add70 0x7fa0f8144260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:45.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.855+0000 7fa0fe200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0f80cff00 0x7fa0f815d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:45.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.855+0000 7fa0fe200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0f815cff0 0x7fa0f8157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:45.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.855+0000 7fa0fe200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f8072790 con 0x7fa0f80add70 2024-04-28T13:32:45.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.855+0000 7fa0fe200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f80724b0 con 0x7fa0f815cff0 2024-04-28T13:32:45.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.855+0000 7fa0fe200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f815d3d0 con 0x7fa0f80cff00 2024-04-28T13:32:45.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.855+0000 7fa0fce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0f80add70 0x7fa0f8144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:45.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.855+0000 7fa0fce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0f80add70 0x7fa0f8144260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43260/0 (socket says 172.21.15.196:43260) 2024-04-28T13:32:45.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.855+0000 7fa0fce00640 1 -- 172.21.15.196:0/391432635 learned_addr learned my addr 172.21.15.196:0/391432635 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:45.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.855+0000 7fa0fd800640 1 --2- 172.21.15.196:0/391432635 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0f815cff0 0x7fa0f8157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:45.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.855+0000 7fa0f7e00640 1 --2- 172.21.15.196:0/391432635 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0f80cff00 0x7fa0f815d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:45.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.855+0000 7fa0fce00640 1 -- 172.21.15.196:0/391432635 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0f80cff00 msgr2=0x7fa0f815d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:45.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.855+0000 7fa0fce00640 1 --2- 172.21.15.196:0/391432635 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0f80cff00 0x7fa0f815d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.855+0000 7fa0fce00640 1 -- 172.21.15.196:0/391432635 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0f815cff0 msgr2=0x7fa0f8157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:45.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.855+0000 7fa0fce00640 1 --2- 172.21.15.196:0/391432635 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0f815cff0 0x7fa0f8157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:45.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.855+0000 7fa0fce00640 1 -- 172.21.15.196:0/391432635 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f809e0c0 con 0x7fa0f80add70 2024-04-28T13:32:45.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.856+0000 7fa0f7e00640 1 --2- 172.21.15.196:0/391432635 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0f80cff00 0x7fa0f815d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:45.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.856+0000 7fa0fce00640 1 --2- 172.21.15.196:0/391432635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0f80add70 0x7fa0f8144260 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00b420 tx=0x7fa0ec00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:45.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.856+0000 7fa0f4c00640 1 -- 172.21.15.196:0/391432635 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec011830 con 0x7fa0f80add70 2024-04-28T13:32:45.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.856+0000 7fa0f4c00640 1 -- 172.21.15.196:0/391432635 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ec016070 con 0x7fa0f80add70 2024-04-28T13:32:45.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.856+0000 7fa0fe200640 1 -- 172.21.15.196:0/391432635 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f815db80 con 0x7fa0f80add70 2024-04-28T13:32:45.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.856+0000 7fa0f4c00640 1 -- 172.21.15.196:0/391432635 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec011c10 con 0x7fa0f80add70 2024-04-28T13:32:45.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.856+0000 7fa0fe200640 1 -- 172.21.15.196:0/391432635 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f809e360 con 0x7fa0f80add70 2024-04-28T13:32:45.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.857+0000 7fa0fe200640 1 -- 172.21.15.196:0/391432635 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f8050730 con 0x7fa0f80add70 2024-04-28T13:32:45.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.858+0000 7fa0f4c00640 1 -- 172.21.15.196:0/391432635 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa0ec01b460 con 0x7fa0f80add70 2024-04-28T13:32:45.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.859+0000 7fa0f4c00640 1 --2- 172.21.15.196:0/391432635 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa0d806c980 0x7fa0d806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:45.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.859+0000 7fa0f4c00640 1 -- 172.21.15.196:0/391432635 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa0ec08b730 con 0x7fa0f80add70 2024-04-28T13:32:45.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.860+0000 7fa0f7e00640 1 --2- 172.21.15.196:0/391432635 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa0d806c980 0x7fa0d806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:45.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.861+0000 7fa0f7e00640 1 --2- 172.21.15.196:0/391432635 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa0d806c980 0x7fa0d806ee40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4034000 tx=0x7fa0e402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:45.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:45.864+0000 7fa0f4c00640 1 -- 172.21.15.196:0/391432635 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa0ec020020 con 0x7fa0f80add70 2024-04-28T13:32:46.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.062+0000 7fa0fe200640 1 -- 172.21.15.196:0/391432635 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa0f806e3a0 con 0x7fa0f80add70 2024-04-28T13:32:46.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.063+0000 7fa0f4c00640 1 -- 172.21.15.196:0/391432635 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7fa0ec059a70 con 0x7fa0f80add70 2024-04-28T13:32:46.063 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:32:46.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.066+0000 7fa0fe200640 1 -- 172.21.15.196:0/391432635 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa0d806c980 msgr2=0x7fa0d806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:46.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.066+0000 7fa0fe200640 1 --2- 172.21.15.196:0/391432635 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa0d806c980 0x7fa0d806ee40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4034000 tx=0x7fa0e402e000 comp rx=0 tx=0).stop 2024-04-28T13:32:46.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.066+0000 7fa0fe200640 1 -- 172.21.15.196:0/391432635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0f80add70 msgr2=0x7fa0f8144260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:46.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.066+0000 7fa0fe200640 1 --2- 172.21.15.196:0/391432635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0f80add70 0x7fa0f8144260 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00b420 tx=0x7fa0ec00b8e0 comp rx=0 tx=0).stop 2024-04-28T13:32:46.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.066+0000 7fa0fe200640 1 -- 172.21.15.196:0/391432635 shutdown_connections 2024-04-28T13:32:46.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.066+0000 7fa0fe200640 1 --2- 172.21.15.196:0/391432635 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa0d806c980 0x7fa0d806ee40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:46.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.066+0000 7fa0fe200640 1 --2- 172.21.15.196:0/391432635 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0f815cff0 0x7fa0f8157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:46.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.066+0000 7fa0fe200640 1 --2- 172.21.15.196:0/391432635 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0f80cff00 0x7fa0f815d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:46.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.066+0000 7fa0fe200640 1 --2- 172.21.15.196:0/391432635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0f80add70 0x7fa0f8144260 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:46.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.066+0000 7fa0fe200640 1 -- 172.21.15.196:0/391432635 >> 172.21.15.196:0/391432635 conn(0x7fa0f80b2990 msgr2=0x7fa0f806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:46.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.066+0000 7fa0fe200640 1 -- 172.21.15.196:0/391432635 shutdown_connections 2024-04-28T13:32:46.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.066+0000 7fa0fe200640 1 -- 172.21.15.196:0/391432635 wait complete. 2024-04-28T13:32:46.085 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T13:32:46.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.287+0000 7f34d0a00640 1 Processor -- start 2024-04-28T13:32:46.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.287+0000 7f34d0a00640 1 -- start start 2024-04-28T13:32:46.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.289+0000 7f34d0a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f34cc0b5ee0 0x7f34cc0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:46.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.289+0000 7f34d0a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f34cc0d1760 0x7f34cc06df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:46.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.289+0000 7f34d0a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34cc15e380 con 0x7f34cc09ec80 2024-04-28T13:32:46.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.289+0000 7f34d0a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f34cc0b4b20 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.289+0000 7f34d0a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f34cc0b3f30 con 0x7f34cc0d1760 2024-04-28T13:32:46.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.289+0000 7f34ca800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f34cc0b5ee0 0x7f34cc0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:46.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.289+0000 7f34c9e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f34cc09ec80 0x7f34cc0d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:40478/0 (socket says 172.21.15.196:40478) 2024-04-28T13:32:46.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.289+0000 7f34c9e00640 1 -- 172.21.15.196:0/319798116 learned_addr learned my addr 172.21.15.196:0/319798116 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:46.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.289+0000 7f34cb200640 1 --2- 172.21.15.196:0/319798116 >> v2:172.21.15.203:3300/0 conn(0x7f34cc0d1760 0x7f34cc06df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:46.290 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.290+0000 7f34ca800640 1 -- 172.21.15.196:0/319798116 >> v2:172.21.15.203:3300/0 conn(0x7f34cc0d1760 msgr2=0x7f34cc06df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:46.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.290+0000 7f34ca800640 1 --2- 172.21.15.196:0/319798116 >> v2:172.21.15.203:3300/0 conn(0x7f34cc0d1760 0x7f34cc06df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:46.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.290+0000 7f34ca800640 1 -- 172.21.15.196:0/319798116 >> v1:172.21.15.196:6789/0 conn(0x7f34cc09ec80 legacy=0x7f34cc0d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:46.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.290+0000 7f34ca800640 1 -- 172.21.15.196:0/319798116 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc0b41f0 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.290+0000 7f34c9400640 1 -- 172.21.15.196:0/319798116 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3648604588 0 0) 0x7f34cc15e380 con 0x7f34cc09ec80 2024-04-28T13:32:46.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.291+0000 7f34ca800640 1 --2- 172.21.15.196:0/319798116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34cc0b5ee0 0x7f34cc0b4f40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f34cc074210 tx=0x7f34bc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88905764d674082c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:46.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.291+0000 7f34c9400640 1 -- 172.21.15.196:0/319798116 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc037070 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.291+0000 7f34c9400640 1 -- 172.21.15.196:0/319798116 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34bc032440 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.291+0000 7f34c9400640 1 -- 172.21.15.196:0/319798116 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc032810 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.291+0000 7f34d0a00640 1 -- 172.21.15.196:0/319798116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34cc0b5ee0 msgr2=0x7f34cc0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:46.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.291+0000 7f34d0a00640 1 --2- 172.21.15.196:0/319798116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34cc0b5ee0 0x7f34cc0b4f40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f34cc074210 tx=0x7f34bc02a1a0 comp rx=0 tx=0).stop 2024-04-28T13:32:46.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.292+0000 7f34d0a00640 1 -- 172.21.15.196:0/319798116 shutdown_connections 2024-04-28T13:32:46.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.292+0000 7f34d0a00640 1 --2- 172.21.15.196:0/319798116 >> v2:172.21.15.203:3300/0 conn(0x7f34cc0d1760 0x7f34cc06df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:46.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.292+0000 7f34d0a00640 1 --2- 172.21.15.196:0/319798116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34cc0b5ee0 0x7f34cc0b4f40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:46.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.292+0000 7f34d0a00640 1 -- 172.21.15.196:0/319798116 >> 172.21.15.196:0/319798116 conn(0x7f34cc06c6c0 msgr2=0x7f34cc06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:46.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.292+0000 7f34d0a00640 1 -- 172.21.15.196:0/319798116 shutdown_connections 2024-04-28T13:32:46.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.292+0000 7f34d0a00640 1 -- 172.21.15.196:0/319798116 wait complete. 2024-04-28T13:32:46.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.293+0000 7f34d0a00640 1 Processor -- start 2024-04-28T13:32:46.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.293+0000 7f34d0a00640 1 -- start start 2024-04-28T13:32:46.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.294+0000 7f34d0a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc09ec80 0x7f34cc152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:46.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.294+0000 7f34d0a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34cc0b5ee0 0x7f34cc158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:46.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.294+0000 7f34d0a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34cc0d1760 0x7f34cc1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:46.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.294+0000 7f34d0a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc0b3c20 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.294+0000 7f34d0a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc0b47b0 con 0x7f34cc0d1760 2024-04-28T13:32:46.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.294+0000 7f34d0a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f34cc0add70 con 0x7f34cc09ec80 2024-04-28T13:32:46.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.295+0000 7f34c9e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34cc0b5ee0 0x7f34cc158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:46.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.295+0000 7f34c9e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34cc0b5ee0 0x7f34cc158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43278/0 (socket says 172.21.15.196:43278) 2024-04-28T13:32:46.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.295+0000 7f34c9e00640 1 -- 172.21.15.196:0/4174520697 learned_addr learned my addr 172.21.15.196:0/4174520697 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:46.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.295+0000 7f34ca800640 1 --2- 172.21.15.196:0/4174520697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc09ec80 0x7f34cc152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:46.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.295+0000 7f34c9e00640 1 -- 172.21.15.196:0/4174520697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc09ec80 msgr2=0x7f34cc152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:46.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.295+0000 7f34c9e00640 1 --2- 172.21.15.196:0/4174520697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc09ec80 0x7f34cc152b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:46.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.295+0000 7f34c9e00640 1 -- 172.21.15.196:0/4174520697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34cc0d1760 msgr2=0x7f34cc1594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:46.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.295+0000 7f34c9e00640 1 --2- 172.21.15.196:0/4174520697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34cc0d1760 0x7f34cc1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:46.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.295+0000 7f34c9e00640 1 -- 172.21.15.196:0/4174520697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc153050 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.295+0000 7f34c9e00640 1 --2- 172.21.15.196:0/4174520697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34cc0b5ee0 0x7f34cc158fa0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f34b800c4b0 tx=0x7f34b800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:46.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.296+0000 7f34c2a00640 1 -- 172.21.15.196:0/4174520697 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8018070 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.296+0000 7f34c2a00640 1 -- 172.21.15.196:0/4174520697 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b8007500 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.296+0000 7f34d0a00640 1 -- 172.21.15.196:0/4174520697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34cc0b17a0 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.296+0000 7f34c2a00640 1 -- 172.21.15.196:0/4174520697 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8004a20 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.296+0000 7f34d0a00640 1 -- 172.21.15.196:0/4174520697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34cc0b1d60 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.297+0000 7f34d0a00640 1 -- 172.21.15.196:0/4174520697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34cc06a290 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.298+0000 7f34c2a00640 1 -- 172.21.15.196:0/4174520697 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f34b801b070 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.299+0000 7f34c2a00640 1 --2- 172.21.15.196:0/4174520697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f34ac06c9b0 0x7f34ac06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:46.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.299+0000 7f34c2a00640 1 -- 172.21.15.196:0/4174520697 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f34b808af80 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.299+0000 7f34ca800640 1 --2- 172.21.15.196:0/4174520697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f34ac06c9b0 0x7f34ac06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:46.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.300+0000 7f34ca800640 1 --2- 172.21.15.196:0/4174520697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f34ac06c9b0 0x7f34ac06ee70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f34bc034000 tx=0x7f34bc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:46.303 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.303+0000 7f34c2a00640 1 -- 172.21.15.196:0/4174520697 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f34b8059380 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.514+0000 7f34d0a00640 1 -- 172.21.15.196:0/4174520697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f34cc15bd70 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.515 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.515+0000 7f34c2a00640 1 -- 172.21.15.196:0/4174520697 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f34b8058c80 con 0x7f34cc0b5ee0 2024-04-28T13:32:46.515 INFO:teuthology.orchestra.run.smithi196.stderr:listed 0 entries 2024-04-28T13:32:46.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.518+0000 7f34d0a00640 1 -- 172.21.15.196:0/4174520697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f34ac06c9b0 msgr2=0x7f34ac06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:46.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.518+0000 7f34d0a00640 1 --2- 172.21.15.196:0/4174520697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f34ac06c9b0 0x7f34ac06ee70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f34bc034000 tx=0x7f34bc02e000 comp rx=0 tx=0).stop 2024-04-28T13:32:46.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.518+0000 7f34d0a00640 1 -- 172.21.15.196:0/4174520697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34cc0b5ee0 msgr2=0x7f34cc158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:46.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.518+0000 7f34d0a00640 1 --2- 172.21.15.196:0/4174520697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34cc0b5ee0 0x7f34cc158fa0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f34b800c4b0 tx=0x7f34b800c970 comp rx=0 tx=0).stop 2024-04-28T13:32:46.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.518+0000 7f34d0a00640 1 -- 172.21.15.196:0/4174520697 shutdown_connections 2024-04-28T13:32:46.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.518+0000 7f34d0a00640 1 --2- 172.21.15.196:0/4174520697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f34ac06c9b0 0x7f34ac06ee70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:46.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.518+0000 7f34d0a00640 1 --2- 172.21.15.196:0/4174520697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34cc0d1760 0x7f34cc1594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:46.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.518+0000 7f34d0a00640 1 --2- 172.21.15.196:0/4174520697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34cc0b5ee0 0x7f34cc158fa0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:46.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.518+0000 7f34d0a00640 1 --2- 172.21.15.196:0/4174520697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc09ec80 0x7f34cc152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:46.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.518+0000 7f34d0a00640 1 -- 172.21.15.196:0/4174520697 >> 172.21.15.196:0/4174520697 conn(0x7f34cc06c6c0 msgr2=0x7f34cc0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:46.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.518+0000 7f34d0a00640 1 -- 172.21.15.196:0/4174520697 shutdown_connections 2024-04-28T13:32:46.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:46.518+0000 7f34d0a00640 1 -- 172.21.15.196:0/4174520697 wait complete. 2024-04-28T13:32:46.535 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-28T13:32:46.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:46.535 DEBUG:teuthology.orchestra.run.smithi196:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-28T13:32:46.632 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:32:46.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:46.633 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-28T13:32:46.681 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:32:46.681 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:46.739 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:32:46.740 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:46.795 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:32:47.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.008+0000 7f1b79200640 1 Processor -- start 2024-04-28T13:32:47.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.008+0000 7f1b79200640 1 -- start start 2024-04-28T13:32:47.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.010+0000 7f1b79200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1b7415f2e0 0x7f1b740b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.010+0000 7f1b79200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b7406dc70 con 0x7f1b7415ef00 2024-04-28T13:32:47.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.010+0000 7f1b79200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b7406df50 con 0x7f1b740d1050 2024-04-28T13:32:47.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.010+0000 7f1b79200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f1b7406e230 con 0x7f1b7415f2e0 2024-04-28T13:32:47.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.010+0000 7f1b73400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f1b7415ef00 0x7f1b7406f7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:40490/0 (socket says 172.21.15.196:40490) 2024-04-28T13:32:47.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.010+0000 7f1b73400640 1 -- 172.21.15.196:0/1767803613 learned_addr learned my addr 172.21.15.196:0/1767803613 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:47.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b6be00640 1 --2- 172.21.15.196:0/1767803613 >> v2:172.21.15.203:3300/0 conn(0x7f1b7415f2e0 0x7f1b740b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b72a00640 1 -- 172.21.15.196:0/1767803613 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2922540601 0 0) 0x7f1b7406dc70 con 0x7f1b7415ef00 2024-04-28T13:32:47.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b72a00640 1 -- 172.21.15.196:0/1767803613 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b58003520 con 0x7f1b7415ef00 2024-04-28T13:32:47.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b72a00640 1 -- 172.21.15.196:0/1767803613 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2415510287 0 0) 0x7f1b58003520 con 0x7f1b7415ef00 2024-04-28T13:32:47.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b72a00640 1 -- 172.21.15.196:0/1767803613 >> v2:172.21.15.203:3300/0 conn(0x7f1b7415f2e0 msgr2=0x7f1b740b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:47.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b72a00640 1 --2- 172.21.15.196:0/1767803613 >> v2:172.21.15.203:3300/0 conn(0x7f1b7415f2e0 0x7f1b740b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b72a00640 1 -- 172.21.15.196:0/1767803613 >> v1:172.21.15.203:6789/0 conn(0x7f1b740d1050 legacy=0x7f1b74072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:47.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b72a00640 1 -- 172.21.15.196:0/1767803613 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7406e510 con 0x7f1b7415ef00 2024-04-28T13:32:47.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b72a00640 1 -- 172.21.15.196:0/1767803613 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f1b64001670 con 0x7f1b7415ef00 2024-04-28T13:32:47.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b72a00640 1 -- 172.21.15.196:0/1767803613 >> v1:172.21.15.196:6789/0 conn(0x7f1b7415ef00 legacy=0x7f1b7406f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:47.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b72a00640 1 --2- 172.21.15.196:0/1767803613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b580039b0 0x7f1b58003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b72a00640 1 --2- 172.21.15.196:0/1767803613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b58004800 0x7f1b58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b72a00640 1 --2- 172.21.15.196:0/1767803613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b580052e0 0x7f1b5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b72a00640 1 -- 172.21.15.196:0/1767803613 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7406e230 con 0x7f1b580039b0 2024-04-28T13:32:47.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b72a00640 1 -- 172.21.15.196:0/1767803613 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5800c190 con 0x7f1b58004800 2024-04-28T13:32:47.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.011+0000 7f1b72a00640 1 -- 172.21.15.196:0/1767803613 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5800c390 con 0x7f1b580052e0 2024-04-28T13:32:47.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.012+0000 7f1b73400640 1 --2- 172.21.15.196:0/1767803613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b580039b0 0x7f1b58003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.012+0000 7f1b6be00640 1 --2- 172.21.15.196:0/1767803613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b58004800 0x7f1b58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.012+0000 7f1b73400640 1 -- 172.21.15.196:0/1767803613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b580052e0 msgr2=0x7f1b5800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:47.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.012+0000 7f1b73400640 1 --2- 172.21.15.196:0/1767803613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b580052e0 0x7f1b5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.012+0000 7f1b73400640 1 -- 172.21.15.196:0/1767803613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b58004800 msgr2=0x7f1b58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:47.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.012+0000 7f1b73400640 1 --2- 172.21.15.196:0/1767803613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b58004800 0x7f1b58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.012+0000 7f1b73400640 1 -- 172.21.15.196:0/1767803613 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b58003520 con 0x7f1b580039b0 2024-04-28T13:32:47.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.012+0000 7f1b73e00640 1 -- 172.21.15.196:0/1767803613 reap_dead start 2024-04-28T13:32:47.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.013+0000 7f1b73400640 1 --2- 172.21.15.196:0/1767803613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b580039b0 0x7f1b58003db0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400c210 tx=0x7f1b58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7aa769d11fc0e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:47.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.013+0000 7f1b72a00640 1 -- 172.21.15.196:0/1767803613 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64007d30 con 0x7f1b580039b0 2024-04-28T13:32:47.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.013+0000 7f1b72a00640 1 -- 172.21.15.196:0/1767803613 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b640058d0 con 0x7f1b580039b0 2024-04-28T13:32:47.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.013+0000 7f1b72a00640 1 -- 172.21.15.196:0/1767803613 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64005d20 con 0x7f1b580039b0 2024-04-28T13:32:47.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.014+0000 7f1b79200640 1 -- 172.21.15.196:0/1767803613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b580039b0 msgr2=0x7f1b58003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:47.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.014+0000 7f1b79200640 1 --2- 172.21.15.196:0/1767803613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b580039b0 0x7f1b58003db0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400c210 tx=0x7f1b58005830 comp rx=0 tx=0).stop 2024-04-28T13:32:47.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.014+0000 7f1b79200640 1 -- 172.21.15.196:0/1767803613 shutdown_connections 2024-04-28T13:32:47.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.014+0000 7f1b79200640 1 --2- 172.21.15.196:0/1767803613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b580039b0 0x7f1b58003db0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.014+0000 7f1b79200640 1 -- 172.21.15.196:0/1767803613 >> 172.21.15.196:0/1767803613 conn(0x7f1b7406c6c0 msgr2=0x7f1b7406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:47.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.014+0000 7f1b79200640 1 -- 172.21.15.196:0/1767803613 shutdown_connections 2024-04-28T13:32:47.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.014+0000 7f1b79200640 1 -- 172.21.15.196:0/1767803613 wait complete. 2024-04-28T13:32:47.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.016+0000 7f1b79200640 1 Processor -- start 2024-04-28T13:32:47.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.016+0000 7f1b79200640 1 -- start start 2024-04-28T13:32:47.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.017+0000 7f1b79200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b580039b0 0x7f1b74158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.017+0000 7f1b79200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b7415f6f0 0x7f1b74162030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.017+0000 7f1b79200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b741518e0 0x7f1b740b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.017+0000 7f1b79200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b740b5fa0 con 0x7f1b741518e0 2024-04-28T13:32:47.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.017+0000 7f1b79200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1b740b5e20 con 0x7f1b580039b0 2024-04-28T13:32:47.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.017+0000 7f1b79200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1b740d14c0 con 0x7f1b7415f6f0 2024-04-28T13:32:47.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.017+0000 7f1b73e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b741518e0 0x7f1b740b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.017+0000 7f1b73e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b741518e0 0x7f1b740b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43300/0 (socket says 172.21.15.196:43300) 2024-04-28T13:32:47.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.017+0000 7f1b73e00640 1 -- 172.21.15.196:0/3126501809 learned_addr learned my addr 172.21.15.196:0/3126501809 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:47.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.018+0000 7f1b73400640 1 --2- 172.21.15.196:0/3126501809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b580039b0 0x7f1b74158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.018+0000 7f1b73e00640 1 -- 172.21.15.196:0/3126501809 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b7415f6f0 msgr2=0x7f1b74162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:47.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.018+0000 7f1b6be00640 1 --2- 172.21.15.196:0/3126501809 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b7415f6f0 0x7f1b74162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.018+0000 7f1b73e00640 1 --2- 172.21.15.196:0/3126501809 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b7415f6f0 0x7f1b74162030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.018+0000 7f1b73e00640 1 -- 172.21.15.196:0/3126501809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b580039b0 msgr2=0x7f1b74158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:47.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.018+0000 7f1b73e00640 1 --2- 172.21.15.196:0/3126501809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b580039b0 0x7f1b74158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.018+0000 7f1b73e00640 1 -- 172.21.15.196:0/3126501809 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7415fb80 con 0x7f1b741518e0 2024-04-28T13:32:47.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.018+0000 7f1b73e00640 1 --2- 172.21.15.196:0/3126501809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b741518e0 0x7f1b740b1260 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1b740b5190 tx=0x7f1b6c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:47.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.018+0000 7f1b6b400640 1 -- 172.21.15.196:0/3126501809 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c017070 con 0x7f1b741518e0 2024-04-28T13:32:47.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.018+0000 7f1b6b400640 1 -- 172.21.15.196:0/3126501809 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c0070d0 con 0x7f1b741518e0 2024-04-28T13:32:47.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.018+0000 7f1b79200640 1 -- 172.21.15.196:0/3126501809 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b74161550 con 0x7f1b741518e0 2024-04-28T13:32:47.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.018+0000 7f1b6b400640 1 -- 172.21.15.196:0/3126501809 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c007500 con 0x7f1b741518e0 2024-04-28T13:32:47.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.019+0000 7f1b79200640 1 -- 172.21.15.196:0/3126501809 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b74161b10 con 0x7f1b741518e0 2024-04-28T13:32:47.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.020+0000 7f1b79200640 1 -- 172.21.15.196:0/3126501809 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b7406a290 con 0x7f1b741518e0 2024-04-28T13:32:47.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.021+0000 7f1b6b400640 1 -- 172.21.15.196:0/3126501809 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f1b6c01a070 con 0x7f1b741518e0 2024-04-28T13:32:47.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.021+0000 7f1b6b400640 1 --2- 172.21.15.196:0/3126501809 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1b5006c980 0x7f1b5006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.022+0000 7f1b6b400640 1 -- 172.21.15.196:0/3126501809 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1b6c08af50 con 0x7f1b741518e0 2024-04-28T13:32:47.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.022+0000 7f1b73400640 1 --2- 172.21.15.196:0/3126501809 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1b5006c980 0x7f1b5006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.023+0000 7f1b73400640 1 --2- 172.21.15.196:0/3126501809 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1b5006c980 0x7f1b5006ee40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1b64033000 tx=0x7f1b6402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:47.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.026+0000 7f1b6b400640 1 -- 172.21.15.196:0/3126501809 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b6c059290 con 0x7f1b741518e0 2024-04-28T13:32:47.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.231+0000 7f1b79200640 1 -- 172.21.15.196:0/3126501809 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b741629e0 con 0x7f1b741518e0 2024-04-28T13:32:47.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.233+0000 7f1b6b400640 1 -- 172.21.15.196:0/3126501809 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16705 (secure 0 0 0) 0x7f1b6c058b90 con 0x7f1b741518e0 2024-04-28T13:32:47.233 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:47.233 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":33,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:33.217952+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:32:29.352348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-28T13:32:30.637498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:32:47.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.235+0000 7f1b79200640 1 -- 172.21.15.196:0/3126501809 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1b5006c980 msgr2=0x7f1b5006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:47.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.235+0000 7f1b79200640 1 --2- 172.21.15.196:0/3126501809 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1b5006c980 0x7f1b5006ee40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1b64033000 tx=0x7f1b6402e000 comp rx=0 tx=0).stop 2024-04-28T13:32:47.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.235+0000 7f1b79200640 1 -- 172.21.15.196:0/3126501809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b741518e0 msgr2=0x7f1b740b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:47.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.235+0000 7f1b79200640 1 --2- 172.21.15.196:0/3126501809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b741518e0 0x7f1b740b1260 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1b740b5190 tx=0x7f1b6c00c980 comp rx=0 tx=0).stop 2024-04-28T13:32:47.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.236+0000 7f1b79200640 1 -- 172.21.15.196:0/3126501809 shutdown_connections 2024-04-28T13:32:47.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.236+0000 7f1b79200640 1 --2- 172.21.15.196:0/3126501809 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1b5006c980 0x7f1b5006ee40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.236+0000 7f1b79200640 1 --2- 172.21.15.196:0/3126501809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b741518e0 0x7f1b740b1260 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.236+0000 7f1b79200640 1 --2- 172.21.15.196:0/3126501809 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b7415f6f0 0x7f1b74162030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.236+0000 7f1b79200640 1 --2- 172.21.15.196:0/3126501809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b580039b0 0x7f1b74158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.236+0000 7f1b79200640 1 -- 172.21.15.196:0/3126501809 >> 172.21.15.196:0/3126501809 conn(0x7f1b7406c6c0 msgr2=0x7f1b7415bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:47.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.236+0000 7f1b79200640 1 -- 172.21.15.196:0/3126501809 shutdown_connections 2024-04-28T13:32:47.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.236+0000 7f1b79200640 1 -- 172.21.15.196:0/3126501809 wait complete. 2024-04-28T13:32:47.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:32:29.352348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:32:30.637498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:32:47.255 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:32:47.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.454+0000 7f49c2a00640 1 Processor -- start 2024-04-28T13:32:47.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.455+0000 7f49c2a00640 1 -- start start 2024-04-28T13:32:47.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.456+0000 7f49c2a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f49bc163620 0x7f49bc06e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.456+0000 7f49c2a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc0d10a0 0x7f49bc0d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.456+0000 7f49c2a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49bc164a10 con 0x7f49bc06fa90 2024-04-28T13:32:47.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.456+0000 7f49c2a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f49bc0741c0 con 0x7f49bc163620 2024-04-28T13:32:47.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.456+0000 7f49c2a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f49bc0d1a10 con 0x7f49bc0d10a0 2024-04-28T13:32:47.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.457+0000 7f49c1600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f49bc163620 0x7f49bc06e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.457+0000 7f49c1600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f49bc163620 0x7f49bc06e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43314/0 (socket says 172.21.15.196:43314) 2024-04-28T13:32:47.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.457+0000 7f49c1600640 1 -- 172.21.15.196:0/3422452591 learned_addr learned my addr 172.21.15.196:0/3422452591 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:47.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.457+0000 7f49c2000640 1 --2- 172.21.15.196:0/3422452591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc0d10a0 0x7f49bc0d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.457+0000 7f49c1600640 1 -- 172.21.15.196:0/3422452591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc0d10a0 msgr2=0x7f49bc0d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:47.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.457+0000 7f49c1600640 1 --2- 172.21.15.196:0/3422452591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc0d10a0 0x7f49bc0d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.457+0000 7f49c1600640 1 -- 172.21.15.196:0/3422452591 >> v1:172.21.15.196:6789/0 conn(0x7f49bc06fa90 legacy=0x7f49bc073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:47.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.457+0000 7f49bbe00640 1 -- 172.21.15.196:0/3422452591 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 190105770 0 0) 0x7f49bc164a10 con 0x7f49bc06fa90 2024-04-28T13:32:47.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.457+0000 7f49c1600640 1 -- 172.21.15.196:0/3422452591 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49bc073ee0 con 0x7f49bc163620 2024-04-28T13:32:47.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.457+0000 7f49c2000640 1 --2- 172.21.15.196:0/3422452591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc0d10a0 0x7f49bc0d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:47.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.458+0000 7f49c1600640 1 --2- 172.21.15.196:0/3422452591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49bc163620 0x7f49bc06e980 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009870 tx=0x7f49b002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba608b59c73a2262 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:47.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.459+0000 7f49bbe00640 1 -- 172.21.15.196:0/3422452591 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0037070 con 0x7f49bc163620 2024-04-28T13:32:47.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.459+0000 7f49bbe00640 1 -- 172.21.15.196:0/3422452591 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49b0032440 con 0x7f49bc163620 2024-04-28T13:32:47.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.459+0000 7f49bbe00640 1 -- 172.21.15.196:0/3422452591 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0032810 con 0x7f49bc163620 2024-04-28T13:32:47.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.459+0000 7f49c2a00640 1 -- 172.21.15.196:0/3422452591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49bc163620 msgr2=0x7f49bc06e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:47.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.459+0000 7f49c2a00640 1 --2- 172.21.15.196:0/3422452591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49bc163620 0x7f49bc06e980 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009870 tx=0x7f49b002a1c0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.459+0000 7f49c2a00640 1 -- 172.21.15.196:0/3422452591 shutdown_connections 2024-04-28T13:32:47.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.459+0000 7f49c2a00640 1 --2- 172.21.15.196:0/3422452591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc0d10a0 0x7f49bc0d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.459+0000 7f49c2a00640 1 --2- 172.21.15.196:0/3422452591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49bc163620 0x7f49bc06e980 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.459+0000 7f49c2a00640 1 -- 172.21.15.196:0/3422452591 >> 172.21.15.196:0/3422452591 conn(0x7f49bc06c6c0 msgr2=0x7f49bc06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:47.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.459+0000 7f49c2a00640 1 -- 172.21.15.196:0/3422452591 shutdown_connections 2024-04-28T13:32:47.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.459+0000 7f49c2a00640 1 -- 172.21.15.196:0/3422452591 wait complete. 2024-04-28T13:32:47.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.461+0000 7f49c2a00640 1 Processor -- start 2024-04-28T13:32:47.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.461+0000 7f49c2a00640 1 -- start start 2024-04-28T13:32:47.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.462+0000 7f49c2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49bc06fa90 0x7f49bc15a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.462+0000 7f49c2a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc0d10a0 0x7f49bc15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.462+0000 7f49c2a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49bc163620 0x7f49bc0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.462+0000 7f49c2a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc06e460 con 0x7f49bc163620 2024-04-28T13:32:47.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.462+0000 7f49c2a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc06e180 con 0x7f49bc06fa90 2024-04-28T13:32:47.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.462+0000 7f49c2a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f49bc0720e0 con 0x7f49bc0d10a0 2024-04-28T13:32:47.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.463+0000 7f49c2000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49bc163620 0x7f49bc0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.463+0000 7f49c2000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49bc163620 0x7f49bc0ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43316/0 (socket says 172.21.15.196:43316) 2024-04-28T13:32:47.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.463+0000 7f49c2000640 1 -- 172.21.15.196:0/3645052823 learned_addr learned my addr 172.21.15.196:0/3645052823 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:47.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.463+0000 7f49c0c00640 1 --2- 172.21.15.196:0/3645052823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc0d10a0 0x7f49bc15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.463+0000 7f49c2000640 1 -- 172.21.15.196:0/3645052823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc0d10a0 msgr2=0x7f49bc15b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:47.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.463+0000 7f49c2000640 1 --2- 172.21.15.196:0/3645052823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc0d10a0 0x7f49bc15b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.463+0000 7f49c2000640 1 -- 172.21.15.196:0/3645052823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49bc06fa90 msgr2=0x7f49bc15a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:47.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.463+0000 7f49c2000640 1 --2- 172.21.15.196:0/3645052823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49bc06fa90 0x7f49bc15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.463+0000 7f49c2000640 1 -- 172.21.15.196:0/3645052823 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49bc15a680 con 0x7f49bc163620 2024-04-28T13:32:47.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.463+0000 7f49c0c00640 1 --2- 172.21.15.196:0/3645052823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc0d10a0 0x7f49bc15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:47.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.463+0000 7f49c2000640 1 --2- 172.21.15.196:0/3645052823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49bc163620 0x7f49bc0ce700 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f49b400cef0 tx=0x7f49b400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:47.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.464+0000 7f49b9600640 1 -- 172.21.15.196:0/3645052823 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b401a070 con 0x7f49bc163620 2024-04-28T13:32:47.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.464+0000 7f49b9600640 1 -- 172.21.15.196:0/3645052823 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49b400d860 con 0x7f49bc163620 2024-04-28T13:32:47.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.464+0000 7f49c2a00640 1 -- 172.21.15.196:0/3645052823 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49bc0cf130 con 0x7f49bc163620 2024-04-28T13:32:47.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.464+0000 7f49b9600640 1 -- 172.21.15.196:0/3645052823 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b400dc50 con 0x7f49bc163620 2024-04-28T13:32:47.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.464+0000 7f49c2a00640 1 -- 172.21.15.196:0/3645052823 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49bc0cf6f0 con 0x7f49bc163620 2024-04-28T13:32:47.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.465+0000 7f49c2a00640 1 -- 172.21.15.196:0/3645052823 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49bc050730 con 0x7f49bc163620 2024-04-28T13:32:47.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.466+0000 7f49b9600640 1 -- 172.21.15.196:0/3645052823 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f49b401d070 con 0x7f49bc163620 2024-04-28T13:32:47.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.467+0000 7f49b9600640 1 --2- 172.21.15.196:0/3645052823 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f499c06c980 0x7f499c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.467+0000 7f49b9600640 1 -- 172.21.15.196:0/3645052823 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f49b408c570 con 0x7f49bc163620 2024-04-28T13:32:47.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.467+0000 7f49c1600640 1 --2- 172.21.15.196:0/3645052823 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f499c06c980 0x7f499c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.468+0000 7f49c1600640 1 --2- 172.21.15.196:0/3645052823 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f499c06c980 0x7f499c06ee40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f49b0034000 tx=0x7f49b002c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:47.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.472+0000 7f49b9600640 1 -- 172.21.15.196:0/3645052823 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f49bc15a680 con 0x7f49bc163620 2024-04-28T13:32:47.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.676+0000 7f49c2a00640 1 -- 172.21.15.196:0/3645052823 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f49bc06ccf0 con 0x7f49bc163620 2024-04-28T13:32:47.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.676+0000 7f49b9600640 1 -- 172.21.15.196:0/3645052823 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f49b405a970 con 0x7f49bc163620 2024-04-28T13:32:47.676 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:32:47.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.679+0000 7f49c2a00640 1 -- 172.21.15.196:0/3645052823 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f499c06c980 msgr2=0x7f499c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:47.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.679+0000 7f49c2a00640 1 --2- 172.21.15.196:0/3645052823 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f499c06c980 0x7f499c06ee40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f49b0034000 tx=0x7f49b002c000 comp rx=0 tx=0).stop 2024-04-28T13:32:47.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.679+0000 7f49c2a00640 1 -- 172.21.15.196:0/3645052823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49bc163620 msgr2=0x7f49bc0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:47.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.679+0000 7f49c2a00640 1 --2- 172.21.15.196:0/3645052823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49bc163620 0x7f49bc0ce700 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f49b400cef0 tx=0x7f49b400a710 comp rx=0 tx=0).stop 2024-04-28T13:32:47.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.679+0000 7f49c2a00640 1 -- 172.21.15.196:0/3645052823 shutdown_connections 2024-04-28T13:32:47.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.679+0000 7f49c2a00640 1 --2- 172.21.15.196:0/3645052823 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f499c06c980 0x7f499c06ee40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.679+0000 7f49c2a00640 1 --2- 172.21.15.196:0/3645052823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49bc163620 0x7f49bc0ce700 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.679+0000 7f49c2a00640 1 --2- 172.21.15.196:0/3645052823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc0d10a0 0x7f49bc15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.679+0000 7f49c2a00640 1 --2- 172.21.15.196:0/3645052823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49bc06fa90 0x7f49bc15a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.679+0000 7f49c2a00640 1 -- 172.21.15.196:0/3645052823 >> 172.21.15.196:0/3645052823 conn(0x7f49bc06c6c0 msgr2=0x7f49bc162160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:47.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.679+0000 7f49c2a00640 1 -- 172.21.15.196:0/3645052823 shutdown_connections 2024-04-28T13:32:47.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.679+0000 7f49c2a00640 1 -- 172.21.15.196:0/3645052823 wait complete. 2024-04-28T13:32:47.698 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:32:47.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.895+0000 7fb6b7200640 1 Processor -- start 2024-04-28T13:32:47.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.895+0000 7fb6b7200640 1 -- start start 2024-04-28T13:32:47.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.897+0000 7fb6b7200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb6b80d10a0 0x7fb6b80b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.897+0000 7fb6b7200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb6b80d1480 0x7fb6b8160c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.897+0000 7fb6b7200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6b815d700 con 0x7fb6b806e8f0 2024-04-28T13:32:47.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.897+0000 7fb6b7200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb6b815e310 con 0x7fb6b80d1480 2024-04-28T13:32:47.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.897+0000 7fb6b7200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb6b815d9e0 con 0x7fb6b80d10a0 2024-04-28T13:32:47.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.897+0000 7fb6b6800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb6b80d1480 0x7fb6b8160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.897+0000 7fb6b6800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb6b80d1480 0x7fb6b8160c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43330/0 (socket says 172.21.15.196:43330) 2024-04-28T13:32:47.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.897+0000 7fb6b6800640 1 -- 172.21.15.196:0/560298680 learned_addr learned my addr 172.21.15.196:0/560298680 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:47.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.897+0000 7fb6b5e00640 1 --2- 172.21.15.196:0/560298680 >> v2:172.21.15.203:3300/0 conn(0x7fb6b80d10a0 0x7fb6b80b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.898+0000 7fb6b6800640 1 -- 172.21.15.196:0/560298680 >> v2:172.21.15.203:3300/0 conn(0x7fb6b80d10a0 msgr2=0x7fb6b80b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:47.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.898+0000 7fb6b6800640 1 --2- 172.21.15.196:0/560298680 >> v2:172.21.15.203:3300/0 conn(0x7fb6b80d10a0 0x7fb6b80b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.898+0000 7fb6b6800640 1 -- 172.21.15.196:0/560298680 >> v1:172.21.15.203:6789/0 conn(0x7fb6b806e8f0 legacy=0x7fb6b80b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:47.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.898+0000 7fb6b6800640 1 -- 172.21.15.196:0/560298680 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b815dcc0 con 0x7fb6b80d1480 2024-04-28T13:32:47.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.899+0000 7fb6b6800640 1 --2- 172.21.15.196:0/560298680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6b80d1480 0x7fb6b8160c90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80d1c30 tx=0x7fb6b002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca767dc9a8a1d8e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:47.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.899+0000 7fb6b4a00640 1 -- 172.21.15.196:0/560298680 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0036070 con 0x7fb6b80d1480 2024-04-28T13:32:47.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.899+0000 7fb6b4a00640 1 -- 172.21.15.196:0/560298680 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b0031600 con 0x7fb6b80d1480 2024-04-28T13:32:47.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.899+0000 7fb6b4a00640 1 -- 172.21.15.196:0/560298680 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b00319d0 con 0x7fb6b80d1480 2024-04-28T13:32:47.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.899+0000 7fb6b7200640 1 -- 172.21.15.196:0/560298680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6b80d1480 msgr2=0x7fb6b8160c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:47.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.899+0000 7fb6b7200640 1 --2- 172.21.15.196:0/560298680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6b80d1480 0x7fb6b8160c90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80d1c30 tx=0x7fb6b002a660 comp rx=0 tx=0).stop 2024-04-28T13:32:47.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.900+0000 7fb6b7200640 1 -- 172.21.15.196:0/560298680 shutdown_connections 2024-04-28T13:32:47.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.900+0000 7fb6b7200640 1 --2- 172.21.15.196:0/560298680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6b80d1480 0x7fb6b8160c90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.900+0000 7fb6b7200640 1 --2- 172.21.15.196:0/560298680 >> v2:172.21.15.203:3300/0 conn(0x7fb6b80d10a0 0x7fb6b80b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.900+0000 7fb6b7200640 1 -- 172.21.15.196:0/560298680 >> 172.21.15.196:0/560298680 conn(0x7fb6b806c6c0 msgr2=0x7fb6b806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:47.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.900+0000 7fb6b7200640 1 -- 172.21.15.196:0/560298680 shutdown_connections 2024-04-28T13:32:47.900 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.900+0000 7fb6b7200640 1 -- 172.21.15.196:0/560298680 wait complete. 2024-04-28T13:32:47.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.901+0000 7fb6b7200640 1 Processor -- start 2024-04-28T13:32:47.901 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.902+0000 7fb6b7200640 1 -- start start 2024-04-28T13:32:47.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.902+0000 7fb6b7200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6b806e8f0 0x7fb6b81f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.902+0000 7fb6b7200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6b80d10a0 0x7fb6b81f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.902+0000 7fb6b7200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6b80d1480 0x7fb6b81f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.902+0000 7fb6b7200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b80d1a30 con 0x7fb6b80d1480 2024-04-28T13:32:47.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.902+0000 7fb6b7200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b80d18b0 con 0x7fb6b806e8f0 2024-04-28T13:32:47.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.902+0000 7fb6b7200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b81611d0 con 0x7fb6b80d10a0 2024-04-28T13:32:47.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.903+0000 7fb6b6800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6b80d1480 0x7fb6b81f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.903+0000 7fb6b6800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6b80d1480 0x7fb6b81f5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43332/0 (socket says 172.21.15.196:43332) 2024-04-28T13:32:47.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.903+0000 7fb6b6800640 1 -- 172.21.15.196:0/3970321918 learned_addr learned my addr 172.21.15.196:0/3970321918 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:47.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.903+0000 7fb6b5e00640 1 --2- 172.21.15.196:0/3970321918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6b806e8f0 0x7fb6b81f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.903+0000 7fb6b6800640 1 -- 172.21.15.196:0/3970321918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6b80d10a0 msgr2=0x7fb6b81f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:47.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.903+0000 7fb6b6800640 1 --2- 172.21.15.196:0/3970321918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6b80d10a0 0x7fb6b81f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.903+0000 7fb6b6800640 1 -- 172.21.15.196:0/3970321918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6b806e8f0 msgr2=0x7fb6b81f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:47.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.903+0000 7fb6b6800640 1 --2- 172.21.15.196:0/3970321918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6b806e8f0 0x7fb6b81f51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:47.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.903+0000 7fb6b6800640 1 -- 172.21.15.196:0/3970321918 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b81512a0 con 0x7fb6b80d1480 2024-04-28T13:32:47.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.904+0000 7fb6b6800640 1 --2- 172.21.15.196:0/3970321918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6b80d1480 0x7fb6b81f5c70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb6b002ac50 tx=0x7fb6b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:47.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.904+0000 7fb6ae000640 1 -- 172.21.15.196:0/3970321918 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0036050 con 0x7fb6b80d1480 2024-04-28T13:32:47.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.904+0000 7fb6ae000640 1 -- 172.21.15.196:0/3970321918 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b0042070 con 0x7fb6b80d1480 2024-04-28T13:32:47.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.904+0000 7fb6ae000640 1 -- 172.21.15.196:0/3970321918 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0009510 con 0x7fb6b80d1480 2024-04-28T13:32:47.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.904+0000 7fb6b7200640 1 -- 172.21.15.196:0/3970321918 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b8151580 con 0x7fb6b80d1480 2024-04-28T13:32:47.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.904+0000 7fb6b7200640 1 -- 172.21.15.196:0/3970321918 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b8151b40 con 0x7fb6b80d1480 2024-04-28T13:32:47.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.905+0000 7fb6b7200640 1 -- 172.21.15.196:0/3970321918 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b8050730 con 0x7fb6b80d1480 2024-04-28T13:32:47.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.906+0000 7fb6ae000640 1 -- 172.21.15.196:0/3970321918 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb6b0007bb0 con 0x7fb6b80d1480 2024-04-28T13:32:47.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.907+0000 7fb6ae000640 1 --2- 172.21.15.196:0/3970321918 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb69806c980 0x7fb69806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:47.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.907+0000 7fb6ae000640 1 -- 172.21.15.196:0/3970321918 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb6b00ace30 con 0x7fb6b80d1480 2024-04-28T13:32:47.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.908+0000 7fb6b5e00640 1 --2- 172.21.15.196:0/3970321918 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb69806c980 0x7fb69806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:47.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.909+0000 7fb6b5e00640 1 --2- 172.21.15.196:0/3970321918 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb69806c980 0x7fb69806ee40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8006fd0 tx=0x7fb6a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:47.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:47.912+0000 7fb6ae000640 1 -- 172.21.15.196:0/3970321918 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb6b007afd0 con 0x7fb6b80d1480 2024-04-28T13:32:48.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.113+0000 7fb6b7200640 1 -- 172.21.15.196:0/3970321918 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb6b806f700 con 0x7fb6b80d1480 2024-04-28T13:32:48.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.114+0000 7fb6ae000640 1 -- 172.21.15.196:0/3970321918 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7fb6b007a8d0 con 0x7fb6b80d1480 2024-04-28T13:32:48.114 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:32:48.116 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.116+0000 7fb6b7200640 1 -- 172.21.15.196:0/3970321918 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb69806c980 msgr2=0x7fb69806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:48.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.116+0000 7fb6b7200640 1 --2- 172.21.15.196:0/3970321918 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb69806c980 0x7fb69806ee40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8006fd0 tx=0x7fb6a8008040 comp rx=0 tx=0).stop 2024-04-28T13:32:48.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.116+0000 7fb6b7200640 1 -- 172.21.15.196:0/3970321918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6b80d1480 msgr2=0x7fb6b81f5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:48.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.116+0000 7fb6b7200640 1 --2- 172.21.15.196:0/3970321918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6b80d1480 0x7fb6b81f5c70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb6b002ac50 tx=0x7fb6b0004a40 comp rx=0 tx=0).stop 2024-04-28T13:32:48.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.116+0000 7fb6b7200640 1 -- 172.21.15.196:0/3970321918 shutdown_connections 2024-04-28T13:32:48.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.116+0000 7fb6b7200640 1 --2- 172.21.15.196:0/3970321918 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb69806c980 0x7fb69806ee40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.116+0000 7fb6b7200640 1 --2- 172.21.15.196:0/3970321918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6b80d1480 0x7fb6b81f5c70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.116+0000 7fb6b7200640 1 --2- 172.21.15.196:0/3970321918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6b80d10a0 0x7fb6b81f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.116+0000 7fb6b7200640 1 --2- 172.21.15.196:0/3970321918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6b806e8f0 0x7fb6b81f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.116+0000 7fb6b7200640 1 -- 172.21.15.196:0/3970321918 >> 172.21.15.196:0/3970321918 conn(0x7fb6b806c6c0 msgr2=0x7fb6b815c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:48.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.117+0000 7fb6b7200640 1 -- 172.21.15.196:0/3970321918 shutdown_connections 2024-04-28T13:32:48.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.117+0000 7fb6b7200640 1 -- 172.21.15.196:0/3970321918 wait complete. 2024-04-28T13:32:48.135 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:32:48.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.335+0000 7f26f1400640 1 Processor -- start 2024-04-28T13:32:48.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.335+0000 7f26f1400640 1 -- start start 2024-04-28T13:32:48.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.337+0000 7f26f1400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f26ec0d0610 0x7f26ec073af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:48.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.337+0000 7f26f1400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26ec0b05b0 con 0x7f26ec15cff0 2024-04-28T13:32:48.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.337+0000 7f26f1400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26ec074180 con 0x7f26ec0add70 2024-04-28T13:32:48.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.337+0000 7f26f1400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f26ec071ef0 con 0x7f26ec0d0610 2024-04-28T13:32:48.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.337+0000 7f26ebe00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f26ec15cff0 0x7f26ec072cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:40514/0 (socket says 172.21.15.196:40514) 2024-04-28T13:32:48.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.337+0000 7f26ebe00640 1 -- 172.21.15.196:0/197108221 learned_addr learned my addr 172.21.15.196:0/197108221 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:48.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.337+0000 7f26f0a00640 1 --2- 172.21.15.196:0/197108221 >> v2:172.21.15.203:3300/0 conn(0x7f26ec0d0610 0x7f26ec073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:48.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.338+0000 7f26eaa00640 1 -- 172.21.15.196:0/197108221 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 597011664 0 0) 0x7f26ec0b05b0 con 0x7f26ec15cff0 2024-04-28T13:32:48.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.338+0000 7f26eaa00640 1 -- 172.21.15.196:0/197108221 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26d0003520 con 0x7f26ec15cff0 2024-04-28T13:32:48.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.338+0000 7f26eaa00640 1 -- 172.21.15.196:0/197108221 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2166350433 0 0) 0x7f26d0003520 con 0x7f26ec15cff0 2024-04-28T13:32:48.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.338+0000 7f26eaa00640 1 -- 172.21.15.196:0/197108221 >> v2:172.21.15.203:3300/0 conn(0x7f26ec0d0610 msgr2=0x7f26ec073af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:48.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.338+0000 7f26eaa00640 1 --2- 172.21.15.196:0/197108221 >> v2:172.21.15.203:3300/0 conn(0x7f26ec0d0610 0x7f26ec073af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.338+0000 7f26eaa00640 1 -- 172.21.15.196:0/197108221 >> v1:172.21.15.203:6789/0 conn(0x7f26ec0add70 legacy=0x7f26ec0cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:48.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.338+0000 7f26eaa00640 1 -- 172.21.15.196:0/197108221 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ec0721d0 con 0x7f26ec15cff0 2024-04-28T13:32:48.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.338+0000 7f26eaa00640 1 -- 172.21.15.196:0/197108221 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f26dc001670 con 0x7f26ec15cff0 2024-04-28T13:32:48.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.338+0000 7f26eaa00640 1 -- 172.21.15.196:0/197108221 >> v1:172.21.15.196:6789/0 conn(0x7f26ec15cff0 legacy=0x7f26ec072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:48.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.338+0000 7f26eaa00640 1 --2- 172.21.15.196:0/197108221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26d00039b0 0x7f26d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:48.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.338+0000 7f26eaa00640 1 --2- 172.21.15.196:0/197108221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26d0004800 0x7f26d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:48.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.338+0000 7f26eaa00640 1 --2- 172.21.15.196:0/197108221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26d00051c0 0x7f26d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:48.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.338+0000 7f26eaa00640 1 -- 172.21.15.196:0/197108221 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec071ef0 con 0x7f26d00039b0 2024-04-28T13:32:48.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.338+0000 7f26eaa00640 1 -- 172.21.15.196:0/197108221 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f26d000c160 con 0x7f26d0004800 2024-04-28T13:32:48.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.338+0000 7f26eaa00640 1 -- 172.21.15.196:0/197108221 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f26d000c360 con 0x7f26d00051c0 2024-04-28T13:32:48.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.339+0000 7f26ebe00640 1 --2- 172.21.15.196:0/197108221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26d00039b0 0x7f26d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:48.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.339+0000 7f26eb400640 1 --2- 172.21.15.196:0/197108221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26d0004800 0x7f26d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:48.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.339+0000 7f26f0a00640 1 --2- 172.21.15.196:0/197108221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26d00051c0 0x7f26d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:48.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.340+0000 7f26ebe00640 1 -- 172.21.15.196:0/197108221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26d00051c0 msgr2=0x7f26d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:48.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.340+0000 7f26ebe00640 1 --2- 172.21.15.196:0/197108221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26d00051c0 0x7f26d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.340+0000 7f26ebe00640 1 -- 172.21.15.196:0/197108221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26d0004800 msgr2=0x7f26d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:48.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.340+0000 7f26ebe00640 1 --2- 172.21.15.196:0/197108221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26d0004800 0x7f26d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.340+0000 7f26f0a00640 1 -- 172.21.15.196:0/197108221 reap_dead start 2024-04-28T13:32:48.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.340+0000 7f26ebe00640 1 -- 172.21.15.196:0/197108221 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d0003520 con 0x7f26d00039b0 2024-04-28T13:32:48.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.341+0000 7f26ebe00640 1 --2- 172.21.15.196:0/197108221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26d00039b0 0x7f26d0003dd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00c240 tx=0x7f26d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa382f27f46a0a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:48.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.342+0000 7f26eaa00640 1 -- 172.21.15.196:0/197108221 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc007d10 con 0x7f26d00039b0 2024-04-28T13:32:48.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.342+0000 7f26eaa00640 1 -- 172.21.15.196:0/197108221 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26dc005ad0 con 0x7f26d00039b0 2024-04-28T13:32:48.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.342+0000 7f26eaa00640 1 -- 172.21.15.196:0/197108221 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc007070 con 0x7f26d00039b0 2024-04-28T13:32:48.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.342+0000 7f26f1400640 1 -- 172.21.15.196:0/197108221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26d00039b0 msgr2=0x7f26d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:48.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.342+0000 7f26f1400640 1 --2- 172.21.15.196:0/197108221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26d00039b0 0x7f26d0003dd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00c240 tx=0x7f26d0005830 comp rx=0 tx=0).stop 2024-04-28T13:32:48.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.343+0000 7f26f1400640 1 -- 172.21.15.196:0/197108221 shutdown_connections 2024-04-28T13:32:48.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.343+0000 7f26f1400640 1 --2- 172.21.15.196:0/197108221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26d00039b0 0x7f26d0003dd0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.343+0000 7f26f1400640 1 -- 172.21.15.196:0/197108221 >> 172.21.15.196:0/197108221 conn(0x7f26ec0b2990 msgr2=0x7f26ec0b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:48.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.343+0000 7f26f1400640 1 -- 172.21.15.196:0/197108221 shutdown_connections 2024-04-28T13:32:48.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.343+0000 7f26f1400640 1 -- 172.21.15.196:0/197108221 wait complete. 2024-04-28T13:32:48.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.344+0000 7f26f1400640 1 Processor -- start 2024-04-28T13:32:48.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.345+0000 7f26f1400640 1 -- start start 2024-04-28T13:32:48.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.345+0000 7f26f1400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26d00039b0 0x7f26ec139890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:48.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.345+0000 7f26f1400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26ec0b43a0 0x7f26ec13a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:48.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.346+0000 7f26f1400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26ec0b4820 0x7f26ec13a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:48.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.346+0000 7f26f1400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec15d3d0 con 0x7f26ec0b43a0 2024-04-28T13:32:48.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.346+0000 7f26f1400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec072790 con 0x7f26ec0b4820 2024-04-28T13:32:48.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.346+0000 7f26f1400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f26ec073af0 con 0x7f26d00039b0 2024-04-28T13:32:48.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.346+0000 7f26eb400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26ec0b43a0 0x7f26ec13a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:48.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.346+0000 7f26eb400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26ec0b43a0 0x7f26ec13a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43346/0 (socket says 172.21.15.196:43346) 2024-04-28T13:32:48.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.346+0000 7f26eb400640 1 -- 172.21.15.196:0/1805239363 learned_addr learned my addr 172.21.15.196:0/1805239363 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:48.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.346+0000 7f26eb400640 1 -- 172.21.15.196:0/1805239363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26d00039b0 msgr2=0x7f26ec139890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:48.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.346+0000 7f26f0a00640 1 --2- 172.21.15.196:0/1805239363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26ec0b4820 0x7f26ec13a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:48.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.346+0000 7f26eb400640 1 --2- 172.21.15.196:0/1805239363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26d00039b0 0x7f26ec139890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.346+0000 7f26eb400640 1 -- 172.21.15.196:0/1805239363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26ec0b4820 msgr2=0x7f26ec13a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:48.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.346+0000 7f26eb400640 1 --2- 172.21.15.196:0/1805239363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26ec0b4820 0x7f26ec13a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.346+0000 7f26eb400640 1 -- 172.21.15.196:0/1805239363 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ec13abf0 con 0x7f26ec0b43a0 2024-04-28T13:32:48.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.347+0000 7f26eb400640 1 --2- 172.21.15.196:0/1805239363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26ec0b43a0 0x7f26ec13a170 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f26d800f5f0 tx=0x7f26d800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:48.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.347+0000 7f26e3e00640 1 -- 172.21.15.196:0/1805239363 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d80052e0 con 0x7f26ec0b43a0 2024-04-28T13:32:48.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.347+0000 7f26e3e00640 1 -- 172.21.15.196:0/1805239363 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d8018070 con 0x7f26ec0b43a0 2024-04-28T13:32:48.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.347+0000 7f26e3e00640 1 -- 172.21.15.196:0/1805239363 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8003810 con 0x7f26ec0b43a0 2024-04-28T13:32:48.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.347+0000 7f26f1400640 1 -- 172.21.15.196:0/1805239363 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ec13aeb0 con 0x7f26ec0b43a0 2024-04-28T13:32:48.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.347+0000 7f26f1400640 1 -- 172.21.15.196:0/1805239363 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ec13b470 con 0x7f26ec0b43a0 2024-04-28T13:32:48.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.348+0000 7f26f1400640 1 -- 172.21.15.196:0/1805239363 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26ec050730 con 0x7f26ec0b43a0 2024-04-28T13:32:48.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.349+0000 7f26e3e00640 1 -- 172.21.15.196:0/1805239363 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f26d80039b0 con 0x7f26ec0b43a0 2024-04-28T13:32:48.350 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.350+0000 7f26e3e00640 1 --2- 172.21.15.196:0/1805239363 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f26cc06c9b0 0x7f26cc06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:48.350 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.350+0000 7f26e3e00640 1 -- 172.21.15.196:0/1805239363 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f26d808b9e0 con 0x7f26ec0b43a0 2024-04-28T13:32:48.350 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.351+0000 7f26ebe00640 1 --2- 172.21.15.196:0/1805239363 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f26cc06c9b0 0x7f26cc06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:48.352 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.352+0000 7f26ebe00640 1 --2- 172.21.15.196:0/1805239363 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f26cc06c9b0 0x7f26cc06ee70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f26dc032000 tx=0x7f26dc009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:48.354 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.354+0000 7f26e3e00640 1 -- 172.21.15.196:0/1805239363 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f26d8059db0 con 0x7f26ec0b43a0 2024-04-28T13:32:48.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.553+0000 7f26f1400640 1 -- 172.21.15.196:0/1805239363 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f26ec06fd10 con 0x7f26ec0b43a0 2024-04-28T13:32:48.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.554+0000 7f26e3e00640 1 -- 172.21.15.196:0/1805239363 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f26d80596b0 con 0x7f26ec0b43a0 2024-04-28T13:32:48.554 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:32:48.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.556+0000 7f26f1400640 1 -- 172.21.15.196:0/1805239363 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f26cc06c9b0 msgr2=0x7f26cc06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:48.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.556+0000 7f26f1400640 1 --2- 172.21.15.196:0/1805239363 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f26cc06c9b0 0x7f26cc06ee70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f26dc032000 tx=0x7f26dc009030 comp rx=0 tx=0).stop 2024-04-28T13:32:48.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.556+0000 7f26f1400640 1 -- 172.21.15.196:0/1805239363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26ec0b43a0 msgr2=0x7f26ec13a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:48.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.556+0000 7f26f1400640 1 --2- 172.21.15.196:0/1805239363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26ec0b43a0 0x7f26ec13a170 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f26d800f5f0 tx=0x7f26d800fab0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.556+0000 7f26f1400640 1 -- 172.21.15.196:0/1805239363 shutdown_connections 2024-04-28T13:32:48.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.556+0000 7f26f1400640 1 --2- 172.21.15.196:0/1805239363 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f26cc06c9b0 0x7f26cc06ee70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.556+0000 7f26f1400640 1 --2- 172.21.15.196:0/1805239363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26ec0b4820 0x7f26ec13a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.556+0000 7f26f1400640 1 --2- 172.21.15.196:0/1805239363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26ec0b43a0 0x7f26ec13a170 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.556+0000 7f26f1400640 1 --2- 172.21.15.196:0/1805239363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26d00039b0 0x7f26ec139890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.556+0000 7f26f1400640 1 -- 172.21.15.196:0/1805239363 >> 172.21.15.196:0/1805239363 conn(0x7f26ec0b2990 msgr2=0x7f26ec0ae600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:48.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.557+0000 7f26f1400640 1 -- 172.21.15.196:0/1805239363 shutdown_connections 2024-04-28T13:32:48.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.557+0000 7f26f1400640 1 -- 172.21.15.196:0/1805239363 wait complete. 2024-04-28T13:32:48.576 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T13:32:48.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.778+0000 7f018d600640 1 Processor -- start 2024-04-28T13:32:48.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.779+0000 7f018d600640 1 -- start start 2024-04-28T13:32:48.781 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.780+0000 7f018d600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01880b5ee0 0x7f01880b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:48.781 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.780+0000 7f018d600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f018809ec80 0x7f01880d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:48.781 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.781+0000 7f018d600640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f018815e380 con 0x7f01880add70 2024-04-28T13:32:48.781 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.781+0000 7f018d600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f01880b4b20 con 0x7f018809ec80 2024-04-28T13:32:48.781 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.781+0000 7f018d600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f01880b3f30 con 0x7f01880b5ee0 2024-04-28T13:32:48.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.781+0000 7f0187e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01880b5ee0 0x7f01880b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:48.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.781+0000 7f017fe00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f018809ec80 0x7f01880d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:48.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.782+0000 7f0187e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01880b5ee0 0x7f01880b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:45636/0 (socket says 172.21.15.196:45636) 2024-04-28T13:32:48.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.782+0000 7f0187e00640 1 -- 172.21.15.196:0/2539899139 learned_addr learned my addr 172.21.15.196:0/2539899139 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:48.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.782+0000 7f017fe00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f018809ec80 0x7f01880d1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:54414/0 (socket says 172.21.15.196:54414) 2024-04-28T13:32:48.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.782+0000 7f018cc00640 1 --1- 172.21.15.196:0/2539899139 >> v1:172.21.15.203:6789/0 conn(0x7f01880add70 0x7f018806df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.196:57420/0 (socket says 172.21.15.196:57420) 2024-04-28T13:32:48.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.782+0000 7f0187e00640 1 -- 172.21.15.196:0/2539899139 >> v2:172.21.15.203:3300/0 conn(0x7f018809ec80 msgr2=0x7f01880d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:48.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.782+0000 7f0187e00640 1 --2- 172.21.15.196:0/2539899139 >> v2:172.21.15.203:3300/0 conn(0x7f018809ec80 0x7f01880d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.782+0000 7f0187e00640 1 -- 172.21.15.196:0/2539899139 >> v1:172.21.15.203:6789/0 conn(0x7f01880add70 legacy=0x7f018806df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:48.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.782+0000 7f0187e00640 1 -- 172.21.15.196:0/2539899139 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01880b41f0 con 0x7f01880b5ee0 2024-04-28T13:32:48.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.784+0000 7f0187e00640 1 --2- 172.21.15.196:0/2539899139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01880b5ee0 0x7f01880b4f40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0178009870 tx=0x7f017802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86ac28fb06be9f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:48.784 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.784+0000 7f0187400640 1 -- 172.21.15.196:0/2539899139 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178037070 con 0x7f01880b5ee0 2024-04-28T13:32:48.784 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.784+0000 7f0187400640 1 -- 172.21.15.196:0/2539899139 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0178032440 con 0x7f01880b5ee0 2024-04-28T13:32:48.784 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.784+0000 7f0187400640 1 -- 172.21.15.196:0/2539899139 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178032810 con 0x7f01880b5ee0 2024-04-28T13:32:48.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.785+0000 7f018d600640 1 -- 172.21.15.196:0/2539899139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01880b5ee0 msgr2=0x7f01880b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:48.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.785+0000 7f018d600640 1 --2- 172.21.15.196:0/2539899139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01880b5ee0 0x7f01880b4f40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0178009870 tx=0x7f017802a270 comp rx=0 tx=0).stop 2024-04-28T13:32:48.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.785+0000 7f018d600640 1 -- 172.21.15.196:0/2539899139 shutdown_connections 2024-04-28T13:32:48.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.785+0000 7f018d600640 1 --2- 172.21.15.196:0/2539899139 >> v2:172.21.15.203:3300/0 conn(0x7f018809ec80 0x7f01880d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.785+0000 7f018d600640 1 --2- 172.21.15.196:0/2539899139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01880b5ee0 0x7f01880b4f40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.785+0000 7f018d600640 1 -- 172.21.15.196:0/2539899139 >> 172.21.15.196:0/2539899139 conn(0x7f018806c6c0 msgr2=0x7f018806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:48.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.785+0000 7f018d600640 1 -- 172.21.15.196:0/2539899139 shutdown_connections 2024-04-28T13:32:48.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.785+0000 7f018d600640 1 -- 172.21.15.196:0/2539899139 wait complete. 2024-04-28T13:32:48.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.787+0000 7f018d600640 1 Processor -- start 2024-04-28T13:32:48.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.787+0000 7f018d600640 1 -- start start 2024-04-28T13:32:48.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.788+0000 7f018d600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f018809ec80 0x7f018815f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:48.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.788+0000 7f018d600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01880add70 0x7f01880b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:48.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.788+0000 7f018d600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01880b5ee0 0x7f01880b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:48.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.788+0000 7f018d600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f01880b3c20 con 0x7f018809ec80 2024-04-28T13:32:48.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.788+0000 7f018d600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f01880b47b0 con 0x7f01880add70 2024-04-28T13:32:48.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.788+0000 7f018d600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f01880d1590 con 0x7f01880b5ee0 2024-04-28T13:32:48.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.788+0000 7f0187e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f018809ec80 0x7f018815f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:48.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.788+0000 7f0187e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f018809ec80 0x7f018815f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43354/0 (socket says 172.21.15.196:43354) 2024-04-28T13:32:48.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.788+0000 7f0187e00640 1 -- 172.21.15.196:0/357595 learned_addr learned my addr 172.21.15.196:0/357595 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:48.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.789+0000 7f017fe00640 1 --2- 172.21.15.196:0/357595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01880add70 0x7f01880b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:48.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.789+0000 7f018cc00640 1 --2- 172.21.15.196:0/357595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01880b5ee0 0x7f01880b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:48.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.789+0000 7f0187e00640 1 -- 172.21.15.196:0/357595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01880b5ee0 msgr2=0x7f01880b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:48.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.789+0000 7f0187e00640 1 --2- 172.21.15.196:0/357595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01880b5ee0 0x7f01880b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.789+0000 7f0187e00640 1 -- 172.21.15.196:0/357595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01880add70 msgr2=0x7f01880b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:48.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.789+0000 7f0187e00640 1 --2- 172.21.15.196:0/357595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01880add70 0x7f01880b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:48.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.789+0000 7f0187e00640 1 -- 172.21.15.196:0/357595 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01880afa00 con 0x7f018809ec80 2024-04-28T13:32:48.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.789+0000 7f017fe00640 1 --2- 172.21.15.196:0/357595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01880add70 0x7f01880b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:48.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.789+0000 7f0187e00640 1 --2- 172.21.15.196:0/357595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f018809ec80 0x7f018815f3f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0178000c00 tx=0x7f01780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:48.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.789+0000 7f0184c00640 1 -- 172.21.15.196:0/357595 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178037040 con 0x7f018809ec80 2024-04-28T13:32:48.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.789+0000 7f0184c00640 1 -- 172.21.15.196:0/357595 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0178042070 con 0x7f018809ec80 2024-04-28T13:32:48.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.789+0000 7f018d600640 1 -- 172.21.15.196:0/357595 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01880afce0 con 0x7f018809ec80 2024-04-28T13:32:48.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.790+0000 7f0184c00640 1 -- 172.21.15.196:0/357595 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178030880 con 0x7f018809ec80 2024-04-28T13:32:48.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.790+0000 7f018d600640 1 -- 172.21.15.196:0/357595 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01880b0580 con 0x7f018809ec80 2024-04-28T13:32:48.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.791+0000 7f018d600640 1 -- 172.21.15.196:0/357595 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f018806a290 con 0x7f018809ec80 2024-04-28T13:32:48.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.792+0000 7f0184c00640 1 -- 172.21.15.196:0/357595 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f0178030a20 con 0x7f018809ec80 2024-04-28T13:32:48.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.793+0000 7f0184c00640 1 --2- 172.21.15.196:0/357595 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f016406c980 0x7f016406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:48.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.794+0000 7f0184c00640 1 -- 172.21.15.196:0/357595 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f01780ac4d0 con 0x7f018809ec80 2024-04-28T13:32:48.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.794+0000 7f017fe00640 1 --2- 172.21.15.196:0/357595 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f016406c980 0x7f016406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:48.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.795+0000 7f017fe00640 1 --2- 172.21.15.196:0/357595 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f016406c980 0x7f016406ee40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f01881f81b0 tx=0x7f0170008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:48.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:48.798+0000 7f0184c00640 1 -- 172.21.15.196:0/357595 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f017807a810 con 0x7f018809ec80 2024-04-28T13:32:49.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.012+0000 7f018d600640 1 -- 172.21.15.196:0/357595 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f018809e850 con 0x7f018809ec80 2024-04-28T13:32:49.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.013+0000 7f0184c00640 1 -- 172.21.15.196:0/357595 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f017807a110 con 0x7f018809ec80 2024-04-28T13:32:49.013 INFO:teuthology.orchestra.run.smithi196.stderr:listed 0 entries 2024-04-28T13:32:49.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.016+0000 7f018d600640 1 -- 172.21.15.196:0/357595 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f016406c980 msgr2=0x7f016406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:49.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.016+0000 7f018d600640 1 --2- 172.21.15.196:0/357595 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f016406c980 0x7f016406ee40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f01881f81b0 tx=0x7f0170008040 comp rx=0 tx=0).stop 2024-04-28T13:32:49.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.016+0000 7f018d600640 1 -- 172.21.15.196:0/357595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f018809ec80 msgr2=0x7f018815f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:49.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.016+0000 7f018d600640 1 --2- 172.21.15.196:0/357595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f018809ec80 0x7f018815f3f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0178000c00 tx=0x7f01780046c0 comp rx=0 tx=0).stop 2024-04-28T13:32:49.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.016+0000 7f018d600640 1 -- 172.21.15.196:0/357595 shutdown_connections 2024-04-28T13:32:49.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.016+0000 7f018d600640 1 --2- 172.21.15.196:0/357595 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f016406c980 0x7f016406ee40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:49.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.016+0000 7f018d600640 1 --2- 172.21.15.196:0/357595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01880b5ee0 0x7f01880b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:49.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.016+0000 7f018d600640 1 --2- 172.21.15.196:0/357595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01880add70 0x7f01880b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:49.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.016+0000 7f018d600640 1 --2- 172.21.15.196:0/357595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f018809ec80 0x7f018815f3f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:49.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.016+0000 7f018d600640 1 -- 172.21.15.196:0/357595 >> 172.21.15.196:0/357595 conn(0x7f018806c6c0 msgr2=0x7f01880cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:49.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.016+0000 7f018d600640 1 -- 172.21.15.196:0/357595 shutdown_connections 2024-04-28T13:32:49.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.016+0000 7f018d600640 1 -- 172.21.15.196:0/357595 wait complete. 2024-04-28T13:32:49.035 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-28T13:32:49.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:49.035 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-28T13:32:49.138 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:32:49.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:49.139 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-28T13:32:49.192 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:32:49.192 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:49.251 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-28T13:32:49.251 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:32:49.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:49.251 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-28T13:32:49.301 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-28T13:32:49.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:32:49.302 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:32:49.302 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:32:49.360 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-28T13:32:49.360 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:32:49.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:49.360 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-28T13:32:49.412 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-28T13:32:49.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:32:49.413 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T13:32:49.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.622+0000 7fdda3200640 1 Processor -- start 2024-04-28T13:32:49.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.622+0000 7fdda3200640 1 -- start start 2024-04-28T13:32:49.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.624+0000 7fdda3200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdda40d1790 0x7fdda4167df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:49.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.624+0000 7fdda3200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdda4160b00 con 0x7fdda40add70 2024-04-28T13:32:49.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.624+0000 7fdda3200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdda4160de0 con 0x7fdda415a9a0 2024-04-28T13:32:49.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.624+0000 7fdda3200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdda41610c0 con 0x7fdda40d1790 2024-04-28T13:32:49.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.624+0000 7fdda1400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fdda40add70 0x7fdda40d1080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:40524/0 (socket says 172.21.15.196:40524) 2024-04-28T13:32:49.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.624+0000 7fdda1400640 1 -- 172.21.15.196:0/2478256616 learned_addr learned my addr 172.21.15.196:0/2478256616 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:49.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.624+0000 7fdda2800640 1 --2- 172.21.15.196:0/2478256616 >> v2:172.21.15.203:3300/0 conn(0x7fdda40d1790 0x7fdda4167df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:49.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.625+0000 7fdda0a00640 1 -- 172.21.15.196:0/2478256616 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 789656173 0 0) 0x7fdda4160b00 con 0x7fdda40add70 2024-04-28T13:32:49.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.625+0000 7fdda0a00640 1 -- 172.21.15.196:0/2478256616 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd88003520 con 0x7fdda40add70 2024-04-28T13:32:49.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.625+0000 7fdda0a00640 1 -- 172.21.15.196:0/2478256616 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1326420675 0 0) 0x7fdd88003520 con 0x7fdda40add70 2024-04-28T13:32:49.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.625+0000 7fdda0a00640 1 -- 172.21.15.196:0/2478256616 >> v2:172.21.15.203:3300/0 conn(0x7fdda40d1790 msgr2=0x7fdda4167df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:49.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.625+0000 7fdda0a00640 1 --2- 172.21.15.196:0/2478256616 >> v2:172.21.15.203:3300/0 conn(0x7fdda40d1790 0x7fdda4167df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:49.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.625+0000 7fdda0a00640 1 -- 172.21.15.196:0/2478256616 >> v1:172.21.15.203:6789/0 conn(0x7fdda415a9a0 legacy=0x7fdda40cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:49.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.625+0000 7fdda0a00640 1 -- 172.21.15.196:0/2478256616 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda41613a0 con 0x7fdda40add70 2024-04-28T13:32:49.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.625+0000 7fdda0a00640 1 -- 172.21.15.196:0/2478256616 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fdd90001670 con 0x7fdda40add70 2024-04-28T13:32:49.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.625+0000 7fdda0a00640 1 -- 172.21.15.196:0/2478256616 >> v1:172.21.15.196:6789/0 conn(0x7fdda40add70 legacy=0x7fdda40d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:49.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.625+0000 7fdda0a00640 1 --2- 172.21.15.196:0/2478256616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd88003a00 0x7fdd88003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:49.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.625+0000 7fdda0a00640 1 --2- 172.21.15.196:0/2478256616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd88004800 0x7fdd88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:49.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.625+0000 7fdda0a00640 1 --2- 172.21.15.196:0/2478256616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd880051c0 0x7fdd8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:49.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.625+0000 7fdda0a00640 1 -- 172.21.15.196:0/2478256616 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41610c0 con 0x7fdd880051c0 2024-04-28T13:32:49.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.625+0000 7fdda0a00640 1 -- 172.21.15.196:0/2478256616 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8800c0c0 con 0x7fdd88003a00 2024-04-28T13:32:49.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.625+0000 7fdda0a00640 1 -- 172.21.15.196:0/2478256616 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdd8800c2c0 con 0x7fdd88004800 2024-04-28T13:32:49.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.626+0000 7fdda2800640 1 --2- 172.21.15.196:0/2478256616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd880051c0 0x7fdd8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:49.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.626+0000 7fdda1400640 1 --2- 172.21.15.196:0/2478256616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd88004800 0x7fdd88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:49.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.626+0000 7fdda1e00640 1 --2- 172.21.15.196:0/2478256616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd88003a00 0x7fdd88003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:49.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.626+0000 7fdda2800640 1 -- 172.21.15.196:0/2478256616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd88004800 msgr2=0x7fdd88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:49.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.626+0000 7fdda2800640 1 --2- 172.21.15.196:0/2478256616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd88004800 0x7fdd88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:49.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.626+0000 7fdda2800640 1 -- 172.21.15.196:0/2478256616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd88003a00 msgr2=0x7fdd88003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:49.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.626+0000 7fdda2800640 1 --2- 172.21.15.196:0/2478256616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd88003a00 0x7fdd88003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:49.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.626+0000 7fdda2800640 1 -- 172.21.15.196:0/2478256616 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd88003520 con 0x7fdd880051c0 2024-04-28T13:32:49.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.627+0000 7fdda2800640 1 -- 172.21.15.196:0/2478256616 reap_dead start 2024-04-28T13:32:49.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.627+0000 7fdda2800640 1 --2- 172.21.15.196:0/2478256616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd880051c0 0x7fdd8800b9c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009fc0 tx=0x7fdd88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae9d0284d1f8e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:49.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.627+0000 7fdda0a00640 1 -- 172.21.15.196:0/2478256616 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c035020 con 0x7fdd880051c0 2024-04-28T13:32:49.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.628+0000 7fdda0a00640 1 -- 172.21.15.196:0/2478256616 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd9c003bd0 con 0x7fdd880051c0 2024-04-28T13:32:49.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.628+0000 7fdda0a00640 1 -- 172.21.15.196:0/2478256616 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c009c10 con 0x7fdd880051c0 2024-04-28T13:32:49.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.628+0000 7fdda3200640 1 -- 172.21.15.196:0/2478256616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd880051c0 msgr2=0x7fdd8800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:49.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.628+0000 7fdda3200640 1 --2- 172.21.15.196:0/2478256616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd880051c0 0x7fdd8800b9c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009fc0 tx=0x7fdd88005830 comp rx=0 tx=0).stop 2024-04-28T13:32:49.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.628+0000 7fdda3200640 1 -- 172.21.15.196:0/2478256616 shutdown_connections 2024-04-28T13:32:49.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.628+0000 7fdda3200640 1 --2- 172.21.15.196:0/2478256616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd880051c0 0x7fdd8800b9c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:49.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.628+0000 7fdda3200640 1 -- 172.21.15.196:0/2478256616 >> 172.21.15.196:0/2478256616 conn(0x7fdda406c6c0 msgr2=0x7fdda406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:49.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.628+0000 7fdda3200640 1 -- 172.21.15.196:0/2478256616 shutdown_connections 2024-04-28T13:32:49.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.628+0000 7fdda3200640 1 -- 172.21.15.196:0/2478256616 wait complete. 2024-04-28T13:32:49.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.630+0000 7fdda3200640 1 Processor -- start 2024-04-28T13:32:49.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.630+0000 7fdda3200640 1 -- start start 2024-04-28T13:32:49.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.631+0000 7fdda3200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd880051c0 0x7fdda409da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:49.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.631+0000 7fdda3200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdda40ca010 0x7fdda40b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:49.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.631+0000 7fdda3200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdda40b5290 0x7fdda40b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:49.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.631+0000 7fdda3200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdda415ad80 con 0x7fdda40ca010 2024-04-28T13:32:49.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.631+0000 7fdda3200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdda40d06e0 con 0x7fdd880051c0 2024-04-28T13:32:49.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.631+0000 7fdda3200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdda409dfc0 con 0x7fdda40b5290 2024-04-28T13:32:49.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.632+0000 7fdda1400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdda40ca010 0x7fdda40b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:49.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.632+0000 7fdda1400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdda40ca010 0x7fdda40b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43364/0 (socket says 172.21.15.196:43364) 2024-04-28T13:32:49.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.632+0000 7fdda1400640 1 -- 172.21.15.196:0/2799120583 learned_addr learned my addr 172.21.15.196:0/2799120583 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:49.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.632+0000 7fdda1e00640 1 --2- 172.21.15.196:0/2799120583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd880051c0 0x7fdda409da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:49.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.632+0000 7fdda1400640 1 -- 172.21.15.196:0/2799120583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdda40b5290 msgr2=0x7fdda40b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:49.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.632+0000 7fdda1400640 1 --2- 172.21.15.196:0/2799120583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdda40b5290 0x7fdda40b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:49.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.632+0000 7fdda1400640 1 -- 172.21.15.196:0/2799120583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd880051c0 msgr2=0x7fdda409da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:49.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.632+0000 7fdda1400640 1 --2- 172.21.15.196:0/2799120583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd880051c0 0x7fdda409da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:49.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.632+0000 7fdda1400640 1 -- 172.21.15.196:0/2799120583 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda409e540 con 0x7fdda40ca010 2024-04-28T13:32:49.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.632+0000 7fdda1400640 1 --2- 172.21.15.196:0/2799120583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdda40ca010 0x7fdda40b31a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdd900018a0 tx=0x7fdd9000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:49.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.632+0000 7fdd9a000640 1 -- 172.21.15.196:0/2799120583 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9000c6c0 con 0x7fdda40ca010 2024-04-28T13:32:49.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.633+0000 7fdd9a000640 1 -- 172.21.15.196:0/2799120583 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd90018070 con 0x7fdda40ca010 2024-04-28T13:32:49.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.633+0000 7fdda3200640 1 -- 172.21.15.196:0/2799120583 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda40ca4f0 con 0x7fdda40ca010 2024-04-28T13:32:49.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.633+0000 7fdd9a000640 1 -- 172.21.15.196:0/2799120583 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9000cae0 con 0x7fdda40ca010 2024-04-28T13:32:49.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.633+0000 7fdda3200640 1 -- 172.21.15.196:0/2799120583 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda40caab0 con 0x7fdda40ca010 2024-04-28T13:32:49.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.634+0000 7fdda3200640 1 -- 172.21.15.196:0/2799120583 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda40b11e0 con 0x7fdda40ca010 2024-04-28T13:32:49.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.635+0000 7fdd9a000640 1 -- 172.21.15.196:0/2799120583 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fdd9000cc80 con 0x7fdda40ca010 2024-04-28T13:32:49.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.636+0000 7fdd9a000640 1 --2- 172.21.15.196:0/2799120583 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdd8406c980 0x7fdd8406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:49.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.636+0000 7fdd9a000640 1 -- 172.21.15.196:0/2799120583 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdd9008b970 con 0x7fdda40ca010 2024-04-28T13:32:49.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.637+0000 7fdda1e00640 1 --2- 172.21.15.196:0/2799120583 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdd8406c980 0x7fdd8406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:49.637 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.638+0000 7fdda1e00640 1 --2- 172.21.15.196:0/2799120583 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdd8406c980 0x7fdd8406ee40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdd94007fd0 tx=0x7fdd94009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:49.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.640+0000 7fdd9a000640 1 -- 172.21.15.196:0/2799120583 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd90059d40 con 0x7fdda40ca010 2024-04-28T13:32:49.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.881+0000 7fdda3200640 1 -- 172.21.15.196:0/2799120583 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdda409f0f0 con 0x7fdda40ca010 2024-04-28T13:32:49.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.882+0000 7fdd9a000640 1 -- 172.21.15.196:0/2799120583 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7fdd90059640 con 0x7fdda40ca010 2024-04-28T13:32:49.882 INFO:teuthology.orchestra.run.smithi196.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-28T13:32:49.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.885+0000 7fdda3200640 1 -- 172.21.15.196:0/2799120583 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdd8406c980 msgr2=0x7fdd8406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:49.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.885+0000 7fdda3200640 1 --2- 172.21.15.196:0/2799120583 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdd8406c980 0x7fdd8406ee40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdd94007fd0 tx=0x7fdd94009040 comp rx=0 tx=0).stop 2024-04-28T13:32:49.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.885+0000 7fdda3200640 1 -- 172.21.15.196:0/2799120583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdda40ca010 msgr2=0x7fdda40b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:49.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.885+0000 7fdda3200640 1 --2- 172.21.15.196:0/2799120583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdda40ca010 0x7fdda40b31a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdd900018a0 tx=0x7fdd9000ec90 comp rx=0 tx=0).stop 2024-04-28T13:32:49.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.885+0000 7fdda3200640 1 -- 172.21.15.196:0/2799120583 shutdown_connections 2024-04-28T13:32:49.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.885+0000 7fdda3200640 1 --2- 172.21.15.196:0/2799120583 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdd8406c980 0x7fdd8406ee40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:49.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.885+0000 7fdda3200640 1 --2- 172.21.15.196:0/2799120583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdda40b5290 0x7fdda40b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:49.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.885+0000 7fdda3200640 1 --2- 172.21.15.196:0/2799120583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdda40ca010 0x7fdda40b31a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:49.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.885+0000 7fdda3200640 1 --2- 172.21.15.196:0/2799120583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd880051c0 0x7fdda409da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:49.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.885+0000 7fdda3200640 1 -- 172.21.15.196:0/2799120583 >> 172.21.15.196:0/2799120583 conn(0x7fdda406c6c0 msgr2=0x7fdda4071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:49.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.885+0000 7fdda3200640 1 -- 172.21.15.196:0/2799120583 shutdown_connections 2024-04-28T13:32:49.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:49.885+0000 7fdda3200640 1 -- 172.21.15.196:0/2799120583 wait complete. 2024-04-28T13:32:49.904 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-28T13:32:49.904 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T13:32:49.904 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-04-28T13:32:49.904 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-28T13:32:49.904 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-28T13:32:49.904 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-28T13:32:49.904 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 2024-04-28T13:32:49.904 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T13:32:49.904 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-28T13:32:49.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:49.904 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-04-28T13:32:49.957 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:32:49.957 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:32:49.957 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:32:49.957 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:49.957 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-28T13:32:49.957 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:49.957 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:49.957 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:49.957 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-04-28T13:32:49.957 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:49.958 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:49.958 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-04-28T13:32:49.958 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:49.958 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:49.958 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T13:32:49.958 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:49.958 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T13:32:49.958 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42581sec preferred_lft 42581sec 2024-04-28T13:32:49.958 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link 2024-04-28T13:32:49.958 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:49.958 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T13:32:49.958 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 1e:4d:24:54:f5:c3 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:49.958 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-28T13:32:49.958 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:49.958 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::3cb7:a8ff:fe55:18c0/64 scope link 2024-04-28T13:32:49.959 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:49.959 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:32:49.959 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether c6:b9:8c:18:7a:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T13:32:49.959 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::c4b9:8cff:fe18:7aec/64 scope link 2024-04-28T13:32:49.959 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:49.959 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:32:49.959 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 1e:4d:24:54:f5:c3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T13:32:49.959 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::1c4d:24ff:fe54:f5c3/64 scope link 2024-04-28T13:32:49.959 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:49.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:49.959 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T13:32:50.036 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-04-28T13:32:50.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:50.042 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-04-28T13:32:50.095 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-04-28T13:32:50.095 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T13:32:50.095 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T13:32:50.095 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T13:32:50.095 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T13:32:50.096 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:32:50.096 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-28T13:32:50.096 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T13:32:50.096 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T13:32:50.096 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T13:32:50.096 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-28T13:32:50.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:32:50.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:32:50.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:50.303 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-28T13:32:50.354 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-28T13:32:50.354 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-28T13:32:50.355 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-28T13:32:50.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:50.355 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-04-28T13:32:50.435 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:32:50.435 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-28T13:32:50.489 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-28T13:32:50.489 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:32:50.489 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-28T13:32:50.539 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 2024-04-28T13:32:50.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:50.540 DEBUG:teuthology.orchestra.run.smithi196:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:32:50.668 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:32:50.668 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-28T13:32:50.668 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:32:50.688 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:32:50.689 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-28T13:32:50.689 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:32:50.689 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:32:50.689 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:32:50.716 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:32:50.716 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-28T13:32:50.716 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:32:50.717 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:32:50.717 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:32:50.717 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-28T13:32:50.717 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790:/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 2024-04-28T13:32:50.717 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=270aea32-2e16-4a97-85f7-aeae39bc8c76" will pass to kernel 2024-04-28T13:32:50.719 INFO:teuthology.orchestra.run.smithi196.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-28T13:32:50.719 INFO:teuthology.orchestra.run.smithi196.stdout: You just mounted a file system that supports labels which does not 2024-04-28T13:32:50.719 INFO:teuthology.orchestra.run.smithi196.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-28T13:32:50.719 INFO:teuthology.orchestra.run.smithi196.stdout: applications will generate AVC messages and not be allowed access to 2024-04-28T13:32:50.719 INFO:teuthology.orchestra.run.smithi196.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-28T13:32:50.724 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-28T13:32:50.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:50.724 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-28T13:32:50.797 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:32:50.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:32:50.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:32:50.940 INFO:teuthology.orchestra.run.smithi196.stdout:client5340 2024-04-28T13:32:50.940 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T13:32:51.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.141+0000 7f3ca5e00640 1 Processor -- start 2024-04-28T13:32:51.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.141+0000 7f3ca5e00640 1 -- start start 2024-04-28T13:32:51.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.143+0000 7f3ca5e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015f050 0x7f3ca00b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:51.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.143+0000 7f3ca5e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3ca006f830 0x7f3ca00d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:51.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.143+0000 7f3ca5e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3ca00d15c0 0x7f3ca0162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:51.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.143+0000 7f3ca5e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3ca006dce0 con 0x7f3ca00d15c0 2024-04-28T13:32:51.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.143+0000 7f3ca5e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3ca006dfc0 con 0x7f3ca006f830 2024-04-28T13:32:51.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.143+0000 7f3ca5e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca006e2a0 con 0x7f3ca015f050 2024-04-28T13:32:51.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.143+0000 7f3ca5400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3ca00d15c0 0x7f3ca0162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:51.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.143+0000 7f3ca5400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3ca00d15c0 0x7f3ca0162d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43372/0 (socket says 172.21.15.196:43372) 2024-04-28T13:32:51.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.143+0000 7f3ca5400640 1 -- 172.21.15.196:0/2999022153 learned_addr learned my addr 172.21.15.196:0/2999022153 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:51.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.144+0000 7f3c9fe00640 1 --2- 172.21.15.196:0/2999022153 >> v2:172.21.15.203:3300/0 conn(0x7f3ca006f830 0x7f3ca00d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:51.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.144+0000 7f3ca4a00640 1 --2- 172.21.15.196:0/2999022153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015f050 0x7f3ca00b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:51.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.144+0000 7f3ca5400640 1 -- 172.21.15.196:0/2999022153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015f050 msgr2=0x7f3ca00b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:51.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.144+0000 7f3ca5400640 1 --2- 172.21.15.196:0/2999022153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015f050 0x7f3ca00b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:51.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.144+0000 7f3ca5400640 1 -- 172.21.15.196:0/2999022153 >> v2:172.21.15.203:3300/0 conn(0x7f3ca006f830 msgr2=0x7f3ca00d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:51.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.144+0000 7f3ca5400640 1 --2- 172.21.15.196:0/2999022153 >> v2:172.21.15.203:3300/0 conn(0x7f3ca006f830 0x7f3ca00d1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:51.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.144+0000 7f3ca5400640 1 -- 172.21.15.196:0/2999022153 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca006e580 con 0x7f3ca00d15c0 2024-04-28T13:32:51.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.145+0000 7f3ca5400640 1 --2- 172.21.15.196:0/2999022153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ca00d15c0 0x7f3ca0162d00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00d1cc0 tx=0x7f3c9802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23678e5ab9c81d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:51.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.146+0000 7f3c9f400640 1 -- 172.21.15.196:0/2999022153 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98036070 con 0x7f3ca00d15c0 2024-04-28T13:32:51.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.146+0000 7f3c9f400640 1 -- 172.21.15.196:0/2999022153 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c98031600 con 0x7f3ca00d15c0 2024-04-28T13:32:51.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.146+0000 7f3c9f400640 1 -- 172.21.15.196:0/2999022153 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c980319d0 con 0x7f3ca00d15c0 2024-04-28T13:32:51.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.147+0000 7f3ca5e00640 1 -- 172.21.15.196:0/2999022153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ca00d15c0 msgr2=0x7f3ca0162d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:51.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.147+0000 7f3ca5e00640 1 --2- 172.21.15.196:0/2999022153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ca00d15c0 0x7f3ca0162d00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00d1cc0 tx=0x7f3c9802a6a0 comp rx=0 tx=0).stop 2024-04-28T13:32:51.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.147+0000 7f3ca5e00640 1 -- 172.21.15.196:0/2999022153 shutdown_connections 2024-04-28T13:32:51.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.147+0000 7f3ca5e00640 1 --2- 172.21.15.196:0/2999022153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ca00d15c0 0x7f3ca0162d00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:51.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.147+0000 7f3ca5e00640 1 --2- 172.21.15.196:0/2999022153 >> v2:172.21.15.203:3300/0 conn(0x7f3ca006f830 0x7f3ca00d1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:51.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.147+0000 7f3ca5e00640 1 --2- 172.21.15.196:0/2999022153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015f050 0x7f3ca00b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:51.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.147+0000 7f3ca5e00640 1 -- 172.21.15.196:0/2999022153 >> 172.21.15.196:0/2999022153 conn(0x7f3ca006c6c0 msgr2=0x7f3ca006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:51.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.147+0000 7f3ca5e00640 1 -- 172.21.15.196:0/2999022153 shutdown_connections 2024-04-28T13:32:51.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.147+0000 7f3ca5e00640 1 -- 172.21.15.196:0/2999022153 wait complete. 2024-04-28T13:32:51.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.149+0000 7f3ca5e00640 1 Processor -- start 2024-04-28T13:32:51.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.149+0000 7f3ca5e00640 1 -- start start 2024-04-28T13:32:51.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.150+0000 7f3ca5e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ca006f830 0x7f3ca014e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:51.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.150+0000 7f3ca5e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ca00d15c0 0x7f3ca00b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:51.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.150+0000 7f3ca5e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015f050 0x7f3ca00b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:51.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.150+0000 7f3ca5e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca015f430 con 0x7f3ca00d15c0 2024-04-28T13:32:51.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.150+0000 7f3ca5e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca006fc50 con 0x7f3ca006f830 2024-04-28T13:32:51.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.150+0000 7f3ca5e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca015f5b0 con 0x7f3ca015f050 2024-04-28T13:32:51.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.151+0000 7f3c9fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ca00d15c0 0x7f3ca00b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:51.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.151+0000 7f3c9fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ca00d15c0 0x7f3ca00b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43386/0 (socket says 172.21.15.196:43386) 2024-04-28T13:32:51.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.151+0000 7f3c9fe00640 1 -- 172.21.15.196:0/1929234850 learned_addr learned my addr 172.21.15.196:0/1929234850 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:51.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.151+0000 7f3ca4a00640 1 --2- 172.21.15.196:0/1929234850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ca006f830 0x7f3ca014e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:51.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.151+0000 7f3ca5400640 1 --2- 172.21.15.196:0/1929234850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015f050 0x7f3ca00b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:51.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.151+0000 7f3c9fe00640 1 -- 172.21.15.196:0/1929234850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015f050 msgr2=0x7f3ca00b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:51.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.151+0000 7f3c9fe00640 1 --2- 172.21.15.196:0/1929234850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015f050 0x7f3ca00b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:51.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.151+0000 7f3c9fe00640 1 -- 172.21.15.196:0/1929234850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ca006f830 msgr2=0x7f3ca014e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:51.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.151+0000 7f3c9fe00640 1 --2- 172.21.15.196:0/1929234850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ca006f830 0x7f3ca014e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:51.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.151+0000 7f3c9fe00640 1 -- 172.21.15.196:0/1929234850 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca015c6e0 con 0x7f3ca00d15c0 2024-04-28T13:32:51.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.152+0000 7f3c9fe00640 1 --2- 172.21.15.196:0/1929234850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ca00d15c0 0x7f3ca00b08e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00d800 tx=0x7f3c8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:51.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.152+0000 7f3c9cc00640 1 -- 172.21.15.196:0/1929234850 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c018020 con 0x7f3ca00d15c0 2024-04-28T13:32:51.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.152+0000 7f3c9cc00640 1 -- 172.21.15.196:0/1929234850 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c8c0096a0 con 0x7f3ca00d15c0 2024-04-28T13:32:51.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.152+0000 7f3ca5e00640 1 -- 172.21.15.196:0/1929234850 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca015c950 con 0x7f3ca00d15c0 2024-04-28T13:32:51.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.152+0000 7f3c9cc00640 1 -- 172.21.15.196:0/1929234850 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c009a70 con 0x7f3ca00d15c0 2024-04-28T13:32:51.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.152+0000 7f3ca5e00640 1 -- 172.21.15.196:0/1929234850 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca015cf10 con 0x7f3ca00d15c0 2024-04-28T13:32:51.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.153+0000 7f3ca5e00640 1 -- 172.21.15.196:0/1929234850 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca009f0f0 con 0x7f3ca00d15c0 2024-04-28T13:32:51.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.154+0000 7f3c9cc00640 1 -- 172.21.15.196:0/1929234850 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f3c8c01b020 con 0x7f3ca00d15c0 2024-04-28T13:32:51.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.155+0000 7f3c9cc00640 1 --2- 172.21.15.196:0/1929234850 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3c8006c980 0x7f3c8006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:51.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.155+0000 7f3c9cc00640 1 -- 172.21.15.196:0/1929234850 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3c8c08c210 con 0x7f3ca00d15c0 2024-04-28T13:32:51.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.156+0000 7f3ca4a00640 1 --2- 172.21.15.196:0/1929234850 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3c8006c980 0x7f3c8006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:51.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.157+0000 7f3ca4a00640 1 --2- 172.21.15.196:0/1929234850 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3c8006c980 0x7f3c8006ee40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3c94006fd0 tx=0x7f3c94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:51.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.160+0000 7f3c9cc00640 1 -- 172.21.15.196:0/1929234850 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3c8c022020 con 0x7f3ca00d15c0 2024-04-28T13:32:51.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.402+0000 7f3ca5e00640 1 -- 172.21.15.196:0/1929234850 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3ca00507a0 con 0x7f3ca00d15c0 2024-04-28T13:32:51.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.403+0000 7f3c9cc00640 1 -- 172.21.15.196:0/1929234850 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f3c8c059f00 con 0x7f3ca00d15c0 2024-04-28T13:32:51.402 INFO:teuthology.orchestra.run.smithi196.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-28T13:32:51.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.405+0000 7f3ca5e00640 1 -- 172.21.15.196:0/1929234850 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3c8006c980 msgr2=0x7f3c8006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:51.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.405+0000 7f3ca5e00640 1 --2- 172.21.15.196:0/1929234850 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3c8006c980 0x7f3c8006ee40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3c94006fd0 tx=0x7f3c94008040 comp rx=0 tx=0).stop 2024-04-28T13:32:51.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.405+0000 7f3ca5e00640 1 -- 172.21.15.196:0/1929234850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ca00d15c0 msgr2=0x7f3ca00b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:51.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.405+0000 7f3ca5e00640 1 --2- 172.21.15.196:0/1929234850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ca00d15c0 0x7f3ca00b08e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00d800 tx=0x7f3c8c00dcc0 comp rx=0 tx=0).stop 2024-04-28T13:32:51.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.405+0000 7f3ca5e00640 1 -- 172.21.15.196:0/1929234850 shutdown_connections 2024-04-28T13:32:51.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.405+0000 7f3ca5e00640 1 --2- 172.21.15.196:0/1929234850 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3c8006c980 0x7f3c8006ee40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:51.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.405+0000 7f3ca5e00640 1 --2- 172.21.15.196:0/1929234850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015f050 0x7f3ca00b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:51.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.405+0000 7f3ca5e00640 1 --2- 172.21.15.196:0/1929234850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ca00d15c0 0x7f3ca00b08e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:51.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.405+0000 7f3ca5e00640 1 --2- 172.21.15.196:0/1929234850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ca006f830 0x7f3ca014e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:51.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.405+0000 7f3ca5e00640 1 -- 172.21.15.196:0/1929234850 >> 172.21.15.196:0/1929234850 conn(0x7f3ca006c6c0 msgr2=0x7f3ca00b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:51.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.406+0000 7f3ca5e00640 1 -- 172.21.15.196:0/1929234850 shutdown_connections 2024-04-28T13:32:51.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:51.406+0000 7f3ca5e00640 1 -- 172.21.15.196:0/1929234850 wait complete. 2024-04-28T13:32:51.424 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-28T13:32:51.424 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T13:32:51.424 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2024-04-28T13:32:51.424 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-28T13:32:51.424 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-28T13:32:51.424 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-28T13:32:51.424 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 2024-04-28T13:32:51.424 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T13:32:51.424 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-28T13:32:51.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:51.424 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2024-04-28T13:32:51.483 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:32:51.483 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:32:51.483 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:32:51.483 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:51.483 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-28T13:32:51.483 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:51.483 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:51.483 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:51.483 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f0 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f1 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42548sec preferred_lft 42548sec 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ec4:7aff:fe8f:4a23/64 scope link 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 1e:04:67:22:4e:82 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:51.484 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::4e3:c8ff:fea0:8dce/64 scope link 2024-04-28T13:32:51.485 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:51.485 INFO:teuthology.orchestra.run.smithi203.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:32:51.485 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 1e:04:67:22:4e:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T13:32:51.485 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::1c04:67ff:fe22:4e82/64 scope link 2024-04-28T13:32:51.485 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:51.485 INFO:teuthology.orchestra.run.smithi203.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:32:51.485 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ae:7c:61:1f:29:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-28T13:32:51.485 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ac7c:61ff:fe1f:297b/64 scope link 2024-04-28T13:32:51.485 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:51.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:51.486 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T13:32:51.566 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-28T13:32:51.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:51.570 DEBUG:teuthology.orchestra.run.smithi203:> route 2024-04-28T13:32:51.625 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2024-04-28T13:32:51.625 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T13:32:51.626 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T13:32:51.626 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T13:32:51.626 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T13:32:51.626 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:32:51.626 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-28T13:32:51.626 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T13:32:51.626 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T13:32:51.626 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T13:32:51.626 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-28T13:32:51.731 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:32:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:32:51.825 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:32:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:32:51.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:51.836 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-28T13:32:51.889 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-28T13:32:51.890 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-28T13:32:51.890 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-28T13:32:51.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:51.890 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2024-04-28T13:32:51.977 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:32:51.978 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-28T13:32:52.025 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-28T13:32:52.026 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:32:52.026 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-28T13:32:52.070 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 2024-04-28T13:32:52.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:52.070 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:32:52.243 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:32:52.243 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-28T13:32:52.243 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:32:52.263 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:32:52.263 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-28T13:32:52.263 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:32:52.263 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:32:52.264 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:32:52.292 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:32:52.292 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-28T13:32:52.292 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:32:52.292 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:32:52.292 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:32:52.292 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-28T13:32:52.293 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790:/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 2024-04-28T13:32:52.293 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=270aea32-2e16-4a97-85f7-aeae39bc8c76" will pass to kernel 2024-04-28T13:32:52.295 INFO:teuthology.orchestra.run.smithi203.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-28T13:32:52.295 INFO:teuthology.orchestra.run.smithi203.stdout: You just mounted a file system that supports labels which does not 2024-04-28T13:32:52.295 INFO:teuthology.orchestra.run.smithi203.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-28T13:32:52.295 INFO:teuthology.orchestra.run.smithi203.stdout: applications will generate AVC messages and not be allowed access to 2024-04-28T13:32:52.295 INFO:teuthology.orchestra.run.smithi203.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-28T13:32:52.299 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-28T13:32:52.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:52.299 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-28T13:32:52.369 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:32:52.468 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:32:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:32:52.473 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:32:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:32:52.483 INFO:teuthology.orchestra.run.smithi203.stdout:client5349 2024-04-28T13:32:52.483 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T13:32:52.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.689+0000 7fc2b5600640 1 Processor -- start 2024-04-28T13:32:52.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.690+0000 7fc2b5600640 1 -- start start 2024-04-28T13:32:52.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.691+0000 7fc2b5600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc2b01485b0 0x7fc2b00afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:52.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.691+0000 7fc2b5600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2b0073900 0x7fc2b006fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:52.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.691+0000 7fc2b5600640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2b00ae8b0 con 0x7fc2b006fa50 2024-04-28T13:32:52.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.691+0000 7fc2b5600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc2b00aeb90 con 0x7fc2b01485b0 2024-04-28T13:32:52.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.691+0000 7fc2b5600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc2b00aee70 con 0x7fc2b0073900 2024-04-28T13:32:52.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.691+0000 7fc2aea00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc2b01485b0 0x7fc2b00afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:52.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.692+0000 7fc2aea00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc2b01485b0 0x7fc2b00afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41776/0 (socket says 172.21.15.196:41776) 2024-04-28T13:32:52.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.692+0000 7fc2aea00640 1 -- 172.21.15.196:0/854918549 learned_addr learned my addr 172.21.15.196:0/854918549 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:52.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.692+0000 7fc2aea00640 1 -- 172.21.15.196:0/854918549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2b0073900 msgr2=0x7fc2b006fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:52.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.692+0000 7fc2aea00640 1 --2- 172.21.15.196:0/854918549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2b0073900 0x7fc2b006fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:52.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.692+0000 7fc2aea00640 1 -- 172.21.15.196:0/854918549 >> v1:172.21.15.203:6789/0 conn(0x7fc2b006fa50 legacy=0x7fc2b00731f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:52.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.692+0000 7fc2aea00640 1 -- 172.21.15.196:0/854918549 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b00af150 con 0x7fc2b01485b0 2024-04-28T13:32:52.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.693+0000 7fc2aea00640 1 --2- 172.21.15.196:0/854918549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2b01485b0 0x7fc2b00afc30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009870 tx=0x7fc2a002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21c96cabd708da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:52.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.693+0000 7fc2ae000640 1 -- 172.21.15.196:0/854918549 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0037070 con 0x7fc2b01485b0 2024-04-28T13:32:52.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.693+0000 7fc2ae000640 1 -- 172.21.15.196:0/854918549 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a0032440 con 0x7fc2b01485b0 2024-04-28T13:32:52.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.693+0000 7fc2ae000640 1 -- 172.21.15.196:0/854918549 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0032810 con 0x7fc2b01485b0 2024-04-28T13:32:52.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.694+0000 7fc2b5600640 1 -- 172.21.15.196:0/854918549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2b01485b0 msgr2=0x7fc2b00afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:52.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.694+0000 7fc2b5600640 1 --2- 172.21.15.196:0/854918549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2b01485b0 0x7fc2b00afc30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009870 tx=0x7fc2a002a230 comp rx=0 tx=0).stop 2024-04-28T13:32:52.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.694+0000 7fc2b5600640 1 -- 172.21.15.196:0/854918549 shutdown_connections 2024-04-28T13:32:52.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.694+0000 7fc2b5600640 1 --2- 172.21.15.196:0/854918549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2b0073900 0x7fc2b006fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:52.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.694+0000 7fc2b5600640 1 --2- 172.21.15.196:0/854918549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2b01485b0 0x7fc2b00afc30 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:52.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.694+0000 7fc2b5600640 1 -- 172.21.15.196:0/854918549 >> 172.21.15.196:0/854918549 conn(0x7fc2b00d0d30 msgr2=0x7fc2b00d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:52.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.694+0000 7fc2b5600640 1 -- 172.21.15.196:0/854918549 shutdown_connections 2024-04-28T13:32:52.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.695+0000 7fc2b5600640 1 -- 172.21.15.196:0/854918549 wait complete. 2024-04-28T13:32:52.697 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.697+0000 7fc2b5600640 1 Processor -- start 2024-04-28T13:32:52.697 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.697+0000 7fc2b5600640 1 -- start start 2024-04-28T13:32:52.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.698+0000 7fc2b5600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2b006fa50 0x7fc2b01400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:52.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.698+0000 7fc2b5600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b0073900 0x7fc2b01405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:52.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.698+0000 7fc2b5600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2b01485b0 0x7fc2b0155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:52.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.698+0000 7fc2b5600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b00b0170 con 0x7fc2b01485b0 2024-04-28T13:32:52.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.698+0000 7fc2b5600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b00af710 con 0x7fc2b0073900 2024-04-28T13:32:52.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.698+0000 7fc2b5600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc2b00b4cd0 con 0x7fc2b006fa50 2024-04-28T13:32:52.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.698+0000 7fc2af400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2b01485b0 0x7fc2b0155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:52.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.698+0000 7fc2af400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2b01485b0 0x7fc2b0155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41780/0 (socket says 172.21.15.196:41780) 2024-04-28T13:32:52.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.698+0000 7fc2af400640 1 -- 172.21.15.196:0/1910463067 learned_addr learned my addr 172.21.15.196:0/1910463067 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:52.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.699+0000 7fc2aea00640 1 --2- 172.21.15.196:0/1910463067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2b006fa50 0x7fc2b01400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:52.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.699+0000 7fc2a7e00640 1 --2- 172.21.15.196:0/1910463067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b0073900 0x7fc2b01405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:52.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.699+0000 7fc2af400640 1 -- 172.21.15.196:0/1910463067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2b006fa50 msgr2=0x7fc2b01400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:52.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.699+0000 7fc2af400640 1 --2- 172.21.15.196:0/1910463067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2b006fa50 0x7fc2b01400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:52.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.699+0000 7fc2af400640 1 -- 172.21.15.196:0/1910463067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b0073900 msgr2=0x7fc2b01405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:52.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.699+0000 7fc2af400640 1 --2- 172.21.15.196:0/1910463067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b0073900 0x7fc2b01405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:52.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.699+0000 7fc2af400640 1 -- 172.21.15.196:0/1910463067 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b013be40 con 0x7fc2b01485b0 2024-04-28T13:32:52.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.699+0000 7fc2a7e00640 1 --2- 172.21.15.196:0/1910463067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b0073900 0x7fc2b01405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:32:52.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.699+0000 7fc2af400640 1 --2- 172.21.15.196:0/1910463067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2b01485b0 0x7fc2b0155130 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc2a800b9e0 tx=0x7fc2a800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:52.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.699+0000 7fc2a6a00640 1 -- 172.21.15.196:0/1910463067 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8019070 con 0x7fc2b01485b0 2024-04-28T13:32:52.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.700+0000 7fc2a6a00640 1 -- 172.21.15.196:0/1910463067 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a800cac0 con 0x7fc2b01485b0 2024-04-28T13:32:52.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.700+0000 7fc2a6a00640 1 -- 172.21.15.196:0/1910463067 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8014440 con 0x7fc2b01485b0 2024-04-28T13:32:52.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.700+0000 7fc2b5600640 1 -- 172.21.15.196:0/1910463067 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b013c120 con 0x7fc2b01485b0 2024-04-28T13:32:52.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.700+0000 7fc2b5600640 1 -- 172.21.15.196:0/1910463067 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b013c6e0 con 0x7fc2b01485b0 2024-04-28T13:32:52.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.701+0000 7fc2b5600640 1 -- 172.21.15.196:0/1910463067 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b00b4010 con 0x7fc2b01485b0 2024-04-28T13:32:52.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.703+0000 7fc2a6a00640 1 -- 172.21.15.196:0/1910463067 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc2a801c070 con 0x7fc2b01485b0 2024-04-28T13:32:52.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.703+0000 7fc2a6a00640 1 --2- 172.21.15.196:0/1910463067 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc28c06c980 0x7fc28c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:52.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.704+0000 7fc2a6a00640 1 -- 172.21.15.196:0/1910463067 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc2a808b7e0 con 0x7fc2b01485b0 2024-04-28T13:32:52.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.704+0000 7fc2aea00640 1 --2- 172.21.15.196:0/1910463067 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc28c06c980 0x7fc28c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:52.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.705+0000 7fc2aea00640 1 --2- 172.21.15.196:0/1910463067 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc28c06c980 0x7fc28c06ee40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0034000 tx=0x7fc2a002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:52.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.709+0000 7fc2a6a00640 1 -- 172.21.15.196:0/1910463067 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc2b00b4010 con 0x7fc2b01485b0 2024-04-28T13:32:52.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.948+0000 7fc2b5600640 1 -- 172.21.15.196:0/1910463067 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc2b013c8e0 con 0x7fc2b01485b0 2024-04-28T13:32:52.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.948+0000 7fc2a6a00640 1 -- 172.21.15.196:0/1910463067 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7fc2b013c8e0 con 0x7fc2b01485b0 2024-04-28T13:32:52.948 INFO:teuthology.orchestra.run.smithi196.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-28T13:32:52.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.951+0000 7fc2b5600640 1 -- 172.21.15.196:0/1910463067 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc28c06c980 msgr2=0x7fc28c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:52.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.951+0000 7fc2b5600640 1 --2- 172.21.15.196:0/1910463067 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc28c06c980 0x7fc28c06ee40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0034000 tx=0x7fc2a002e000 comp rx=0 tx=0).stop 2024-04-28T13:32:52.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.951+0000 7fc2b5600640 1 -- 172.21.15.196:0/1910463067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2b01485b0 msgr2=0x7fc2b0155130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:52.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.951+0000 7fc2b5600640 1 --2- 172.21.15.196:0/1910463067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2b01485b0 0x7fc2b0155130 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc2a800b9e0 tx=0x7fc2a800bea0 comp rx=0 tx=0).stop 2024-04-28T13:32:52.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.951+0000 7fc2b5600640 1 -- 172.21.15.196:0/1910463067 shutdown_connections 2024-04-28T13:32:52.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.951+0000 7fc2b5600640 1 --2- 172.21.15.196:0/1910463067 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc28c06c980 0x7fc28c06ee40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:52.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.951+0000 7fc2b5600640 1 --2- 172.21.15.196:0/1910463067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2b01485b0 0x7fc2b0155130 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:52.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.951+0000 7fc2b5600640 1 --2- 172.21.15.196:0/1910463067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2b0073900 0x7fc2b01405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:52.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.951+0000 7fc2b5600640 1 --2- 172.21.15.196:0/1910463067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2b006fa50 0x7fc2b01400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:52.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.951+0000 7fc2b5600640 1 -- 172.21.15.196:0/1910463067 >> 172.21.15.196:0/1910463067 conn(0x7fc2b00d0d30 msgr2=0x7fc2b01458b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:52.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.951+0000 7fc2b5600640 1 -- 172.21.15.196:0/1910463067 shutdown_connections 2024-04-28T13:32:52.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:52.951+0000 7fc2b5600640 1 -- 172.21.15.196:0/1910463067 wait complete. 2024-04-28T13:32:52.970 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-28T13:32:52.970 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T13:32:52.971 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-04-28T13:32:52.971 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-28T13:32:52.971 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-28T13:32:52.971 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-28T13:32:52.971 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 2024-04-28T13:32:52.971 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T13:32:52.971 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-28T13:32:52.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:52.971 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-04-28T13:32:53.026 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:32:53.026 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:32:53.026 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:32:53.026 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:53.027 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-28T13:32:53.027 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:53.027 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:53.027 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:53.027 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-04-28T13:32:53.027 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:53.027 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:53.027 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-04-28T13:32:53.027 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:53.027 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:53.027 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T13:32:53.027 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:53.027 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T13:32:53.028 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42577sec preferred_lft 42577sec 2024-04-28T13:32:53.028 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link 2024-04-28T13:32:53.028 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:53.028 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T13:32:53.028 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 1e:4d:24:54:f5:c3 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:53.028 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-28T13:32:53.028 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:53.028 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::3cb7:a8ff:fe55:18c0/64 scope link 2024-04-28T13:32:53.028 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:53.028 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:32:53.028 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether c6:b9:8c:18:7a:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T13:32:53.028 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::c4b9:8cff:fe18:7aec/64 scope link 2024-04-28T13:32:53.028 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:53.029 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:32:53.029 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 1e:4d:24:54:f5:c3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T13:32:53.029 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::1c4d:24ff:fe54:f5c3/64 scope link 2024-04-28T13:32:53.029 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:53.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:53.029 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T13:32:53.097 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-04-28T13:32:53.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:53.101 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-04-28T13:32:53.148 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-04-28T13:32:53.148 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T13:32:53.148 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T13:32:53.148 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T13:32:53.148 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T13:32:53.148 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:32:53.148 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-28T13:32:53.148 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T13:32:53.149 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T13:32:53.149 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T13:32:53.149 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-28T13:32:53.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:32:53.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:32:53.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:53.404 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-28T13:32:53.453 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-28T13:32:53.454 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-28T13:32:53.454 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-28T13:32:53.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:53.455 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.1 up 2024-04-28T13:32:53.534 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:32:53.534 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-28T13:32:53.586 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-28T13:32:53.586 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:32:53.586 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-28T13:32:53.636 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 2024-04-28T13:32:53.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:53.637 DEBUG:teuthology.orchestra.run.smithi196:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-28T13:32:53.762 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-28T13:32:53.762 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-28T13:32:53.762 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:32:53.783 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-28T13:32:53.783 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-28T13:32:53.784 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:32:53.784 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:32:53.784 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:32:53.814 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-28T13:32:53.814 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-28T13:32:53.814 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:32:53.814 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:32:53.814 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:32:53.814 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-28T13:32:53.814 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790:/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 2024-04-28T13:32:53.814 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=270aea32-2e16-4a97-85f7-aeae39bc8c76" will pass to kernel 2024-04-28T13:32:53.817 INFO:teuthology.orchestra.run.smithi196.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-28T13:32:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: You just mounted a file system that supports labels which does not 2024-04-28T13:32:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-28T13:32:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: applications will generate AVC messages and not be allowed access to 2024-04-28T13:32:53.817 INFO:teuthology.orchestra.run.smithi196.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-28T13:32:53.821 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-28T13:32:53.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:53.821 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-28T13:32:53.891 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:32:53.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:32:53.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:32:54.006 INFO:teuthology.orchestra.run.smithi196.stdout:client4673 2024-04-28T13:32:54.006 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T13:32:54.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.205+0000 7fe5a2200640 1 Processor -- start 2024-04-28T13:32:54.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.205+0000 7fe5a2200640 1 -- start start 2024-04-28T13:32:54.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.206+0000 7fe5a2200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe59c0cf2e0 0x7fe59c06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:54.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.206+0000 7fe5a2200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe59c06e490 0x7fe59c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:54.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.206+0000 7fe5a2200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe59c0d1740 con 0x7fe59c0d11b0 2024-04-28T13:32:54.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.206+0000 7fe5a2200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe59c0d1a20 con 0x7fe59c0cf2e0 2024-04-28T13:32:54.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.206+0000 7fe5a2200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe59c0d1d00 con 0x7fe59c06e490 2024-04-28T13:32:54.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.207+0000 7fe59aa00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fe59c0d11b0 0x7fe59c06dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:45452/0 (socket says 172.21.15.196:45452) 2024-04-28T13:32:54.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.207+0000 7fe59aa00640 1 -- 172.21.15.196:0/2306449806 learned_addr learned my addr 172.21.15.196:0/2306449806 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:54.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.207+0000 7fe59b400640 1 --2- 172.21.15.196:0/2306449806 >> v2:172.21.15.196:3300/0 conn(0x7fe59c0cf2e0 0x7fe59c06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:54.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.207+0000 7fe59be00640 1 --2- 172.21.15.196:0/2306449806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe59c06e490 0x7fe59c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:54.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.207+0000 7fe59b400640 1 -- 172.21.15.196:0/2306449806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe59c06e490 msgr2=0x7fe59c15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:54.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.207+0000 7fe59b400640 1 --2- 172.21.15.196:0/2306449806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe59c06e490 0x7fe59c15ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:54.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.207+0000 7fe59b400640 1 -- 172.21.15.196:0/2306449806 >> v1:172.21.15.196:6789/0 conn(0x7fe59c0d11b0 legacy=0x7fe59c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:54.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.207+0000 7fe59b400640 1 -- 172.21.15.196:0/2306449806 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59c0d1fe0 con 0x7fe59c0cf2e0 2024-04-28T13:32:54.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.207+0000 7fe59a000640 1 -- 172.21.15.196:0/2306449806 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 990125767 0 0) 0x7fe59c0d1740 con 0x7fe59c0d11b0 2024-04-28T13:32:54.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.207+0000 7fe59be00640 1 --2- 172.21.15.196:0/2306449806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe59c06e490 0x7fe59c15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:54.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.208+0000 7fe59b400640 1 --2- 172.21.15.196:0/2306449806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c0cf2e0 0x7fe59c06d4d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0097d0 tx=0x7fe58c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec65f6c302c2e1fd server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:54.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.209+0000 7fe59a000640 1 -- 172.21.15.196:0/2306449806 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c037070 con 0x7fe59c0cf2e0 2024-04-28T13:32:54.209 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.209+0000 7fe59a000640 1 -- 172.21.15.196:0/2306449806 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe58c032440 con 0x7fe59c0cf2e0 2024-04-28T13:32:54.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.209+0000 7fe59a000640 1 -- 172.21.15.196:0/2306449806 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c032810 con 0x7fe59c0cf2e0 2024-04-28T13:32:54.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.209+0000 7fe5a2200640 1 -- 172.21.15.196:0/2306449806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c0cf2e0 msgr2=0x7fe59c06d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:54.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.209+0000 7fe5a2200640 1 --2- 172.21.15.196:0/2306449806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c0cf2e0 0x7fe59c06d4d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0097d0 tx=0x7fe58c02a1c0 comp rx=0 tx=0).stop 2024-04-28T13:32:54.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.209+0000 7fe5a2200640 1 -- 172.21.15.196:0/2306449806 shutdown_connections 2024-04-28T13:32:54.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.209+0000 7fe5a2200640 1 --2- 172.21.15.196:0/2306449806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe59c06e490 0x7fe59c15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:54.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.209+0000 7fe5a2200640 1 --2- 172.21.15.196:0/2306449806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c0cf2e0 0x7fe59c06d4d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:54.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.209+0000 7fe5a2200640 1 -- 172.21.15.196:0/2306449806 >> 172.21.15.196:0/2306449806 conn(0x7fe59c0b4a60 msgr2=0x7fe59c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:54.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.209+0000 7fe5a2200640 1 -- 172.21.15.196:0/2306449806 shutdown_connections 2024-04-28T13:32:54.210 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.210+0000 7fe5a2200640 1 -- 172.21.15.196:0/2306449806 wait complete. 2024-04-28T13:32:54.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.211+0000 7fe5a2200640 1 Processor -- start 2024-04-28T13:32:54.211 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.211+0000 7fe5a2200640 1 -- start start 2024-04-28T13:32:54.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.212+0000 7fe5a2200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c06e490 0x7fe59c0b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:54.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.212+0000 7fe5a2200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe59c0cf2e0 0x7fe59c161be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:54.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.212+0000 7fe5a2200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe59c0d11b0 0x7fe59c070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:54.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.212+0000 7fe5a2200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c0cf6c0 con 0x7fe59c06e490 2024-04-28T13:32:54.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.212+0000 7fe5a2200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c06da10 con 0x7fe59c0d11b0 2024-04-28T13:32:54.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.212+0000 7fe5a2200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe59c0cf840 con 0x7fe59c0cf2e0 2024-04-28T13:32:54.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.213+0000 7fe59b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c06e490 0x7fe59c0b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:54.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.213+0000 7fe59b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c06e490 0x7fe59c0b6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41792/0 (socket says 172.21.15.196:41792) 2024-04-28T13:32:54.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.213+0000 7fe59b400640 1 -- 172.21.15.196:0/2600231992 learned_addr learned my addr 172.21.15.196:0/2600231992 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:54.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.213+0000 7fe59b400640 1 -- 172.21.15.196:0/2600231992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe59c0cf2e0 msgr2=0x7fe59c161be0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-28T13:32:54.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.213+0000 7fe59b400640 1 --2- 172.21.15.196:0/2600231992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe59c0cf2e0 0x7fe59c161be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:54.213 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.213+0000 7fe59b400640 1 -- 172.21.15.196:0/2600231992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe59c0d11b0 msgr2=0x7fe59c070540 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-28T13:32:54.214 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.213+0000 7fe59b400640 1 --2- 172.21.15.196:0/2600231992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe59c0d11b0 0x7fe59c070540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:54.214 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.213+0000 7fe59b400640 1 -- 172.21.15.196:0/2600231992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59c0b85d0 con 0x7fe59c06e490 2024-04-28T13:32:54.214 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.213+0000 7fe59b400640 1 --2- 172.21.15.196:0/2600231992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c06e490 0x7fe59c0b6d90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe58c000c00 tx=0x7fe58c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:54.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.213+0000 7fe593400640 1 -- 172.21.15.196:0/2600231992 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c037040 con 0x7fe59c06e490 2024-04-28T13:32:54.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.214+0000 7fe593400640 1 -- 172.21.15.196:0/2600231992 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe58c042070 con 0x7fe59c06e490 2024-04-28T13:32:54.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.214+0000 7fe593400640 1 -- 172.21.15.196:0/2600231992 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c030a70 con 0x7fe59c06e490 2024-04-28T13:32:54.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.214+0000 7fe5a2200640 1 -- 172.21.15.196:0/2600231992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59c0b7530 con 0x7fe59c06e490 2024-04-28T13:32:54.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.214+0000 7fe5a2200640 1 -- 172.21.15.196:0/2600231992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe59c0b7af0 con 0x7fe59c06e490 2024-04-28T13:32:54.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.215+0000 7fe5a2200640 1 -- 172.21.15.196:0/2600231992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe59c06a290 con 0x7fe59c06e490 2024-04-28T13:32:54.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.216+0000 7fe593400640 1 -- 172.21.15.196:0/2600231992 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe58c030c10 con 0x7fe59c06e490 2024-04-28T13:32:54.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.217+0000 7fe593400640 1 --2- 172.21.15.196:0/2600231992 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe57c06c980 0x7fe57c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:54.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.217+0000 7fe593400640 1 -- 172.21.15.196:0/2600231992 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe58c0ac730 con 0x7fe59c06e490 2024-04-28T13:32:54.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.218+0000 7fe59aa00640 1 --2- 172.21.15.196:0/2600231992 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe57c06c980 0x7fe57c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:54.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.219+0000 7fe59aa00640 1 --2- 172.21.15.196:0/2600231992 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe57c06c980 0x7fe57c06ee40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe588007fd0 tx=0x7fe588007960 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:54.221 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.221+0000 7fe593400640 1 -- 172.21.15.196:0/2600231992 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe58c07aa70 con 0x7fe59c06e490 2024-04-28T13:32:54.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.463+0000 7fe5a2200640 1 -- 172.21.15.196:0/2600231992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe59c0b8ef0 con 0x7fe59c06e490 2024-04-28T13:32:54.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.463+0000 7fe593400640 1 -- 172.21.15.196:0/2600231992 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7fe58c07a370 con 0x7fe59c06e490 2024-04-28T13:32:54.463 INFO:teuthology.orchestra.run.smithi196.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-28T13:32:54.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.466+0000 7fe5a2200640 1 -- 172.21.15.196:0/2600231992 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe57c06c980 msgr2=0x7fe57c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:54.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.466+0000 7fe5a2200640 1 --2- 172.21.15.196:0/2600231992 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe57c06c980 0x7fe57c06ee40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe588007fd0 tx=0x7fe588007960 comp rx=0 tx=0).stop 2024-04-28T13:32:54.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.466+0000 7fe5a2200640 1 -- 172.21.15.196:0/2600231992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c06e490 msgr2=0x7fe59c0b6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:54.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.466+0000 7fe5a2200640 1 --2- 172.21.15.196:0/2600231992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c06e490 0x7fe59c0b6d90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe58c000c00 tx=0x7fe58c0046c0 comp rx=0 tx=0).stop 2024-04-28T13:32:54.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.466+0000 7fe5a2200640 1 -- 172.21.15.196:0/2600231992 shutdown_connections 2024-04-28T13:32:54.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.466+0000 7fe5a2200640 1 --2- 172.21.15.196:0/2600231992 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe57c06c980 0x7fe57c06ee40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:54.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.466+0000 7fe5a2200640 1 --2- 172.21.15.196:0/2600231992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe59c0d11b0 0x7fe59c070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:54.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.466+0000 7fe5a2200640 1 --2- 172.21.15.196:0/2600231992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe59c0cf2e0 0x7fe59c161be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:54.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.466+0000 7fe5a2200640 1 --2- 172.21.15.196:0/2600231992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c06e490 0x7fe59c0b6d90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:54.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.466+0000 7fe5a2200640 1 -- 172.21.15.196:0/2600231992 >> 172.21.15.196:0/2600231992 conn(0x7fe59c0b4a60 msgr2=0x7fe59c0b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:54.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.467+0000 7fe5a2200640 1 -- 172.21.15.196:0/2600231992 shutdown_connections 2024-04-28T13:32:54.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:54.467+0000 7fe5a2200640 1 -- 172.21.15.196:0/2600231992 wait complete. 2024-04-28T13:32:54.485 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-28T13:32:54.485 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T13:32:54.486 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2024-04-28T13:32:54.486 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-28T13:32:54.486 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-28T13:32:54.486 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-28T13:32:54.486 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 2024-04-28T13:32:54.486 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T13:32:54.486 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-28T13:32:54.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:54.486 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2024-04-28T13:32:54.544 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:32:54.545 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:32:54.545 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:32:54.545 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:54.545 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-28T13:32:54.545 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:54.545 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:54.545 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:54.545 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f0 2024-04-28T13:32:54.545 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:54.545 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:54.545 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f1 2024-04-28T13:32:54.545 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:32:54.545 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:54.545 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T13:32:54.545 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:54.546 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T13:32:54.546 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42545sec preferred_lft 42545sec 2024-04-28T13:32:54.546 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ec4:7aff:fe8f:4a23/64 scope link 2024-04-28T13:32:54.546 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:54.546 INFO:teuthology.orchestra.run.smithi203.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T13:32:54.546 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 1e:04:67:22:4e:82 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:32:54.546 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-28T13:32:54.546 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:54.546 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::4e3:c8ff:fea0:8dce/64 scope link 2024-04-28T13:32:54.546 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:54.546 INFO:teuthology.orchestra.run.smithi203.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:32:54.546 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 1e:04:67:22:4e:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T13:32:54.546 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::1c04:67ff:fe22:4e82/64 scope link 2024-04-28T13:32:54.546 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:54.547 INFO:teuthology.orchestra.run.smithi203.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:32:54.547 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ae:7c:61:1f:29:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-28T13:32:54.547 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ac7c:61ff:fe1f:297b/64 scope link 2024-04-28T13:32:54.547 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:32:54.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:54.548 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T13:32:54.626 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-28T13:32:54.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:54.630 DEBUG:teuthology.orchestra.run.smithi203:> route 2024-04-28T13:32:54.685 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2024-04-28T13:32:54.685 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T13:32:54.685 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T13:32:54.685 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T13:32:54.685 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T13:32:54.685 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:32:54.686 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-28T13:32:54.686 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T13:32:54.686 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T13:32:54.686 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T13:32:54.686 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-28T13:32:54.787 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:32:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:32:54.882 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:32:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:32:54.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:54.892 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-28T13:32:54.936 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-28T13:32:54.937 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-28T13:32:54.938 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-28T13:32:54.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:54.938 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.1 up 2024-04-28T13:32:55.069 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:32:55.069 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-28T13:32:55.123 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-28T13:32:55.123 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:32:55.123 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-28T13:32:55.173 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 2024-04-28T13:32:55.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:55.174 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-28T13:32:55.298 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-28T13:32:55.298 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-28T13:32:55.298 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:32:55.319 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-28T13:32:55.319 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-28T13:32:55.319 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:32:55.319 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:32:55.319 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:32:55.345 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-28T13:32:55.345 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-28T13:32:55.345 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:32:55.345 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:32:55.345 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:32:55.345 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-28T13:32:55.346 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790:/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 2024-04-28T13:32:55.346 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=270aea32-2e16-4a97-85f7-aeae39bc8c76" will pass to kernel 2024-04-28T13:32:55.348 INFO:teuthology.orchestra.run.smithi203.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-28T13:32:55.348 INFO:teuthology.orchestra.run.smithi203.stdout: You just mounted a file system that supports labels which does not 2024-04-28T13:32:55.348 INFO:teuthology.orchestra.run.smithi203.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-28T13:32:55.348 INFO:teuthology.orchestra.run.smithi203.stdout: applications will generate AVC messages and not be allowed access to 2024-04-28T13:32:55.348 INFO:teuthology.orchestra.run.smithi203.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-28T13:32:55.352 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-28T13:32:55.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:55.353 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-28T13:32:55.422 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:32:55.519 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:32:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:32:55.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:32:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:32:55.532 INFO:teuthology.orchestra.run.smithi203.stdout:client4676 2024-04-28T13:32:55.533 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'touch file') 2024-04-28T13:32:55.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:32:55.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:32:55.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:32:55.679 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quiesce.block -v 1 file) 2024-04-28T13:32:55.749 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 quiesce path /volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 --wait 2024-04-28T13:32:55.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.957+0000 7f3181400640 1 Processor -- start 2024-04-28T13:32:55.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.958+0000 7f3181400640 1 -- start start 2024-04-28T13:32:55.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.959+0000 7f3181400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f317c06e330 0x7f317c0d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:55.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.959+0000 7f3181400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f317c0d16d0 0x7f317c0cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:55.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.959+0000 7f3181400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f317c0d1ae0 con 0x7f317c1454c0 2024-04-28T13:32:55.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.959+0000 7f3181400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f317c0721a0 con 0x7f317c0d16d0 2024-04-28T13:32:55.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.959+0000 7f3181400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f317c0ceca0 con 0x7f317c06e330 2024-04-28T13:32:55.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.959+0000 7f317a800640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f317c1454c0 0x7f317c0d05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:45456/0 (socket says 172.21.15.196:45456) 2024-04-28T13:32:55.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.959+0000 7f317a800640 1 -- 172.21.15.196:0/2798953471 learned_addr learned my addr 172.21.15.196:0/2798953471 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:55.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.959+0000 7f317b200640 1 --2- 172.21.15.196:0/2798953471 >> v2:172.21.15.196:3300/0 conn(0x7f317c0d16d0 0x7f317c0cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:55.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.960+0000 7f3171e00640 1 --2- 172.21.15.196:0/2798953471 >> v2:172.21.15.203:3300/0 conn(0x7f317c06e330 0x7f317c0d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:55.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.960+0000 7f3179e00640 1 -- 172.21.15.196:0/2798953471 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1221888145 0 0) 0x7f316c001110 con 0x7f317c1454c0 2024-04-28T13:32:55.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.960+0000 7f3179e00640 1 -- 172.21.15.196:0/2798953471 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3160003520 con 0x7f317c1454c0 2024-04-28T13:32:55.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.960+0000 7f317b200640 1 -- 172.21.15.196:0/2798953471 >> v2:172.21.15.203:3300/0 conn(0x7f317c06e330 msgr2=0x7f317c0d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:55.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.960+0000 7f317b200640 1 --2- 172.21.15.196:0/2798953471 >> v2:172.21.15.203:3300/0 conn(0x7f317c06e330 0x7f317c0d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.960+0000 7f317b200640 1 -- 172.21.15.196:0/2798953471 >> v1:172.21.15.196:6789/0 conn(0x7f317c1454c0 legacy=0x7f317c0d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:55.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.960+0000 7f317b200640 1 -- 172.21.15.196:0/2798953471 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c06e830 con 0x7f317c0d16d0 2024-04-28T13:32:55.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.961+0000 7f317b200640 1 --2- 172.21.15.196:0/2798953471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f317c0d16d0 0x7f317c0cf130 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f317c0d1e30 tx=0x7f317402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294af4eecb018618 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:55.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.961+0000 7f3179e00640 1 -- 172.21.15.196:0/2798953471 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174007740 con 0x7f317c0d16d0 2024-04-28T13:32:55.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.961+0000 7f3179e00640 1 -- 172.21.15.196:0/2798953471 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31740078e0 con 0x7f317c0d16d0 2024-04-28T13:32:55.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.961+0000 7f3179e00640 1 -- 172.21.15.196:0/2798953471 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174007cb0 con 0x7f317c0d16d0 2024-04-28T13:32:55.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.962+0000 7f3181400640 1 -- 172.21.15.196:0/2798953471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f317c0d16d0 msgr2=0x7f317c0cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:55.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.962+0000 7f3181400640 1 --2- 172.21.15.196:0/2798953471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f317c0d16d0 0x7f317c0cf130 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f317c0d1e30 tx=0x7f317402a680 comp rx=0 tx=0).stop 2024-04-28T13:32:55.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.962+0000 7f3181400640 1 -- 172.21.15.196:0/2798953471 shutdown_connections 2024-04-28T13:32:55.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.962+0000 7f3181400640 1 --2- 172.21.15.196:0/2798953471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f317c0d16d0 0x7f317c0cf130 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.962+0000 7f3181400640 1 --2- 172.21.15.196:0/2798953471 >> v2:172.21.15.203:3300/0 conn(0x7f317c06e330 0x7f317c0d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.962+0000 7f3181400640 1 -- 172.21.15.196:0/2798953471 >> 172.21.15.196:0/2798953471 conn(0x7f317c06c690 msgr2=0x7f317c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:55.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.962+0000 7f3181400640 1 -- 172.21.15.196:0/2798953471 shutdown_connections 2024-04-28T13:32:55.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.962+0000 7f3181400640 1 -- 172.21.15.196:0/2798953471 wait complete. 2024-04-28T13:32:55.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.964+0000 7f3181400640 1 Processor -- start 2024-04-28T13:32:55.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.964+0000 7f3181400640 1 -- start start 2024-04-28T13:32:55.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.965+0000 7f3181400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f317c06e330 0x7f317c14db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:55.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.965+0000 7f3181400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f317c0d16d0 0x7f317c0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:55.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.965+0000 7f3181400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317c1454c0 0x7f317c0c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:55.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.965+0000 7f3181400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f317c06f0d0 con 0x7f317c06e330 2024-04-28T13:32:55.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.965+0000 7f3181400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f317c06edf0 con 0x7f317c1454c0 2024-04-28T13:32:55.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.965+0000 7f3181400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f317c15d8a0 con 0x7f317c0d16d0 2024-04-28T13:32:55.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.965+0000 7f317a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f317c06e330 0x7f317c14db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:55.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.965+0000 7f317a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f317c06e330 0x7f317c14db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41804/0 (socket says 172.21.15.196:41804) 2024-04-28T13:32:55.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.965+0000 7f317a800640 1 -- 172.21.15.196:0/1568156248 learned_addr learned my addr 172.21.15.196:0/1568156248 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:55.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.966+0000 7f317b200640 1 --2- 172.21.15.196:0/1568156248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317c1454c0 0x7f317c0c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:55.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.966+0000 7f317a800640 1 -- 172.21.15.196:0/1568156248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f317c0d16d0 msgr2=0x7f317c0cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:55.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.966+0000 7f317a800640 1 --2- 172.21.15.196:0/1568156248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f317c0d16d0 0x7f317c0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.966+0000 7f317a800640 1 -- 172.21.15.196:0/1568156248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317c1454c0 msgr2=0x7f317c0c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:55.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.966+0000 7f317a800640 1 --2- 172.21.15.196:0/1568156248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317c1454c0 0x7f317c0c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.966+0000 7f317a800640 1 -- 172.21.15.196:0/1568156248 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c14e040 con 0x7f317c06e330 2024-04-28T13:32:55.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.966+0000 7f317a800640 1 --2- 172.21.15.196:0/1568156248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f317c06e330 0x7f317c14db00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f316c00c760 tx=0x7f316c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:55.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.966+0000 7f3173400640 1 -- 172.21.15.196:0/1568156248 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c017070 con 0x7f317c06e330 2024-04-28T13:32:55.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.966+0000 7f3173400640 1 -- 172.21.15.196:0/1568156248 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c005ad0 con 0x7f317c06e330 2024-04-28T13:32:55.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.966+0000 7f3173400640 1 -- 172.21.15.196:0/1568156248 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c007e20 con 0x7f317c06e330 2024-04-28T13:32:55.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.966+0000 7f3181400640 1 -- 172.21.15.196:0/1568156248 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317c13db80 con 0x7f317c06e330 2024-04-28T13:32:55.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.967+0000 7f3181400640 1 -- 172.21.15.196:0/1568156248 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317c13e140 con 0x7f317c06e330 2024-04-28T13:32:55.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.969+0000 7f3173400640 1 -- 172.21.15.196:0/1568156248 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f316c01a070 con 0x7f317c06e330 2024-04-28T13:32:55.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.970+0000 7f3173400640 1 --2- 172.21.15.196:0/1568156248 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f315806c980 0x7f315806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:55.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.970+0000 7f3173400640 1 -- 172.21.15.196:0/1568156248 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f316c08af20 con 0x7f317c06e330 2024-04-28T13:32:55.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.970+0000 7f3171e00640 1 --2- 172.21.15.196:0/1568156248 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f315806c980 0x7f315806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:55.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.971+0000 7f31841f8740 1 Processor -- start 2024-04-28T13:32:55.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.971+0000 7f31841f8740 1 -- start start 2024-04-28T13:32:55.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.972+0000 7f3171e00640 1 --2- 172.21.15.196:0/1568156248 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f315806c980 0x7f315806ee40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3164006fd0 tx=0x7f3164008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:55.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.972+0000 7f31841f8740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86fbaaa0 0x55af86f90cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:55.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.972+0000 7f31841f8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fb8230 0x55af86fbae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:55.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.972+0000 7f31841f8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86f917c0 0x55af86fbb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:55.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.972+0000 7f31841f8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af86ef50b0 con 0x55af86f917c0 2024-04-28T13:32:55.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.972+0000 7f31841f8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55af86e91550 con 0x55af86fb8230 2024-04-28T13:32:55.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.972+0000 7f31841f8740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55af86ef1070 con 0x55af86fbaaa0 2024-04-28T13:32:55.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.972+0000 7f317b200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86f917c0 0x55af86fbb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:55.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.972+0000 7f317b200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86f917c0 0x55af86fbb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41816/0 (socket says 172.21.15.196:41816) 2024-04-28T13:32:55.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.972+0000 7f317b200640 1 -- 172.21.15.196:0/191184362 learned_addr learned my addr 172.21.15.196:0/191184362 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:55.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.973+0000 7f3171e00640 1 --2- 172.21.15.196:0/191184362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fb8230 0x55af86fbae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:55.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.973+0000 7f317a800640 1 --2- 172.21.15.196:0/191184362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86fbaaa0 0x55af86f90cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:55.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.973+0000 7f317b200640 1 -- 172.21.15.196:0/191184362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86fbaaa0 msgr2=0x55af86f90cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:55.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.973+0000 7f317b200640 1 --2- 172.21.15.196:0/191184362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86fbaaa0 0x55af86f90cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.973+0000 7f317b200640 1 -- 172.21.15.196:0/191184362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fb8230 msgr2=0x55af86fbae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:55.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.973+0000 7f317b200640 1 --2- 172.21.15.196:0/191184362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fb8230 0x55af86fbae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.973+0000 7f317b200640 1 -- 172.21.15.196:0/191184362 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af86e832c0 con 0x55af86f917c0 2024-04-28T13:32:55.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.973+0000 7f317b200640 1 --2- 172.21.15.196:0/191184362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86f917c0 0x55af86fbb3c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f317402ac70 tx=0x7f317402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcae9c36648fc0d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:55.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.973+0000 7f316b000640 1 -- 172.21.15.196:0/191184362 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174007740 con 0x55af86f917c0 2024-04-28T13:32:55.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.974+0000 7f316b000640 1 -- 172.21.15.196:0/191184362 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31740078e0 con 0x55af86f917c0 2024-04-28T13:32:55.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.974+0000 7f316b000640 1 -- 172.21.15.196:0/191184362 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174007cb0 con 0x55af86f917c0 2024-04-28T13:32:55.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.974+0000 7f31841f8740 1 -- 172.21.15.196:0/191184362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86f917c0 msgr2=0x55af86fbb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:55.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.974+0000 7f31841f8740 1 --2- 172.21.15.196:0/191184362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86f917c0 0x55af86fbb3c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f317402ac70 tx=0x7f317402e000 comp rx=0 tx=0).stop 2024-04-28T13:32:55.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.974+0000 7f31841f8740 1 -- 172.21.15.196:0/191184362 shutdown_connections 2024-04-28T13:32:55.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.974+0000 7f31841f8740 1 --2- 172.21.15.196:0/191184362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86f917c0 0x55af86fbb3c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.974+0000 7f31841f8740 1 --2- 172.21.15.196:0/191184362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fb8230 0x55af86fbae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.974+0000 7f31841f8740 1 --2- 172.21.15.196:0/191184362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86fbaaa0 0x55af86f90cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.974+0000 7f31841f8740 1 -- 172.21.15.196:0/191184362 >> 172.21.15.196:0/191184362 conn(0x55af86f57a70 msgr2=0x55af86fd5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:55.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.974+0000 7f31841f8740 1 -- 172.21.15.196:0/191184362 shutdown_connections 2024-04-28T13:32:55.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.974+0000 7f31841f8740 1 -- 172.21.15.196:0/191184362 wait complete. 2024-04-28T13:32:55.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.975+0000 7f31841f8740 1 -- start start 2024-04-28T13:32:55.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.975+0000 7f31841f8740 1 Processor -- start 2024-04-28T13:32:55.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.976+0000 7f31841f8740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86f917c0 0x55af86fb2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:55.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.976+0000 7f31841f8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86fb8230 0x55af86fe9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:55.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.976+0000 7f31841f8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fbaaa0 0x55af86fed6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:55.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.976+0000 7f31841f8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af86f8b000 con 0x55af86fb8230 2024-04-28T13:32:55.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.976+0000 7f31841f8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55af86e7c590 con 0x55af86fbaaa0 2024-04-28T13:32:55.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.976+0000 7f31841f8740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55af86fd6a20 con 0x55af86f917c0 2024-04-28T13:32:55.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.976+0000 7f31841f8740 10 client.0 _ll_register_callbacks cb 0x55af86eebf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T13:32:55.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.977+0000 7f3171e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86fb8230 0x55af86fe9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:55.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.977+0000 7f3171e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86fb8230 0x55af86fe9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41832/0 (socket says 172.21.15.196:41832) 2024-04-28T13:32:55.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.977+0000 7f3171e00640 1 -- 172.21.15.196:0/2539495362 learned_addr learned my addr 172.21.15.196:0/2539495362 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:55.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.977+0000 7f317a800640 1 --2- 172.21.15.196:0/2539495362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86f917c0 0x55af86fb2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:55.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.977+0000 7f317b200640 1 --2- 172.21.15.196:0/2539495362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fbaaa0 0x55af86fed6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:55.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.977+0000 7f3171e00640 1 -- 172.21.15.196:0/2539495362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86f917c0 msgr2=0x55af86fb2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:55.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.977+0000 7f3171e00640 1 --2- 172.21.15.196:0/2539495362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86f917c0 0x55af86fb2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.977+0000 7f3171e00640 1 -- 172.21.15.196:0/2539495362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fbaaa0 msgr2=0x55af86fed6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:55.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.977+0000 7f3171e00640 1 --2- 172.21.15.196:0/2539495362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fbaaa0 0x55af86fed6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.977+0000 7f3171e00640 1 -- 172.21.15.196:0/2539495362 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af86cd3030 con 0x55af86fb8230 2024-04-28T13:32:55.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.977+0000 7f317b200640 1 --2- 172.21.15.196:0/2539495362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fbaaa0 0x55af86fed6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:55.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.977+0000 7f3171e00640 1 --2- 172.21.15.196:0/2539495362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86fb8230 0x55af86fe9bd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f316400de70 tx=0x7f31640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9b77d53343f7bf server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:55.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.977+0000 7f315f400640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-04-28T13:32:55.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.978+0000 7f315f400640 1 -- 172.21.15.196:0/2539495362 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164020450 con 0x55af86fb8230 2024-04-28T13:32:55.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.978+0000 7f31841f8740 1 -- 172.21.15.196:0/2539495362 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31740078e0 con 0x55af86fb8230 2024-04-28T13:32:55.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.978+0000 7f315f400640 1 -- 172.21.15.196:0/2539495362 <== mon.0 v2:172.21.15.196:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f31640205f0 con 0x55af86fb8230 2024-04-28T13:32:55.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.978+0000 7f315f400640 1 -- 172.21.15.196:0/2539495362 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f3164005990 con 0x55af86fb8230 2024-04-28T13:32:55.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.978+0000 7f31841f8740 10 client.5379 fetch_fsmap learned FSMap version 16 2024-04-28T13:32:55.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.978+0000 7f31841f8740 1 -- 172.21.15.196:0/2539495362 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55af86c06c20 con 0x55af86fb8230 2024-04-28T13:32:55.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.978+0000 7f315f400640 1 -- 172.21.15.196:0/2539495362 <== mon.0 v2:172.21.15.196:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31740078e0 con 0x55af86fb8230 2024-04-28T13:32:55.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.978+0000 7f315f400640 1 -- 172.21.15.196:0/2539495362 <== mon.0 v2:172.21.15.196:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f3164030080 con 0x55af86fb8230 2024-04-28T13:32:55.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.978+0000 7f31841f8740 10 client.5379 fetch_fsmap finished waiting for FSMap version 16 2024-04-28T13:32:55.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.978+0000 7f31841f8740 10 client.5379 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-04-28T13:32:55.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.978+0000 7f31841f8740 20 client.5379 populate_metadata read hostname 'smithi196' 2024-04-28T13:32:55.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.978+0000 7f31841f8740 1 --2- 172.21.15.196:0/2539495362 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55af86fd7310 0x55af86fd7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:55.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.979+0000 7f31841f8740 4 client.5379 mds_command: new command op to 4735 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T13:32:55.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.979+0000 7f31841f8740 1 -- 172.21.15.196:0/2539495362 --> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af86f02cc0 con 0x55af86fd7310 2024-04-28T13:32:55.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.979+0000 7f317a800640 1 --2- 172.21.15.196:0/2539495362 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55af86fd7310 0x55af86fd7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:55.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.980+0000 7f317a800640 1 --2- 172.21.15.196:0/2539495362 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55af86fd7310 0x55af86fd7790 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98aadbf88ff62779 server_cookie=487ba61a39bea276 in_seq=0 out_seq=0 2024-04-28T13:32:55.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.980+0000 7f315f400640 10 client.5379 ms_handle_connect on v2:172.21.15.203:6835/459640695 2024-04-28T13:32:55.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.982+0000 7f315f400640 1 -- 172.21.15.196:0/2539495362 <== mds.0 v2:172.21.15.203:6835/459640695 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af86f02cc0 con 0x55af86fd7310 2024-04-28T13:32:55.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.982+0000 7f315f400640 10 client.5379 handle_command_reply: tid=0 2024-04-28T13:32:55.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.982+0000 7f31841f8740 1 client.5379 shutdown 2024-04-28T13:32:55.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.982+0000 7f31841f8740 1 -- 172.21.15.196:0/2539495362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86fb8230 msgr2=0x55af86fe9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:55.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.983+0000 7f31841f8740 1 --2- 172.21.15.196:0/2539495362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86fb8230 0x55af86fe9bd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f316400de70 tx=0x7f31640046c0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.983+0000 7f31841f8740 1 -- 172.21.15.196:0/2539495362 shutdown_connections 2024-04-28T13:32:55.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.983+0000 7f31841f8740 1 --2- 172.21.15.196:0/2539495362 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55af86fd7310 0x55af86fd7790 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.983+0000 7f31841f8740 1 --2- 172.21.15.196:0/2539495362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fbaaa0 0x55af86fed6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.983+0000 7f31841f8740 1 --2- 172.21.15.196:0/2539495362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86fb8230 0x55af86fe9bd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.983+0000 7f31841f8740 1 --2- 172.21.15.196:0/2539495362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86f917c0 0x55af86fb2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:55.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.983+0000 7f31841f8740 1 -- 172.21.15.196:0/2539495362 >> 172.21.15.196:0/2539495362 conn(0x55af86f57a70 msgr2=0x55af86fd5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:55.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.983+0000 7f315f400640 0 client.5379 ms_handle_reset on v2:172.21.15.203:6835/459640695 2024-04-28T13:32:55.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.983+0000 7f31841f8740 1 -- 172.21.15.196:0/2539495362 shutdown_connections 2024-04-28T13:32:55.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.983+0000 7f31841f8740 1 -- 172.21.15.196:0/2539495362 wait complete. 2024-04-28T13:32:55.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:55.983+0000 7f31841f8740 20 client.5379 trim_cache size 0 max 16384 2024-04-28T13:32:56.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.001+0000 7f31841f8740 1 Processor -- start 2024-04-28T13:32:56.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.001+0000 7f31841f8740 1 -- start start 2024-04-28T13:32:56.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.002+0000 7f31841f8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86f917c0 0x55af86fd6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.002+0000 7f31841f8740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86fb8230 0x55af86f89ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.002+0000 7f31841f8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fbaaa0 0x55af86fd76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.002+0000 7f31841f8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f317c1fd260 con 0x55af86f917c0 2024-04-28T13:32:56.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.002+0000 7f31841f8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f317c13fd50 con 0x55af86fbaaa0 2024-04-28T13:32:56.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.002+0000 7f31841f8740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f317c073f20 con 0x55af86fb8230 2024-04-28T13:32:56.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.003+0000 7f317a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86f917c0 0x55af86fd6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.003+0000 7f317a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86f917c0 0x55af86fd6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41840/0 (socket says 172.21.15.196:41840) 2024-04-28T13:32:56.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.003+0000 7f317a800640 1 -- 172.21.15.196:0/790751037 learned_addr learned my addr 172.21.15.196:0/790751037 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:56.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.003+0000 7f317a800640 1 -- 172.21.15.196:0/790751037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86fb8230 msgr2=0x55af86f89ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:56.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.003+0000 7f317a800640 1 --2- 172.21.15.196:0/790751037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86fb8230 0x55af86f89ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.003+0000 7f317a800640 1 -- 172.21.15.196:0/790751037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fbaaa0 msgr2=0x55af86fd76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:56.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.003+0000 7f317a800640 1 --2- 172.21.15.196:0/790751037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fbaaa0 0x55af86fd76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.003+0000 7f317a800640 1 -- 172.21.15.196:0/790751037 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af86bc6920 con 0x55af86f917c0 2024-04-28T13:32:56.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.003+0000 7f317a800640 1 --2- 172.21.15.196:0/790751037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86f917c0 0x55af86fd6000 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f316c0550c0 tx=0x7f316c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2601f229fc082d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:56.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.004+0000 7f316a600640 1 -- 172.21.15.196:0/790751037 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c087070 con 0x55af86f917c0 2024-04-28T13:32:56.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.004+0000 7f316a600640 1 -- 172.21.15.196:0/790751037 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c071100 con 0x55af86f917c0 2024-04-28T13:32:56.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.004+0000 7f316a600640 1 -- 172.21.15.196:0/790751037 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c0613b0 con 0x55af86f917c0 2024-04-28T13:32:56.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.004+0000 7f31841f8740 1 -- 172.21.15.196:0/790751037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86f917c0 msgr2=0x55af86fd6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:56.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.004+0000 7f31841f8740 1 --2- 172.21.15.196:0/790751037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86f917c0 0x55af86fd6000 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f316c0550c0 tx=0x7f316c000bc0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.004+0000 7f31841f8740 1 -- 172.21.15.196:0/790751037 shutdown_connections 2024-04-28T13:32:56.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.004+0000 7f31841f8740 1 --2- 172.21.15.196:0/790751037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fbaaa0 0x55af86fd76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.004+0000 7f31841f8740 1 --2- 172.21.15.196:0/790751037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86fb8230 0x55af86f89ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.004+0000 7f31841f8740 1 --2- 172.21.15.196:0/790751037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86f917c0 0x55af86fd6000 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.004+0000 7f31841f8740 1 -- 172.21.15.196:0/790751037 >> 172.21.15.196:0/790751037 conn(0x55af86f57a70 msgr2=0x55af86f8db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:56.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.004+0000 7f31841f8740 1 -- 172.21.15.196:0/790751037 shutdown_connections 2024-04-28T13:32:56.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.004+0000 7f31841f8740 1 -- 172.21.15.196:0/790751037 wait complete. 2024-04-28T13:32:56.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.005+0000 7f31841f8740 1 -- start start 2024-04-28T13:32:56.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.005+0000 7f31841f8740 1 Processor -- start 2024-04-28T13:32:56.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.006+0000 7f31841f8740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86f917c0 0x55af86f89ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.006+0000 7f31841f8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fb8230 0x55af86fd4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.006+0000 7f31841f8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86fbaaa0 0x55af86fd7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.006+0000 7f31841f8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af86f8a410 con 0x55af86fbaaa0 2024-04-28T13:32:56.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.006+0000 7f31841f8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f317c13f700 con 0x55af86fb8230 2024-04-28T13:32:56.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.006+0000 7f31841f8740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55af86f8a620 con 0x55af86f917c0 2024-04-28T13:32:56.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.006+0000 7f31841f8740 10 client.0 _ll_register_callbacks cb 0x7f316400b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T13:32:56.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.006+0000 7f317b200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86fbaaa0 0x55af86fd7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.006+0000 7f317b200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86fbaaa0 0x55af86fd7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41856/0 (socket says 172.21.15.196:41856) 2024-04-28T13:32:56.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.006+0000 7f317b200640 1 -- 172.21.15.196:0/3338928425 learned_addr learned my addr 172.21.15.196:0/3338928425 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:56.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.007+0000 7f3171e00640 1 --2- 172.21.15.196:0/3338928425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fb8230 0x55af86fd4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.007+0000 7f317b200640 1 -- 172.21.15.196:0/3338928425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86f917c0 msgr2=0x55af86f89ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:56.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.007+0000 7f317b200640 1 --2- 172.21.15.196:0/3338928425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86f917c0 0x55af86f89ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.007+0000 7f317b200640 1 -- 172.21.15.196:0/3338928425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fb8230 msgr2=0x55af86fd4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:56.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.007+0000 7f317b200640 1 --2- 172.21.15.196:0/3338928425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fb8230 0x55af86fd4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.007+0000 7f317b200640 1 -- 172.21.15.196:0/3338928425 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af86e9b420 con 0x55af86fbaaa0 2024-04-28T13:32:56.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.007+0000 7f317b200640 1 --2- 172.21.15.196:0/3338928425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86fbaaa0 0x55af86fd7af0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f317402a6c0 tx=0x7f3174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82c93beec802be8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:56.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.007+0000 7f315f400640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-04-28T13:32:56.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.007+0000 7f315f400640 1 -- 172.21.15.196:0/3338928425 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174003ce0 con 0x55af86fbaaa0 2024-04-28T13:32:56.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.008+0000 7f315f400640 1 -- 172.21.15.196:0/3338928425 <== mon.0 v2:172.21.15.196:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3174004040 con 0x55af86fbaaa0 2024-04-28T13:32:56.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.008+0000 7f31841f8740 1 -- 172.21.15.196:0/3338928425 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f316c071100 con 0x55af86fbaaa0 2024-04-28T13:32:56.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.008+0000 7f315f400640 1 -- 172.21.15.196:0/3338928425 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f3174057050 con 0x55af86fbaaa0 2024-04-28T13:32:56.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.008+0000 7f31841f8740 10 client.5385 fetch_fsmap learned FSMap version 16 2024-04-28T13:32:56.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.008+0000 7f31841f8740 1 -- 172.21.15.196:0/3338928425 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55af86e9af70 con 0x55af86fbaaa0 2024-04-28T13:32:56.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.008+0000 7f315f400640 1 -- 172.21.15.196:0/3338928425 <== mon.0 v2:172.21.15.196:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c071100 con 0x55af86fbaaa0 2024-04-28T13:32:56.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.008+0000 7f315f400640 1 -- 172.21.15.196:0/3338928425 <== mon.0 v2:172.21.15.196:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f317405a020 con 0x55af86fbaaa0 2024-04-28T13:32:56.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.008+0000 7f31841f8740 10 client.5385 fetch_fsmap finished waiting for FSMap version 16 2024-04-28T13:32:56.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.008+0000 7f31841f8740 10 client.5385 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-04-28T13:32:56.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.008+0000 7f31841f8740 20 client.5385 populate_metadata read hostname 'smithi196' 2024-04-28T13:32:56.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.008+0000 7f31841f8740 1 --2- 172.21.15.196:0/3338928425 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55af86fbe410 0x55af86fd1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.008+0000 7f31841f8740 4 client.5385 mds_command: new command op to 4735 tid=0 multi_id=0 [{"prefix": "quiesce path", "path": "/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1", "wait": true}] 2024-04-28T13:32:56.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.008+0000 7f31841f8740 1 -- 172.21.15.196:0/3338928425 --> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] -- command(tid 0: {"prefix": "quiesce path", "path": "/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1", "wait": true}) -- 0x55af86efcb30 con 0x55af86fbe410 2024-04-28T13:32:56.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.009+0000 7f317a800640 1 --2- 172.21.15.196:0/3338928425 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55af86fbe410 0x55af86fd1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.010+0000 7f317a800640 1 --2- 172.21.15.196:0/3338928425 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55af86fbe410 0x55af86fd1e50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7303af7b21de68c6 server_cookie=b9335cf4516cc756 in_seq=0 out_seq=0 2024-04-28T13:32:56.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.010+0000 7f315f400640 10 client.5385 ms_handle_connect on v2:172.21.15.203:6835/459640695 2024-04-28T13:32:56.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.022+0000 7f315f400640 1 -- 172.21.15.196:0/3338928425 <== mds.0 v2:172.21.15.203:6835/459640695 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2332 (crc 0 0 0) 0x55af86efcb30 con 0x55af86fbe410 2024-04-28T13:32:56.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.022+0000 7f315f400640 10 client.5385 handle_command_reply: tid=0 2024-04-28T13:32:56.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.022+0000 7f31841f8740 1 client.5385 shutdown 2024-04-28T13:32:56.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.022+0000 7f31841f8740 1 -- 172.21.15.196:0/3338928425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86fbaaa0 msgr2=0x55af86fd7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:56.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.022+0000 7f31841f8740 1 --2- 172.21.15.196:0/3338928425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86fbaaa0 0x55af86fd7af0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f317402a6c0 tx=0x7f3174004a40 comp rx=0 tx=0).stop 2024-04-28T13:32:56.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.022+0000 7f31841f8740 1 -- 172.21.15.196:0/3338928425 shutdown_connections 2024-04-28T13:32:56.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.022+0000 7f31841f8740 1 --2- 172.21.15.196:0/3338928425 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55af86fbe410 0x55af86fd1e50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.022+0000 7f31841f8740 1 --2- 172.21.15.196:0/3338928425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af86fbaaa0 0x55af86fd7af0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.022+0000 7f31841f8740 1 --2- 172.21.15.196:0/3338928425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af86fb8230 0x55af86fd4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.022+0000 7f31841f8740 1 --2- 172.21.15.196:0/3338928425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55af86f917c0 0x55af86f89ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.022+0000 7f315f400640 0 client.5385 ms_handle_reset on v2:172.21.15.203:6835/459640695 2024-04-28T13:32:56.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.022+0000 7f31841f8740 1 -- 172.21.15.196:0/3338928425 >> 172.21.15.196:0/3338928425 conn(0x55af86f57a70 msgr2=0x55af86fe8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:56.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.022+0000 7f31841f8740 1 -- 172.21.15.196:0/3338928425 shutdown_connections 2024-04-28T13:32:56.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.022+0000 7f31841f8740 1 -- 172.21.15.196:0/3338928425 wait complete. 2024-04-28T13:32:56.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.023+0000 7f31841f8740 20 client.5385 trim_cache size 0 max 16384 2024-04-28T13:32:56.024 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:32:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: "op": { 2024-04-28T13:32:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: "result": 0, 2024-04-28T13:32:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: "flag_point": "quiesce complete", 2024-04-28T13:32:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: "reqid": { 2024-04-28T13:32:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": { 2024-04-28T13:32:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "mds", 2024-04-28T13:32:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 0 2024-04-28T13:32:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.024 INFO:teuthology.orchestra.run.smithi196.stdout: "tid": 1 2024-04-28T13:32:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "op_type": "internal_op", 2024-04-28T13:32:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "internal_op": 5384, 2024-04-28T13:32:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "op_name": "quiesce_path", 2024-04-28T13:32:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "events": [ 2024-04-28T13:32:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-28T13:32:56.011057+0000", 2024-04-28T13:32:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "initiated" 2024-04-28T13:32:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-28T13:32:56.011057+0000", 2024-04-28T13:32:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "header_read" 2024-04-28T13:32:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:56.025 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-28T13:32:56.011057+0000", 2024-04-28T13:32:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "throttled" 2024-04-28T13:32:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-28T13:32:56.011057+0000", 2024-04-28T13:32:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "all_read" 2024-04-28T13:32:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-28T13:32:56.011057+0000", 2024-04-28T13:32:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "dispatched" 2024-04-28T13:32:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-28T13:32:56.011241+0000", 2024-04-28T13:32:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "acquired locks" 2024-04-28T13:32:56.026 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-28T13:32:56.011294+0000", 2024-04-28T13:32:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "acquired locks" 2024-04-28T13:32:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-28T13:32:56.011342+0000", 2024-04-28T13:32:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "acquired locks" 2024-04-28T13:32:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-28T13:32:56.011393+0000", 2024-04-28T13:32:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "acquired locks" 2024-04-28T13:32:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-28T13:32:56.021249+0000", 2024-04-28T13:32:56.027 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "acquired locks" 2024-04-28T13:32:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-28T13:32:56.021303+0000", 2024-04-28T13:32:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "acquired locks" 2024-04-28T13:32:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-28T13:32:56.021356+0000", 2024-04-28T13:32:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "acquired locks" 2024-04-28T13:32:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-28T13:32:56.021410+0000", 2024-04-28T13:32:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "acquired locks" 2024-04-28T13:32:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:32:56.028 INFO:teuthology.orchestra.run.smithi196.stdout: "time": "2024-04-28T13:32:56.021486+0000", 2024-04-28T13:32:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "event": "quiesce complete" 2024-04-28T13:32:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: ], 2024-04-28T13:32:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "locks": null 2024-04-28T13:32:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:32:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "state": { 2024-04-28T13:32:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 2, 2024-04-28T13:32:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_quiesced": 1, 2024-04-28T13:32:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_blocked": 1, 2024-04-28T13:32:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_dropped": 0, 2024-04-28T13:32:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: "failed": [] 2024-04-28T13:32:56.029 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:32:56.029 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:32:56.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.023+0000 7f3181400640 1 -- 172.21.15.196:0/1568156248 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f315806c980 msgr2=0x7f315806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:56.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.023+0000 7f3181400640 1 --2- 172.21.15.196:0/1568156248 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f315806c980 0x7f315806ee40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3164006fd0 tx=0x7f3164008040 comp rx=0 tx=0).stop 2024-04-28T13:32:56.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.023+0000 7f3181400640 1 -- 172.21.15.196:0/1568156248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f317c06e330 msgr2=0x7f317c14db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:56.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.023+0000 7f3181400640 1 --2- 172.21.15.196:0/1568156248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f317c06e330 0x7f317c14db00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f316c00c760 tx=0x7f316c00cc20 comp rx=0 tx=0).stop 2024-04-28T13:32:56.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.023+0000 7f3181400640 1 -- 172.21.15.196:0/1568156248 shutdown_connections 2024-04-28T13:32:56.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.023+0000 7f3181400640 1 --2- 172.21.15.196:0/1568156248 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f315806c980 0x7f315806ee40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.023+0000 7f3181400640 1 --2- 172.21.15.196:0/1568156248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317c1454c0 0x7f317c0c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.023+0000 7f3181400640 1 --2- 172.21.15.196:0/1568156248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f317c0d16d0 0x7f317c0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.023+0000 7f3181400640 1 --2- 172.21.15.196:0/1568156248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f317c06e330 0x7f317c14db00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.023+0000 7f3181400640 1 -- 172.21.15.196:0/1568156248 >> 172.21.15.196:0/1568156248 conn(0x7f317c06c690 msgr2=0x7f317c144180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:56.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.023+0000 7f3181400640 1 -- 172.21.15.196:0/1568156248 shutdown_connections 2024-04-28T13:32:56.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.024+0000 7f3181400640 1 -- 172.21.15.196:0/1568156248 wait complete. 2024-04-28T13:32:56.041 DEBUG:tasks.cephfs.test_quiesce:{'op': {'result': 0, 'flag_point': 'quiesce complete', 'reqid': {'entity': {'type': 'mds', 'num': 0}, 'tid': 1}, 'op_type': 'internal_op', 'internal_op': 5384, 'op_name': 'quiesce_path', 'events': [{'time': '2024-04-28T13:32:56.011057+0000', 'event': 'initiated'}, {'time': '2024-04-28T13:32:56.011057+0000', 'event': 'header_read'}, {'time': '2024-04-28T13:32:56.011057+0000', 'event': 'throttled'}, {'time': '2024-04-28T13:32:56.011057+0000', 'event': 'all_read'}, {'time': '2024-04-28T13:32:56.011057+0000', 'event': 'dispatched'}, {'time': '2024-04-28T13:32:56.011241+0000', 'event': 'acquired locks'}, {'time': '2024-04-28T13:32:56.011294+0000', 'event': 'acquired locks'}, {'time': '2024-04-28T13:32:56.011342+0000', 'event': 'acquired locks'}, {'time': '2024-04-28T13:32:56.011393+0000', 'event': 'acquired locks'}, {'time': '2024-04-28T13:32:56.021249+0000', 'event': 'acquired locks'}, {'time': '2024-04-28T13:32:56.021303+0000', 'event': 'acquired locks'}, {'time': '2024-04-28T13:32:56.021356+0000', 'event': 'acquired locks'}, {'time': '2024-04-28T13:32:56.021410+0000', 'event': 'acquired locks'}, {'time': '2024-04-28T13:32:56.021486+0000', 'event': 'quiesce complete'}], 'locks': None}, 'state': {'inodes': 2, 'inodes_quiesced': 1, 'inodes_blocked': 1, 'inodes_dropped': 0, 'failed': []}} 2024-04-28T13:32:56.041 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:32:56.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.238+0000 7f8dc1000640 1 Processor -- start 2024-04-28T13:32:56.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.238+0000 7f8dc1000640 1 -- start start 2024-04-28T13:32:56.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.239+0000 7f8dc1000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc1629c0 0x7f8dbc0d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.239+0000 7f8dc1000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8dbc163bf0 0x7f8dbc163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.239+0000 7f8dc1000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8dbc163570 0x7f8dbc0b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.239+0000 7f8dc1000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8dbc162da0 con 0x7f8dbc163570 2024-04-28T13:32:56.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.239+0000 7f8dc1000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f8dbc0b6190 con 0x7f8dbc163bf0 2024-04-28T13:32:56.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.239+0000 7f8dc1000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8dbc0b6470 con 0x7f8dbc1629c0 2024-04-28T13:32:56.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.240+0000 7f8dbbe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8dbc163570 0x7f8dbc0b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.240+0000 7f8dbbe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8dbc163570 0x7f8dbc0b46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41864/0 (socket says 172.21.15.196:41864) 2024-04-28T13:32:56.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.240+0000 7f8dbbe00640 1 -- 172.21.15.196:0/179287795 learned_addr learned my addr 172.21.15.196:0/179287795 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:56.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.240+0000 7f8db2a00640 1 --2- 172.21.15.196:0/179287795 >> v2:172.21.15.203:3300/0 conn(0x7f8dbc163bf0 0x7f8dbc163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.240+0000 7f8dbbe00640 1 -- 172.21.15.196:0/179287795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc1629c0 msgr2=0x7f8dbc0d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:56.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.240+0000 7f8dbbe00640 1 --2- 172.21.15.196:0/179287795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc1629c0 0x7f8dbc0d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.240+0000 7f8dbbe00640 1 -- 172.21.15.196:0/179287795 >> v2:172.21.15.203:3300/0 conn(0x7f8dbc163bf0 msgr2=0x7f8dbc163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:56.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.240+0000 7f8dbbe00640 1 --2- 172.21.15.196:0/179287795 >> v2:172.21.15.203:3300/0 conn(0x7f8dbc163bf0 0x7f8dbc163030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.240+0000 7f8dbbe00640 1 -- 172.21.15.196:0/179287795 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dbc0b6750 con 0x7f8dbc163570 2024-04-28T13:32:56.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.241+0000 7f8dbbe00640 1 --2- 172.21.15.196:0/179287795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dbc163570 0x7f8dbc0b46b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc08f950 tx=0x7f8db402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19895eeaad7c9ec server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:56.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.242+0000 7f8dbaa00640 1 -- 172.21.15.196:0/179287795 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db4037070 con 0x7f8dbc163570 2024-04-28T13:32:56.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.242+0000 7f8dbaa00640 1 -- 172.21.15.196:0/179287795 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db4032440 con 0x7f8dbc163570 2024-04-28T13:32:56.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.242+0000 7f8dbaa00640 1 -- 172.21.15.196:0/179287795 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db4032810 con 0x7f8dbc163570 2024-04-28T13:32:56.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.242+0000 7f8dc1000640 1 -- 172.21.15.196:0/179287795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dbc163570 msgr2=0x7f8dbc0b46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:56.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.242+0000 7f8dc1000640 1 --2- 172.21.15.196:0/179287795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dbc163570 0x7f8dbc0b46b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc08f950 tx=0x7f8db402a1e0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.243+0000 7f8dc1000640 1 -- 172.21.15.196:0/179287795 shutdown_connections 2024-04-28T13:32:56.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.243+0000 7f8dc1000640 1 --2- 172.21.15.196:0/179287795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dbc163570 0x7f8dbc0b46b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.243+0000 7f8dc1000640 1 --2- 172.21.15.196:0/179287795 >> v2:172.21.15.203:3300/0 conn(0x7f8dbc163bf0 0x7f8dbc163030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.243+0000 7f8dc1000640 1 --2- 172.21.15.196:0/179287795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc1629c0 0x7f8dbc0d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.243+0000 7f8dc1000640 1 -- 172.21.15.196:0/179287795 >> 172.21.15.196:0/179287795 conn(0x7f8dbc09e570 msgr2=0x7f8dbc0d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:56.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.243+0000 7f8dc1000640 1 -- 172.21.15.196:0/179287795 shutdown_connections 2024-04-28T13:32:56.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.243+0000 7f8dc1000640 1 -- 172.21.15.196:0/179287795 wait complete. 2024-04-28T13:32:56.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.245+0000 7f8dc1000640 1 Processor -- start 2024-04-28T13:32:56.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.245+0000 7f8dc1000640 1 -- start start 2024-04-28T13:32:56.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.246+0000 7f8dc1000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dbc1629c0 0x7f8dbc0d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.246+0000 7f8dc1000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dbc163570 0x7f8dbc15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.246+0000 7f8dc1000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc163bf0 0x7f8dbc15d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.246+0000 7f8dc1000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc0d3c50 con 0x7f8dbc1629c0 2024-04-28T13:32:56.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.246+0000 7f8dc1000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc0b1e60 con 0x7f8dbc163570 2024-04-28T13:32:56.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.246+0000 7f8dc1000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8dbc0d3dd0 con 0x7f8dbc163bf0 2024-04-28T13:32:56.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.246+0000 7f8dbb400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dbc1629c0 0x7f8dbc0d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.247+0000 7f8dbb400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dbc1629c0 0x7f8dbc0d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41870/0 (socket says 172.21.15.196:41870) 2024-04-28T13:32:56.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.247+0000 7f8dbb400640 1 -- 172.21.15.196:0/2830158482 learned_addr learned my addr 172.21.15.196:0/2830158482 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:56.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.247+0000 7f8db2a00640 1 --2- 172.21.15.196:0/2830158482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dbc163570 0x7f8dbc15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.247+0000 7f8dbb400640 1 -- 172.21.15.196:0/2830158482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc163bf0 msgr2=0x7f8dbc15d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:56.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.247+0000 7f8dbb400640 1 --2- 172.21.15.196:0/2830158482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc163bf0 0x7f8dbc15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.247+0000 7f8dbb400640 1 -- 172.21.15.196:0/2830158482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dbc163570 msgr2=0x7f8dbc15cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:56.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.247+0000 7f8dbb400640 1 --2- 172.21.15.196:0/2830158482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dbc163570 0x7f8dbc15cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.247+0000 7f8dbb400640 1 -- 172.21.15.196:0/2830158482 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dbc0ba870 con 0x7f8dbc1629c0 2024-04-28T13:32:56.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.247+0000 7f8dbb400640 1 --2- 172.21.15.196:0/2830158482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dbc1629c0 0x7f8dbc0d12e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00ba80 tx=0x7f8dac00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:56.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.248+0000 7f8db3e00640 1 -- 172.21.15.196:0/2830158482 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac019070 con 0x7f8dbc1629c0 2024-04-28T13:32:56.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.248+0000 7f8db3e00640 1 -- 172.21.15.196:0/2830158482 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dac00cc20 con 0x7f8dbc1629c0 2024-04-28T13:32:56.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.248+0000 7f8dc1000640 1 -- 172.21.15.196:0/2830158482 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dbc0b7f80 con 0x7f8dbc1629c0 2024-04-28T13:32:56.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.248+0000 7f8db3e00640 1 -- 172.21.15.196:0/2830158482 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac014440 con 0x7f8dbc1629c0 2024-04-28T13:32:56.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.248+0000 7f8dc1000640 1 -- 172.21.15.196:0/2830158482 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dbc0b8500 con 0x7f8dbc1629c0 2024-04-28T13:32:56.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.249+0000 7f8dc1000640 1 -- 172.21.15.196:0/2830158482 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dbc06e480 con 0x7f8dbc1629c0 2024-04-28T13:32:56.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.250+0000 7f8db3e00640 1 -- 172.21.15.196:0/2830158482 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8dac01c070 con 0x7f8dbc1629c0 2024-04-28T13:32:56.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.251+0000 7f8db3e00640 1 --2- 172.21.15.196:0/2830158482 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8d9806c980 0x7f8d9806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.251+0000 7f8db3e00640 1 -- 172.21.15.196:0/2830158482 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8dac08b990 con 0x7f8dbc1629c0 2024-04-28T13:32:56.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.252+0000 7f8db2a00640 1 --2- 172.21.15.196:0/2830158482 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8d9806c980 0x7f8d9806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.253+0000 7f8db2a00640 1 --2- 172.21.15.196:0/2830158482 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8d9806c980 0x7f8d9806ee40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc1f89e0 tx=0x7f8da4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:56.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.256+0000 7f8db3e00640 1 -- 172.21.15.196:0/2830158482 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8dbc06e480 con 0x7f8dbc1629c0 2024-04-28T13:32:56.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.497+0000 7f8dc1000640 1 -- 172.21.15.196:0/2830158482 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8dbc0cb9f0 con 0x7f8dbc1629c0 2024-04-28T13:32:56.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.498+0000 7f8db3e00640 1 -- 172.21.15.196:0/2830158482 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f8dac059680 con 0x7f8dbc1629c0 2024-04-28T13:32:56.498 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:32:56.498 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/931076983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":931076983},{"type":"v1","addr":"172.21.15.196:6834","nonce":931076983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1114474485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1114474485},{"type":"v1","addr":"172.21.15.196:6835","nonce":1114474485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2070123371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2070123371},{"type":"v1","addr":"172.21.15.203:6837","nonce":2070123371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:32:31.669472+0000","modified":"2024-04-28T13:32:34.725159+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4735":{"gid":4735,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6836/459640695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":459640695},{"type":"v1","addr":"172.21.15.203:6836","nonce":459640695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4735,"qdb_cluster":[4735]},"id":2}]} 2024-04-28T13:32:56.499 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 16 2024-04-28T13:32:56.501 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.501+0000 7f8dc1000640 1 -- 172.21.15.196:0/2830158482 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8d9806c980 msgr2=0x7f8d9806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:56.501 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.501+0000 7f8dc1000640 1 --2- 172.21.15.196:0/2830158482 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8d9806c980 0x7f8d9806ee40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc1f89e0 tx=0x7f8da4008040 comp rx=0 tx=0).stop 2024-04-28T13:32:56.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.501+0000 7f8dc1000640 1 -- 172.21.15.196:0/2830158482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dbc1629c0 msgr2=0x7f8dbc0d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:56.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.501+0000 7f8dc1000640 1 --2- 172.21.15.196:0/2830158482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dbc1629c0 0x7f8dbc0d12e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00ba80 tx=0x7f8dac00bf40 comp rx=0 tx=0).stop 2024-04-28T13:32:56.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.501+0000 7f8dc1000640 1 -- 172.21.15.196:0/2830158482 shutdown_connections 2024-04-28T13:32:56.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.501+0000 7f8dc1000640 1 --2- 172.21.15.196:0/2830158482 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8d9806c980 0x7f8d9806ee40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.501+0000 7f8dc1000640 1 --2- 172.21.15.196:0/2830158482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc163bf0 0x7f8dbc15d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.501+0000 7f8dc1000640 1 --2- 172.21.15.196:0/2830158482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dbc163570 0x7f8dbc15cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.501+0000 7f8dc1000640 1 --2- 172.21.15.196:0/2830158482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dbc1629c0 0x7f8dbc0d12e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.501+0000 7f8dc1000640 1 -- 172.21.15.196:0/2830158482 >> 172.21.15.196:0/2830158482 conn(0x7f8dbc09e570 msgr2=0x7f8dbc0b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:56.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.502+0000 7f8dc1000640 1 -- 172.21.15.196:0/2830158482 shutdown_connections 2024-04-28T13:32:56.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.502+0000 7f8dc1000640 1 -- 172.21.15.196:0/2830158482 wait complete. 2024-04-28T13:32:56.520 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump tree /volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 0 2024-04-28T13:32:56.753 INFO:teuthology.orchestra.run.smithi203.stdout:[{"ino":1099511627779,"rdev":0,"ctime":"2024-04-28T13:32:55.654113+0000","btime":"2024-04-28T13:32:43.558491+0000","mode":17407,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":5,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-28T13:32:55.654113+0000","atime":"2024-04-28T13:32:43.558491+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-28T13:32:55.654113+0000","num_files":1,"num_subdirs":0,"change_attr":1},"rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-28T13:32:55.740111+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-28T13:32:55.740111+0000"},"version":20,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"request":1,"lock":2,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"waiter":0,"authpin":1},"nref":8,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{"gather_set":[],"state":"sync","type":"ipolicy","is_leased":false,"num_rdlocks":1,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":1,"xlock_by":{"reqid":{"entity":{"type":"mds","num":0},"tid":2}}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":4658,"pending":"pAsLsXs","issued":"pAsLsXs","wanted":"-","last_sent":9},{"client_id":4673,"pending":"pAsLsXsFs","issued":"pAsLsXsFs","wanted":"pAsLsXsFs","last_sent":8},{"client_id":4676,"pending":"pAsLsXsFs","issued":"pAsLsXsFs","wanted":"pAsLsXsFs","last_sent":8},{"client_id":5340,"pending":"pAsLsXsFs","issued":"pAsLsXsFs","wanted":"pAsLsXsFsxcral","last_sent":30},{"client_id":5349,"pending":"pAsLsXsFs","issued":"pAsLsXsFs","wanted":"pAsLsXsFs","last_sent":8}],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"7","version":"7","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1/pyvenv.cfg","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":1,"projected_version":0,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"lock":0,"authpin":0},"nref":0,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth"]},{"path":"volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1/file","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629282,"remote_type":"","version":6,"projected_version":6,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-04-28T13:32:56.754 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-04-28T13:32:55.740111+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-28T13:32:43.558491+0000", "auth_pins": 1, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-28T13:32:43.558491+0000", "change_attr": 3, "client_caps": [ { "client_id": 4658, "issued": "pAsLsXs", "last_sent": 9, "pending": "pAsLsXs", "wanted": "-" }, { "client_id": 4673, "issued": "pAsLsXsFs", "last_sent": 8, "pending": "pAsLsXsFs", "wanted": "pAsLsXsFs" }, { "client_id": 4676, "issued": "pAsLsXsFs", "last_sent": 8, "pending": "pAsLsXsFs", "wanted": "pAsLsXsFs" }, { "client_id": 5340, "issued": "pAsLsXsFs", "last_sent": 30, "pending": "pAsLsXsFs", "wanted": "pAsLsXsFsxcral" }, { "client_id": 5349, "issued": "pAsLsXsFs", "last_sent": 8, "pending": "pAsLsXsFs", "wanted": "pAsLsXsFs" } ], "client_ranges": [], "ctime": "2024-04-28T13:32:55.654113+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 0, "path": "volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1/pyvenv.cfg", "path_ino": 1, "pins": { "authpin": 0, "lock": 0 }, "projected_version": 0, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth" ], "version": 1, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629282, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1/file", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 6, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 6, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 1, "mtime": "2024-04-28T13:32:55.654113+0000", "num_files": 1, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 0, "ino": 1099511627779, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 5, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 17407, "mtime": "2024-04-28T13:32:55.654113+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 8, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 1, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 2, "request": 1, "waiter": 0 }, "policylock": { "gather_set": [], "is_leased": false, "num_rdlocks": 1, "num_wrlocks": 0, "num_xlocks": 0, "state": "sync", "type": "ipolicy", "xlock_by": {} }, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "state": "lock", "type": "iquiesce", "xlock_by": { "reqid": { "entity": { "num": 0, "type": "mds" }, "tid": 2 } } }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-04-28T13:32:55.740111+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 0, "version": 20, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } ] 2024-04-28T13:32:56.755 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 ops --flags=locks --path=/tmp/mds.0-ops 2024-04-28T13:32:56.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.971+0000 7f78d7c00640 1 Processor -- start 2024-04-28T13:32:56.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.972+0000 7f78d7c00640 1 -- start start 2024-04-28T13:32:56.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.973+0000 7f78d7c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78d0145960 0x7f78d00af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.973+0000 7f78d7c00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f78d00afdb0 0x7f78d00b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.973+0000 7f78d7c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78d01607d0 con 0x7f78d015fa10 2024-04-28T13:32:56.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.973+0000 7f78d7c00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f78d01605b0 con 0x7f78d00afdb0 2024-04-28T13:32:56.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.973+0000 7f78d7c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f78d00aea50 con 0x7f78d0145960 2024-04-28T13:32:56.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.974+0000 7f78d5c00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f78d015fa10 0x7f78d015fea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:45458/0 (socket says 172.21.15.196:45458) 2024-04-28T13:32:56.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.974+0000 7f78d5c00640 1 -- 172.21.15.196:0/4033650920 learned_addr learned my addr 172.21.15.196:0/4033650920 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:56.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.974+0000 7f78c7e00640 1 --2- 172.21.15.196:0/4033650920 >> v2:172.21.15.203:3300/0 conn(0x7f78d00afdb0 0x7f78d00b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.974+0000 7f78d5200640 1 --2- 172.21.15.196:0/4033650920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78d0145960 0x7f78d00af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.974+0000 7f78cfe00640 1 -- 172.21.15.196:0/4033650920 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 961653175 0 0) 0x7f78d01607d0 con 0x7f78d015fa10 2024-04-28T13:32:56.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.974+0000 7f78cfe00640 1 -- 172.21.15.196:0/4033650920 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78b0003520 con 0x7f78d015fa10 2024-04-28T13:32:56.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.975+0000 7f78cfe00640 1 -- 172.21.15.196:0/4033650920 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1960941904 0 0) 0x7f78b0003520 con 0x7f78d015fa10 2024-04-28T13:32:56.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.975+0000 7f78cfe00640 1 -- 172.21.15.196:0/4033650920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78d0145960 msgr2=0x7f78d00af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:56.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.975+0000 7f78cfe00640 1 --2- 172.21.15.196:0/4033650920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78d0145960 0x7f78d00af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.975+0000 7f78cfe00640 1 -- 172.21.15.196:0/4033650920 >> v2:172.21.15.203:3300/0 conn(0x7f78d00afdb0 msgr2=0x7f78d00b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:56.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.975+0000 7f78cfe00640 1 --2- 172.21.15.196:0/4033650920 >> v2:172.21.15.203:3300/0 conn(0x7f78d00afdb0 0x7f78d00b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.975+0000 7f78cfe00640 1 -- 172.21.15.196:0/4033650920 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d00aed30 con 0x7f78d015fa10 2024-04-28T13:32:56.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.975+0000 7f78cfe00640 1 -- 172.21.15.196:0/4033650920 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f78c80019f0 con 0x7f78d015fa10 2024-04-28T13:32:56.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.975+0000 7f78cfe00640 1 -- 172.21.15.196:0/4033650920 >> v1:172.21.15.196:6789/0 conn(0x7f78d015fa10 legacy=0x7f78d015fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:56.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.975+0000 7f78cfe00640 1 --2- 172.21.15.196:0/4033650920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78b00039b0 0x7f78b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.975+0000 7f78cfe00640 1 --2- 172.21.15.196:0/4033650920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78d0145960 0x7f78b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.975+0000 7f78cfe00640 1 --2- 172.21.15.196:0/4033650920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78b0004d40 0x7f78b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.975+0000 7f78cfe00640 1 -- 172.21.15.196:0/4033650920 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78d01605b0 con 0x7f78b00039b0 2024-04-28T13:32:56.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.975+0000 7f78cfe00640 1 -- 172.21.15.196:0/4033650920 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f78d00aea50 con 0x7f78b0004d40 2024-04-28T13:32:56.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.975+0000 7f78cfe00640 1 -- 172.21.15.196:0/4033650920 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f78b000b9c0 con 0x7f78d0145960 2024-04-28T13:32:56.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.975+0000 7f78d5200640 1 --2- 172.21.15.196:0/4033650920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78b00039b0 0x7f78b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.976+0000 7f78d5c00640 1 --2- 172.21.15.196:0/4033650920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78b0004d40 0x7f78b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.976+0000 7f78c7e00640 1 --2- 172.21.15.196:0/4033650920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78d0145960 0x7f78b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.976+0000 7f78d5200640 1 -- 172.21.15.196:0/4033650920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78d0145960 msgr2=0x7f78b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:56.978 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.976+0000 7f78d5200640 1 --2- 172.21.15.196:0/4033650920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78d0145960 0x7f78b0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.978 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.976+0000 7f78d5200640 1 -- 172.21.15.196:0/4033650920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78b0004d40 msgr2=0x7f78b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:56.978 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.976+0000 7f78d5200640 1 --2- 172.21.15.196:0/4033650920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78b0004d40 0x7f78b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.978 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.976+0000 7f78d5200640 1 -- 172.21.15.196:0/4033650920 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b0003520 con 0x7f78b00039b0 2024-04-28T13:32:56.978 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.976+0000 7f78d5c00640 1 --2- 172.21.15.196:0/4033650920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78b0004d40 0x7f78b00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:56.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.977+0000 7f78d5200640 1 --2- 172.21.15.196:0/4033650920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78b00039b0 0x7f78b0003dd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009ac0 tx=0x7f78b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba9cad541e0f4c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:56.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.977+0000 7f78cfe00640 1 -- 172.21.15.196:0/4033650920 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8036070 con 0x7f78b00039b0 2024-04-28T13:32:56.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.977+0000 7f78cfe00640 1 -- 172.21.15.196:0/4033650920 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b8031400 con 0x7f78b00039b0 2024-04-28T13:32:56.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.977+0000 7f78cfe00640 1 -- 172.21.15.196:0/4033650920 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8009690 con 0x7f78b00039b0 2024-04-28T13:32:56.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.978+0000 7f78d7c00640 1 -- 172.21.15.196:0/4033650920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78b00039b0 msgr2=0x7f78b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:56.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.978+0000 7f78d7c00640 1 --2- 172.21.15.196:0/4033650920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78b00039b0 0x7f78b0003dd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009ac0 tx=0x7f78b0005830 comp rx=0 tx=0).stop 2024-04-28T13:32:56.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.978+0000 7f78d5c00640 1 -- 172.21.15.196:0/4033650920 reap_dead start 2024-04-28T13:32:56.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.978+0000 7f78d7c00640 1 -- 172.21.15.196:0/4033650920 shutdown_connections 2024-04-28T13:32:56.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.978+0000 7f78d7c00640 1 -- 172.21.15.196:0/4033650920 >> 172.21.15.196:0/4033650920 conn(0x7f78d00cd8a0 msgr2=0x7f78d00cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:56.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.978+0000 7f78d7c00640 1 -- 172.21.15.196:0/4033650920 shutdown_connections 2024-04-28T13:32:56.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.978+0000 7f78d7c00640 1 -- 172.21.15.196:0/4033650920 wait complete. 2024-04-28T13:32:56.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.980+0000 7f78d7c00640 1 Processor -- start 2024-04-28T13:32:56.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.980+0000 7f78d7c00640 1 -- start start 2024-04-28T13:32:56.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.981+0000 7f78d7c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78b00039b0 0x7f78d00c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.981+0000 7f78d7c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78d01f57c0 0x7f78d015e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.981+0000 7f78d7c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00cb890 0x7f78d00cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.981+0000 7f78d7c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78d00af530 con 0x7f78d00cb890 2024-04-28T13:32:56.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.981+0000 7f78d7c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f78d00b0750 con 0x7f78b00039b0 2024-04-28T13:32:56.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.981+0000 7f78d7c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f78d00af6b0 con 0x7f78d01f57c0 2024-04-28T13:32:56.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.981+0000 7f78d5c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00cb890 0x7f78d00cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.981+0000 7f78d5c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00cb890 0x7f78d00cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41890/0 (socket says 172.21.15.196:41890) 2024-04-28T13:32:56.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.981+0000 7f78d5c00640 1 -- 172.21.15.196:0/4164415417 learned_addr learned my addr 172.21.15.196:0/4164415417 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:56.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.981+0000 7f78d5200640 1 --2- 172.21.15.196:0/4164415417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78b00039b0 0x7f78d00c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.981+0000 7f78c7e00640 1 --2- 172.21.15.196:0/4164415417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78d01f57c0 0x7f78d015e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.982+0000 7f78d5c00640 1 -- 172.21.15.196:0/4164415417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78d01f57c0 msgr2=0x7f78d015e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:56.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.982+0000 7f78d5c00640 1 --2- 172.21.15.196:0/4164415417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78d01f57c0 0x7f78d015e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.982+0000 7f78d5c00640 1 -- 172.21.15.196:0/4164415417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78b00039b0 msgr2=0x7f78d00c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:56.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.982+0000 7f78d5c00640 1 --2- 172.21.15.196:0/4164415417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78b00039b0 0x7f78d00c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.982+0000 7f78d5c00640 1 -- 172.21.15.196:0/4164415417 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d00c9110 con 0x7f78d00cb890 2024-04-28T13:32:56.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.982+0000 7f78d5200640 1 --2- 172.21.15.196:0/4164415417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78b00039b0 0x7f78d00c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:56.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.982+0000 7f78d5c00640 1 --2- 172.21.15.196:0/4164415417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00cb890 0x7f78d00cbd20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f78d015fdf0 tx=0x7f78c800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:56.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.982+0000 7f78cd600640 1 -- 172.21.15.196:0/4164415417 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c800bb20 con 0x7f78d00cb890 2024-04-28T13:32:56.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.982+0000 7f78cd600640 1 -- 172.21.15.196:0/4164415417 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c800bcc0 con 0x7f78d00cb890 2024-04-28T13:32:56.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.982+0000 7f78d7c00640 1 -- 172.21.15.196:0/4164415417 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d01f5c40 con 0x7f78d00cb890 2024-04-28T13:32:56.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.982+0000 7f78cd600640 1 -- 172.21.15.196:0/4164415417 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8007290 con 0x7f78d00cb890 2024-04-28T13:32:56.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.982+0000 7f78d7c00640 1 -- 172.21.15.196:0/4164415417 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d00c76b0 con 0x7f78d00cb890 2024-04-28T13:32:56.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.985+0000 7f78cd600640 1 -- 172.21.15.196:0/4164415417 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f78c801b020 con 0x7f78d00cb890 2024-04-28T13:32:56.985 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.985+0000 7f78cd600640 1 --2- 172.21.15.196:0/4164415417 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f78ac06c980 0x7f78ac06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.985 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.986+0000 7f78cd600640 1 -- 172.21.15.196:0/4164415417 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f78c808bf60 con 0x7f78d00cb890 2024-04-28T13:32:56.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.986+0000 7f78da965740 1 Processor -- start 2024-04-28T13:32:56.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.986+0000 7f78d5200640 1 --2- 172.21.15.196:0/4164415417 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f78ac06c980 0x7f78ac06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.986+0000 7f78da965740 1 -- start start 2024-04-28T13:32:56.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.987+0000 7f78da965740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a200620 0x55ea7a203da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.987+0000 7f78da965740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a2042e0 0x55ea7a220220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.987+0000 7f78da965740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a220760 0x55ea7a220bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.987+0000 7f78da965740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea7a135590 con 0x55ea7a200620 2024-04-28T13:32:56.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.987+0000 7f78da965740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ea7a0d1d60 con 0x55ea7a220760 2024-04-28T13:32:56.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.987+0000 7f78da965740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55ea7a131550 con 0x55ea7a2042e0 2024-04-28T13:32:56.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.988+0000 7f78d5200640 1 --2- 172.21.15.196:0/4164415417 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f78ac06c980 0x7f78ac06ee40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f78b8033000 tx=0x7f78b8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:56.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.988+0000 7f78d5200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a200620 0x55ea7a203da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.988+0000 7f78d5200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a200620 0x55ea7a203da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41892/0 (socket says 172.21.15.196:41892) 2024-04-28T13:32:56.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.988+0000 7f78d5200640 1 -- 172.21.15.196:0/3809530644 learned_addr learned my addr 172.21.15.196:0/3809530644 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:56.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.988+0000 7f78c7e00640 1 --2- 172.21.15.196:0/3809530644 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a2042e0 0x55ea7a220220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.988+0000 7f78d5c00640 1 --2- 172.21.15.196:0/3809530644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a220760 0x55ea7a220bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.988+0000 7f78d5200640 1 -- 172.21.15.196:0/3809530644 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a2042e0 msgr2=0x55ea7a220220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:56.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.988+0000 7f78d5200640 1 --2- 172.21.15.196:0/3809530644 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a2042e0 0x55ea7a220220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.988+0000 7f78d5200640 1 -- 172.21.15.196:0/3809530644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a220760 msgr2=0x55ea7a220bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:56.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.988+0000 7f78d5200640 1 --2- 172.21.15.196:0/3809530644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a220760 0x55ea7a220bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.988+0000 7f78d5200640 1 -- 172.21.15.196:0/3809530644 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea7a0f2b20 con 0x55ea7a200620 2024-04-28T13:32:56.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.988+0000 7f78d5c00640 1 --2- 172.21.15.196:0/3809530644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a220760 0x55ea7a220bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:56.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.989+0000 7f78d5200640 1 --2- 172.21.15.196:0/3809530644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a200620 0x55ea7a203da0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f78b802abb0 tx=0x7f78b802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a494846b92ba68d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:56.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.989+0000 7f78c5600640 1 -- 172.21.15.196:0/3809530644 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8036050 con 0x55ea7a200620 2024-04-28T13:32:56.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.989+0000 7f78c5600640 1 -- 172.21.15.196:0/3809530644 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b8037040 con 0x55ea7a200620 2024-04-28T13:32:56.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.989+0000 7f78c5600640 1 -- 172.21.15.196:0/3809530644 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8003ae0 con 0x55ea7a200620 2024-04-28T13:32:56.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.989+0000 7f78da965740 1 -- 172.21.15.196:0/3809530644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a200620 msgr2=0x55ea7a203da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:56.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.989+0000 7f78da965740 1 --2- 172.21.15.196:0/3809530644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a200620 0x55ea7a203da0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f78b802abb0 tx=0x7f78b802e000 comp rx=0 tx=0).stop 2024-04-28T13:32:56.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.990+0000 7f78da965740 1 -- 172.21.15.196:0/3809530644 shutdown_connections 2024-04-28T13:32:56.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.990+0000 7f78da965740 1 --2- 172.21.15.196:0/3809530644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a220760 0x55ea7a220bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.990+0000 7f78da965740 1 --2- 172.21.15.196:0/3809530644 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a2042e0 0x55ea7a220220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.992 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.990+0000 7f78da965740 1 --2- 172.21.15.196:0/3809530644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a200620 0x55ea7a203da0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.992 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.990+0000 7f78da965740 1 -- 172.21.15.196:0/3809530644 >> 172.21.15.196:0/3809530644 conn(0x55ea7a1d6600 msgr2=0x55ea7a1d7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:56.992 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.990+0000 7f78da965740 1 -- 172.21.15.196:0/3809530644 shutdown_connections 2024-04-28T13:32:56.992 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.990+0000 7f78da965740 1 -- 172.21.15.196:0/3809530644 wait complete. 2024-04-28T13:32:56.992 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.990+0000 7f78da965740 1 -- start start 2024-04-28T13:32:56.992 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.991+0000 7f78da965740 1 Processor -- start 2024-04-28T13:32:56.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.992+0000 7f78da965740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a2042e0 0x55ea7a1fc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.992+0000 7f78da965740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a220760 0x55ea7a1fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.992+0000 7f78da965740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a21a280 0x55ea7a207580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.992+0000 7f78da965740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea7a1d4430 con 0x55ea7a220760 2024-04-28T13:32:56.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.992+0000 7f78da965740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ea7a0bcd80 con 0x55ea7a2042e0 2024-04-28T13:32:56.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.992+0000 7f78da965740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55ea7a1d4710 con 0x55ea7a21a280 2024-04-28T13:32:56.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.992+0000 7f78da965740 10 client.0 _ll_register_callbacks cb 0x55ea7a12b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T13:32:56.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.992+0000 7f78c7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a220760 0x55ea7a1fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.992+0000 7f78c7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a220760 0x55ea7a1fd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41898/0 (socket says 172.21.15.196:41898) 2024-04-28T13:32:56.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.992+0000 7f78c7e00640 1 -- 172.21.15.196:0/724545077 learned_addr learned my addr 172.21.15.196:0/724545077 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:56.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.992+0000 7f78c7e00640 1 -- 172.21.15.196:0/724545077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a21a280 msgr2=0x55ea7a207580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:56.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.992+0000 7f78c7e00640 1 --2- 172.21.15.196:0/724545077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a21a280 0x55ea7a207580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.992+0000 7f78c7e00640 1 -- 172.21.15.196:0/724545077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a2042e0 msgr2=0x55ea7a1fc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:56.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.992+0000 7f78c7e00640 1 --2- 172.21.15.196:0/724545077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a2042e0 0x55ea7a1fc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.992+0000 7f78c7e00640 1 -- 172.21.15.196:0/724545077 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea79e47420 con 0x55ea7a220760 2024-04-28T13:32:56.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.993+0000 7f78c7e00640 1 --2- 172.21.15.196:0/724545077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a220760 0x55ea7a1fd3b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009b80 tx=0x7f78c00136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d0b36146228aea server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:56.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.993+0000 7f78be000640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-04-28T13:32:56.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.993+0000 7f78be000640 1 -- 172.21.15.196:0/724545077 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c001a030 con 0x55ea7a220760 2024-04-28T13:32:56.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.993+0000 7f78be000640 1 -- 172.21.15.196:0/724545077 <== mon.0 v2:172.21.15.196:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f78c0016460 con 0x55ea7a220760 2024-04-28T13:32:56.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.993+0000 7f78da965740 1 -- 172.21.15.196:0/724545077 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78b8037040 con 0x55ea7a220760 2024-04-28T13:32:56.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.993+0000 7f78be000640 1 -- 172.21.15.196:0/724545077 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f78c0013e80 con 0x55ea7a220760 2024-04-28T13:32:56.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.993+0000 7f78da965740 10 client.5403 fetch_fsmap learned FSMap version 16 2024-04-28T13:32:56.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.993+0000 7f78da965740 1 -- 172.21.15.196:0/724545077 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55ea79f13830 con 0x55ea7a220760 2024-04-28T13:32:56.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.993+0000 7f78be000640 1 -- 172.21.15.196:0/724545077 <== mon.0 v2:172.21.15.196:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b8037040 con 0x55ea7a220760 2024-04-28T13:32:56.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.993+0000 7f78be000640 1 -- 172.21.15.196:0/724545077 <== mon.0 v2:172.21.15.196:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f78c0029080 con 0x55ea7a220760 2024-04-28T13:32:56.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.993+0000 7f78da965740 10 client.5403 fetch_fsmap finished waiting for FSMap version 16 2024-04-28T13:32:56.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.993+0000 7f78da965740 10 client.5403 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-04-28T13:32:56.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.994+0000 7f78da965740 20 client.5403 populate_metadata read hostname 'smithi196' 2024-04-28T13:32:56.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.994+0000 7f78da965740 1 --2- 172.21.15.196:0/724545077 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55ea7a21b110 0x55ea7a21b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:56.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.994+0000 7f78da965740 4 client.5403 mds_command: new command op to 4735 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T13:32:56.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.994+0000 7f78da965740 1 -- 172.21.15.196:0/724545077 --> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea7a1431a0 con 0x55ea7a21b110 2024-04-28T13:32:56.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.994+0000 7f78d5200640 1 --2- 172.21.15.196:0/724545077 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55ea7a21b110 0x55ea7a21b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:56.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.995+0000 7f78d5200640 1 --2- 172.21.15.196:0/724545077 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55ea7a21b110 0x55ea7a21b590 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fbe72ad41bda985 server_cookie=70948be28e8712b2 in_seq=0 out_seq=0 2024-04-28T13:32:56.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.995+0000 7f78be000640 10 client.5403 ms_handle_connect on v2:172.21.15.203:6835/459640695 2024-04-28T13:32:56.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.997+0000 7f78be000640 1 -- 172.21.15.196:0/724545077 <== mds.0 v2:172.21.15.203:6835/459640695 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea7a1431a0 con 0x55ea7a21b110 2024-04-28T13:32:56.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.997+0000 7f78be000640 10 client.5403 handle_command_reply: tid=0 2024-04-28T13:32:56.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.997+0000 7f78da965740 1 client.5403 shutdown 2024-04-28T13:32:56.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.997+0000 7f78da965740 1 -- 172.21.15.196:0/724545077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a220760 msgr2=0x55ea7a1fd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:56.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.997+0000 7f78da965740 1 --2- 172.21.15.196:0/724545077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a220760 0x55ea7a1fd3b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009b80 tx=0x7f78c00136b0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.997+0000 7f78da965740 1 -- 172.21.15.196:0/724545077 shutdown_connections 2024-04-28T13:32:56.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.997+0000 7f78da965740 1 --2- 172.21.15.196:0/724545077 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55ea7a21b110 0x55ea7a21b590 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.997+0000 7f78da965740 1 --2- 172.21.15.196:0/724545077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a21a280 0x55ea7a207580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.997+0000 7f78da965740 1 --2- 172.21.15.196:0/724545077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a220760 0x55ea7a1fd3b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.997+0000 7f78be000640 0 client.5403 ms_handle_reset on v2:172.21.15.203:6835/459640695 2024-04-28T13:32:56.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.997+0000 7f78da965740 1 --2- 172.21.15.196:0/724545077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a2042e0 0x55ea7a1fc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:56.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.998+0000 7f78da965740 1 -- 172.21.15.196:0/724545077 >> 172.21.15.196:0/724545077 conn(0x55ea7a200620 msgr2=0x55ea7a21f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:56.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.998+0000 7f78da965740 1 -- 172.21.15.196:0/724545077 shutdown_connections 2024-04-28T13:32:56.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.998+0000 7f78da965740 1 -- 172.21.15.196:0/724545077 wait complete. 2024-04-28T13:32:57.000 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:56.998+0000 7f78da965740 20 client.5403 trim_cache size 0 max 16384 2024-04-28T13:32:57.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.015+0000 7f78da965740 1 Processor -- start 2024-04-28T13:32:57.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.015+0000 7f78da965740 1 -- start start 2024-04-28T13:32:57.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.016+0000 7f78da965740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a2042e0 0x55ea7a21ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:57.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.016+0000 7f78da965740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a21a280 0x55ea7a231040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:57.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.016+0000 7f78da965740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a220760 0x55ea7a231580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:57.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.016+0000 7f78da965740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78d00ace10 con 0x55ea7a2042e0 2024-04-28T13:32:57.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.016+0000 7f78da965740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f78d00abe70 con 0x55ea7a220760 2024-04-28T13:32:57.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.016+0000 7f78da965740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f78d00ca5e0 con 0x55ea7a21a280 2024-04-28T13:32:57.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.016+0000 7f78d5200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a2042e0 0x55ea7a21ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:57.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.016+0000 7f78d5200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a2042e0 0x55ea7a21ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41912/0 (socket says 172.21.15.196:41912) 2024-04-28T13:32:57.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.016+0000 7f78d5200640 1 -- 172.21.15.196:0/1756165380 learned_addr learned my addr 172.21.15.196:0/1756165380 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:57.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.017+0000 7f78d5c00640 1 --2- 172.21.15.196:0/1756165380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a220760 0x55ea7a231580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:57.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.017+0000 7f78c7e00640 1 --2- 172.21.15.196:0/1756165380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a21a280 0x55ea7a231040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:57.018 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.017+0000 7f78d5200640 1 -- 172.21.15.196:0/1756165380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a21a280 msgr2=0x55ea7a231040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:57.018 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.017+0000 7f78d5200640 1 --2- 172.21.15.196:0/1756165380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a21a280 0x55ea7a231040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.018 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.017+0000 7f78d5200640 1 -- 172.21.15.196:0/1756165380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a220760 msgr2=0x55ea7a231580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:57.018 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.017+0000 7f78d5200640 1 --2- 172.21.15.196:0/1756165380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a220760 0x55ea7a231580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.018 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.017+0000 7f78d5200640 1 -- 172.21.15.196:0/1756165380 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea7a219b40 con 0x55ea7a2042e0 2024-04-28T13:32:57.018 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.017+0000 7f78d5200640 1 --2- 172.21.15.196:0/1756165380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a2042e0 0x55ea7a21ef60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009a90 tx=0x7f78b802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430b1e8091d3de03 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:57.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.017+0000 7f78c4c00640 1 -- 172.21.15.196:0/1756165380 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8058070 con 0x55ea7a2042e0 2024-04-28T13:32:57.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.017+0000 7f78c4c00640 1 -- 172.21.15.196:0/1756165380 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b8037040 con 0x55ea7a2042e0 2024-04-28T13:32:57.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.017+0000 7f78c4c00640 1 -- 172.21.15.196:0/1756165380 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8036070 con 0x55ea7a2042e0 2024-04-28T13:32:57.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.018+0000 7f78da965740 1 -- 172.21.15.196:0/1756165380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a2042e0 msgr2=0x55ea7a21ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:57.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.018+0000 7f78da965740 1 --2- 172.21.15.196:0/1756165380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a2042e0 0x55ea7a21ef60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009a90 tx=0x7f78b802c000 comp rx=0 tx=0).stop 2024-04-28T13:32:57.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.018+0000 7f78da965740 1 -- 172.21.15.196:0/1756165380 shutdown_connections 2024-04-28T13:32:57.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.018+0000 7f78da965740 1 --2- 172.21.15.196:0/1756165380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a220760 0x55ea7a231580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.018+0000 7f78da965740 1 --2- 172.21.15.196:0/1756165380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a21a280 0x55ea7a231040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.018+0000 7f78da965740 1 --2- 172.21.15.196:0/1756165380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a2042e0 0x55ea7a21ef60 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.018+0000 7f78da965740 1 -- 172.21.15.196:0/1756165380 >> 172.21.15.196:0/1756165380 conn(0x55ea7a200620 msgr2=0x55ea7a22e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:57.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.018+0000 7f78da965740 1 -- 172.21.15.196:0/1756165380 shutdown_connections 2024-04-28T13:32:57.019 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.018+0000 7f78da965740 1 -- 172.21.15.196:0/1756165380 wait complete. 2024-04-28T13:32:57.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.019+0000 7f78da965740 1 -- start start 2024-04-28T13:32:57.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.019+0000 7f78da965740 1 Processor -- start 2024-04-28T13:32:57.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.020+0000 7f78da965740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a2042e0 0x55ea7a222320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:57.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.020+0000 7f78da965740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a21a280 0x55ea7a240890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:57.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.020+0000 7f78da965740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a220760 0x55ea7a21f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:57.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.020+0000 7f78da965740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea7a1d5d80 con 0x55ea7a220760 2024-04-28T13:32:57.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.020+0000 7f78da965740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f78d00ab820 con 0x55ea7a21a280 2024-04-28T13:32:57.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.020+0000 7f78da965740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55ea7a1ffc50 con 0x55ea7a2042e0 2024-04-28T13:32:57.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.020+0000 7f78da965740 10 client.0 _ll_register_callbacks cb 0x7f78c0009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T13:32:57.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.020+0000 7f78d5c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a220760 0x55ea7a21f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:57.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.020+0000 7f78d5c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a220760 0x55ea7a21f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41922/0 (socket says 172.21.15.196:41922) 2024-04-28T13:32:57.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.020+0000 7f78d5c00640 1 -- 172.21.15.196:0/2585975008 learned_addr learned my addr 172.21.15.196:0/2585975008 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:57.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.021+0000 7f78c7e00640 1 --2- 172.21.15.196:0/2585975008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a21a280 0x55ea7a240890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:57.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.021+0000 7f78d5200640 1 --2- 172.21.15.196:0/2585975008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a2042e0 0x55ea7a222320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:57.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.021+0000 7f78d5c00640 1 -- 172.21.15.196:0/2585975008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a2042e0 msgr2=0x55ea7a222320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:57.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.021+0000 7f78d5c00640 1 --2- 172.21.15.196:0/2585975008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a2042e0 0x55ea7a222320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.021+0000 7f78d5c00640 1 -- 172.21.15.196:0/2585975008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a21a280 msgr2=0x55ea7a240890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:57.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.021+0000 7f78d5c00640 1 --2- 172.21.15.196:0/2585975008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a21a280 0x55ea7a240890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.021+0000 7f78d5c00640 1 -- 172.21.15.196:0/2585975008 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea7a206d80 con 0x55ea7a220760 2024-04-28T13:32:57.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.021+0000 7f78d5c00640 1 --2- 172.21.15.196:0/2585975008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a220760 0x55ea7a21f360 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f78c8055ef0 tx=0x7f78c800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55a098f3eee9d51 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:57.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.021+0000 7f78be000640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-04-28T13:32:57.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.021+0000 7f78be000640 1 -- 172.21.15.196:0/2585975008 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8088060 con 0x55ea7a220760 2024-04-28T13:32:57.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.021+0000 7f78be000640 1 -- 172.21.15.196:0/2585975008 <== mon.0 v2:172.21.15.196:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f78c80623e0 con 0x55ea7a220760 2024-04-28T13:32:57.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.021+0000 7f78da965740 1 -- 172.21.15.196:0/2585975008 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78b8037040 con 0x55ea7a220760 2024-04-28T13:32:57.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.021+0000 7f78be000640 1 -- 172.21.15.196:0/2585975008 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f78c805de40 con 0x55ea7a220760 2024-04-28T13:32:57.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.021+0000 7f78da965740 10 client.5409 fetch_fsmap learned FSMap version 16 2024-04-28T13:32:57.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.021+0000 7f78da965740 1 -- 172.21.15.196:0/2585975008 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55ea7a220590 con 0x55ea7a220760 2024-04-28T13:32:57.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.022+0000 7f78be000640 1 -- 172.21.15.196:0/2585975008 <== mon.0 v2:172.21.15.196:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b8037040 con 0x55ea7a220760 2024-04-28T13:32:57.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.022+0000 7f78be000640 1 -- 172.21.15.196:0/2585975008 <== mon.0 v2:172.21.15.196:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f78c80600f0 con 0x55ea7a220760 2024-04-28T13:32:57.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.022+0000 7f78da965740 10 client.5409 fetch_fsmap finished waiting for FSMap version 16 2024-04-28T13:32:57.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.022+0000 7f78da965740 10 client.5409 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-04-28T13:32:57.024 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.022+0000 7f78da965740 20 client.5409 populate_metadata read hostname 'smithi196' 2024-04-28T13:32:57.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.022+0000 7f78da965740 1 --2- 172.21.15.196:0/2585975008 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55ea7a29d450 0x55ea7a202fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:57.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.022+0000 7f78da965740 4 client.5409 mds_command: new command op to 4735 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/tmp/mds.0-ops"}] 2024-04-28T13:32:57.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.022+0000 7f78da965740 1 -- 172.21.15.196:0/2585975008 --> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/tmp/mds.0-ops"}) -- 0x55ea7a221da0 con 0x55ea7a29d450 2024-04-28T13:32:57.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.022+0000 7f78d5200640 1 --2- 172.21.15.196:0/2585975008 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55ea7a29d450 0x55ea7a202fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:57.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.024+0000 7f78d5200640 1 --2- 172.21.15.196:0/2585975008 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55ea7a29d450 0x55ea7a202fd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ede95eb1310a2f5 server_cookie=f23f61d8f5681e59 in_seq=0 out_seq=0 2024-04-28T13:32:57.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.024+0000 7f78be000640 10 client.5409 ms_handle_connect on v2:172.21.15.203:6835/459640695 2024-04-28T13:32:57.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.026+0000 7f78be000640 1 -- 172.21.15.196:0/2585975008 <== mds.0 v2:172.21.15.203:6835/459640695 1 ==== command_reply(tid 0: 0 ) ==== 8+0+33 (crc 0 0 0) 0x55ea7a221da0 con 0x55ea7a29d450 2024-04-28T13:32:57.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.026+0000 7f78be000640 10 client.5409 handle_command_reply: tid=0 2024-04-28T13:32:57.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.026+0000 7f78da965740 1 client.5409 shutdown 2024-04-28T13:32:57.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.026+0000 7f78da965740 1 -- 172.21.15.196:0/2585975008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a220760 msgr2=0x55ea7a21f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:57.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.026+0000 7f78da965740 1 --2- 172.21.15.196:0/2585975008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a220760 0x55ea7a21f360 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f78c8055ef0 tx=0x7f78c800bf30 comp rx=0 tx=0).stop 2024-04-28T13:32:57.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.026+0000 7f78da965740 1 -- 172.21.15.196:0/2585975008 shutdown_connections 2024-04-28T13:32:57.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.026+0000 7f78da965740 1 --2- 172.21.15.196:0/2585975008 >> [v2:172.21.15.203:6835/459640695,v1:172.21.15.203:6836/459640695] conn(0x55ea7a29d450 0x55ea7a202fd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.026+0000 7f78da965740 1 --2- 172.21.15.196:0/2585975008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea7a220760 0x55ea7a21f360 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.026+0000 7f78da965740 1 --2- 172.21.15.196:0/2585975008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea7a21a280 0x55ea7a240890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.026+0000 7f78be000640 0 client.5409 ms_handle_reset on v2:172.21.15.203:6835/459640695 2024-04-28T13:32:57.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.026+0000 7f78da965740 1 --2- 172.21.15.196:0/2585975008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ea7a2042e0 0x55ea7a222320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.026+0000 7f78da965740 1 -- 172.21.15.196:0/2585975008 >> 172.21.15.196:0/2585975008 conn(0x55ea7a200620 msgr2=0x55ea7a1caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:57.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.026+0000 7f78da965740 1 -- 172.21.15.196:0/2585975008 shutdown_connections 2024-04-28T13:32:57.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.026+0000 7f78da965740 1 -- 172.21.15.196:0/2585975008 wait complete. 2024-04-28T13:32:57.027 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.026+0000 7f78da965740 20 client.5409 trim_cache size 0 max 16384 2024-04-28T13:32:57.028 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:32:57.028 INFO:teuthology.orchestra.run.smithi196.stdout: "path": "/tmp/mds.0-ops" 2024-04-28T13:32:57.028 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:32:57.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.027+0000 7f78d7c00640 1 -- 172.21.15.196:0/4164415417 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f78ac06c980 msgr2=0x7f78ac06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:57.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.027+0000 7f78d7c00640 1 --2- 172.21.15.196:0/4164415417 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f78ac06c980 0x7f78ac06ee40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f78b8033000 tx=0x7f78b8008030 comp rx=0 tx=0).stop 2024-04-28T13:32:57.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.027+0000 7f78d7c00640 1 -- 172.21.15.196:0/4164415417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00cb890 msgr2=0x7f78d00cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:57.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.027+0000 7f78d7c00640 1 --2- 172.21.15.196:0/4164415417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00cb890 0x7f78d00cbd20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f78d015fdf0 tx=0x7f78c800b4a0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.027+0000 7f78d7c00640 1 -- 172.21.15.196:0/4164415417 shutdown_connections 2024-04-28T13:32:57.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.027+0000 7f78d7c00640 1 --2- 172.21.15.196:0/4164415417 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f78ac06c980 0x7f78ac06ee40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.027+0000 7f78d7c00640 1 --2- 172.21.15.196:0/4164415417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00cb890 0x7f78d00cbd20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.027+0000 7f78d7c00640 1 --2- 172.21.15.196:0/4164415417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78d01f57c0 0x7f78d015e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.027+0000 7f78d7c00640 1 --2- 172.21.15.196:0/4164415417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78b00039b0 0x7f78d00c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:57.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.027+0000 7f78d7c00640 1 -- 172.21.15.196:0/4164415417 >> 172.21.15.196:0/4164415417 conn(0x7f78d00cd8a0 msgr2=0x7f78d006e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:57.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.027+0000 7f78d7c00640 1 -- 172.21.15.196:0/4164415417 shutdown_connections 2024-04-28T13:32:57.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:57.027+0000 7f78d7c00640 1 -- 172.21.15.196:0/4164415417 wait complete. 2024-04-28T13:32:57.043 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-04-28T13:32:57.099 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.70kahbBTth 2024-04-28T13:32:57.099 DEBUG:teuthology.orchestra.run.smithi203:> sudo cp /tmp/mds.0-ops /tmp/tmp.70kahbBTth 2024-04-28T13:32:57.185 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.70kahbBTth 2024-04-28T13:32:57.328 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.70kahbBTth is 8KB 2024-04-28T13:32:57.388 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.70kahbBTth 2024-04-28T13:32:57.440 DEBUG:tasks.cephfs.filesystem:read 7841B of ops 2024-04-28T13:32:57.441 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump tree /volumes/_nogroup/subvol_quiesce/98a19700-b32b-45c8-aa64-134b105010c1 --path=/tmp/mds.0-cache 2024-04-28T13:32:57.664 INFO:teuthology.orchestra.run.smithi203.stdout:{"path":"/tmp/mds.0-cache"} 2024-04-28T13:32:57.664 DEBUG:tasks.cephfs.filesystem:_json_asok output { "path": "/tmp/mds.0-cache" } 2024-04-28T13:32:57.664 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-04-28T13:32:57.719 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.vE47u3P1ex 2024-04-28T13:32:57.719 DEBUG:teuthology.orchestra.run.smithi203:> sudo cp /tmp/mds.0-cache /tmp/tmp.vE47u3P1ex 2024-04-28T13:32:57.799 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.vE47u3P1ex 2024-04-28T13:32:57.942 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.vE47u3P1ex is 8KB 2024-04-28T13:32:58.003 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.vE47u3P1ex 2024-04-28T13:32:58.053 DEBUG:tasks.cephfs.filesystem:read 7745B of cache 2024-04-28T13:32:58.054 INFO:tasks.cephfs.test_quiesce:qp = 1; qi = 1; qib = 1; qina = 0 2024-04-28T13:32:58.054 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-28T13:32:58.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.269+0000 7f5e1d800640 1 Processor -- start 2024-04-28T13:32:58.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.269+0000 7f5e1d800640 1 -- start start 2024-04-28T13:32:58.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.270+0000 7f5e1d800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e1815b6f0 0x7f5e1806f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:58.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.270+0000 7f5e1d800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5e18073040 0x7f5e180cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:58.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.270+0000 7f5e1d800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e180cefb0 con 0x7f5e1806e330 2024-04-28T13:32:58.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.270+0000 7f5e1d800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5e180b6bc0 con 0x7f5e18073040 2024-04-28T13:32:58.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.270+0000 7f5e1d800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5e1815bad0 con 0x7f5e1815b6f0 2024-04-28T13:32:58.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.271+0000 7f5e17400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5e18073040 0x7f5e180cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:58.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.271+0000 7f5e17400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5e18073040 0x7f5e180cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41934/0 (socket says 172.21.15.196:41934) 2024-04-28T13:32:58.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.271+0000 7f5e17400640 1 -- 172.21.15.196:0/3740694615 learned_addr learned my addr 172.21.15.196:0/3740694615 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:58.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.271+0000 7f5e16a00640 1 --2- 172.21.15.196:0/3740694615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e1815b6f0 0x7f5e1806f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:58.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.271+0000 7f5e17400640 1 -- 172.21.15.196:0/3740694615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e1815b6f0 msgr2=0x7f5e1806f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:58.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.271+0000 7f5e17400640 1 --2- 172.21.15.196:0/3740694615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e1815b6f0 0x7f5e1806f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:58.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.271+0000 7f5e17400640 1 -- 172.21.15.196:0/3740694615 >> v1:172.21.15.203:6789/0 conn(0x7f5e1806e330 legacy=0x7f5e18072930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:58.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.271+0000 7f5e17400640 1 -- 172.21.15.196:0/3740694615 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1806f8e0 con 0x7f5e18073040 2024-04-28T13:32:58.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.272+0000 7f5e17400640 1 --2- 172.21.15.196:0/3740694615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e18073040 0x7f5e180cfa70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009e80 tx=0x7f5e1002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63da172876e619e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:58.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.272+0000 7f5e15600640 1 -- 172.21.15.196:0/3740694615 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10036070 con 0x7f5e18073040 2024-04-28T13:32:58.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.272+0000 7f5e15600640 1 -- 172.21.15.196:0/3740694615 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e10031600 con 0x7f5e18073040 2024-04-28T13:32:58.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.272+0000 7f5e15600640 1 -- 172.21.15.196:0/3740694615 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100319d0 con 0x7f5e18073040 2024-04-28T13:32:58.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.273+0000 7f5e1d800640 1 -- 172.21.15.196:0/3740694615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e18073040 msgr2=0x7f5e180cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:58.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.273+0000 7f5e1d800640 1 --2- 172.21.15.196:0/3740694615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e18073040 0x7f5e180cfa70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009e80 tx=0x7f5e1002a6f0 comp rx=0 tx=0).stop 2024-04-28T13:32:58.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.273+0000 7f5e1d800640 1 -- 172.21.15.196:0/3740694615 shutdown_connections 2024-04-28T13:32:58.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.273+0000 7f5e1d800640 1 --2- 172.21.15.196:0/3740694615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e18073040 0x7f5e180cfa70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:58.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.273+0000 7f5e1d800640 1 --2- 172.21.15.196:0/3740694615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e1815b6f0 0x7f5e1806f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:58.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.273+0000 7f5e1d800640 1 -- 172.21.15.196:0/3740694615 >> 172.21.15.196:0/3740694615 conn(0x7f5e180cd830 msgr2=0x7f5e180cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:58.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.273+0000 7f5e1d800640 1 -- 172.21.15.196:0/3740694615 shutdown_connections 2024-04-28T13:32:58.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.273+0000 7f5e1d800640 1 -- 172.21.15.196:0/3740694615 wait complete. 2024-04-28T13:32:58.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.275+0000 7f5e1d800640 1 Processor -- start 2024-04-28T13:32:58.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.275+0000 7f5e1d800640 1 -- start start 2024-04-28T13:32:58.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.276+0000 7f5e1d800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e1806e330 0x7f5e18155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:58.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.276+0000 7f5e1d800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e18073040 0x7f5e18147c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:58.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.276+0000 7f5e1d800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e1815b6f0 0x7f5e18148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:58.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.276+0000 7f5e1d800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5e180cf550 con 0x7f5e18073040 2024-04-28T13:32:58.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.276+0000 7f5e1d800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5e180cf270 con 0x7f5e1806e330 2024-04-28T13:32:58.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.276+0000 7f5e1d800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5e180b4020 con 0x7f5e1815b6f0 2024-04-28T13:32:58.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.276+0000 7f5e16000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e18073040 0x7f5e18147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:58.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.276+0000 7f5e16000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e18073040 0x7f5e18147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41940/0 (socket says 172.21.15.196:41940) 2024-04-28T13:32:58.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.276+0000 7f5e16000640 1 -- 172.21.15.196:0/3148685493 learned_addr learned my addr 172.21.15.196:0/3148685493 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:58.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.277+0000 7f5e17400640 1 --2- 172.21.15.196:0/3148685493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e1815b6f0 0x7f5e18148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:58.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.277+0000 7f5e16000640 1 -- 172.21.15.196:0/3148685493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e1815b6f0 msgr2=0x7f5e18148170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:58.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.277+0000 7f5e16000640 1 --2- 172.21.15.196:0/3148685493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e1815b6f0 0x7f5e18148170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:58.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.277+0000 7f5e16000640 1 -- 172.21.15.196:0/3148685493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e1806e330 msgr2=0x7f5e18155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:58.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.277+0000 7f5e16a00640 1 --2- 172.21.15.196:0/3148685493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e1806e330 0x7f5e18155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:58.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.277+0000 7f5e16000640 1 --2- 172.21.15.196:0/3148685493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e1806e330 0x7f5e18155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:58.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.277+0000 7f5e16000640 1 -- 172.21.15.196:0/3148685493 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e181564a0 con 0x7f5e18073040 2024-04-28T13:32:58.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.277+0000 7f5e16000640 1 --2- 172.21.15.196:0/3148685493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e18073040 0x7f5e18147c30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400c410 tx=0x7f5e0400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:58.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.277+0000 7f5e0ea00640 1 -- 172.21.15.196:0/3148685493 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04017070 con 0x7f5e18073040 2024-04-28T13:32:58.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.277+0000 7f5e0ea00640 1 -- 172.21.15.196:0/3148685493 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e040035f0 con 0x7f5e18073040 2024-04-28T13:32:58.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.277+0000 7f5e0ea00640 1 -- 172.21.15.196:0/3148685493 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04003a20 con 0x7f5e18073040 2024-04-28T13:32:58.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.277+0000 7f5e1d800640 1 -- 172.21.15.196:0/3148685493 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e18156780 con 0x7f5e18073040 2024-04-28T13:32:58.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.277+0000 7f5e1d800640 1 -- 172.21.15.196:0/3148685493 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e18156d40 con 0x7f5e18073040 2024-04-28T13:32:58.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.278+0000 7f5e1d800640 1 -- 172.21.15.196:0/3148685493 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e1806a290 con 0x7f5e18073040 2024-04-28T13:32:58.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.279+0000 7f5e0ea00640 1 -- 172.21.15.196:0/3148685493 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f5e04003bc0 con 0x7f5e18073040 2024-04-28T13:32:58.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.280+0000 7f5e0ea00640 1 --2- 172.21.15.196:0/3148685493 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5df806c9b0 0x7f5df806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:58.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.280+0000 7f5e0ea00640 1 -- 172.21.15.196:0/3148685493 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5e0408af80 con 0x7f5e18073040 2024-04-28T13:32:58.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.281+0000 7f5e16a00640 1 --2- 172.21.15.196:0/3148685493 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5df806c9b0 0x7f5df806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:58.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.282+0000 7f5e16a00640 1 --2- 172.21.15.196:0/3148685493 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5df806c9b0 0x7f5df806ee70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5e181479f0 tx=0x7f5e08008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:58.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.285+0000 7f5e0ea00640 1 -- 172.21.15.196:0/3148685493 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5e040592c0 con 0x7f5e18073040 2024-04-28T13:32:58.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.526+0000 7f5e1d800640 1 -- 172.21.15.196:0/3148685493 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5e180b5d00 con 0x7f5e18073040 2024-04-28T13:32:58.527 INFO:tasks.ceph.mon.a.smithi196.stderr:2024-04-28T13:32:58.527+0000 7f340ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-28T13:32:58.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.530+0000 7f5e0ea00640 1 -- 172.21.15.196:0/3148685493 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f5e04058bc0 con 0x7f5e18073040 2024-04-28T13:32:58.531 INFO:teuthology.orchestra.run.smithi196.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-28T13:32:58.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.534+0000 7f5e1d800640 1 -- 172.21.15.196:0/3148685493 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5df806c9b0 msgr2=0x7f5df806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:58.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.534+0000 7f5e1d800640 1 --2- 172.21.15.196:0/3148685493 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5df806c9b0 0x7f5df806ee70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5e181479f0 tx=0x7f5e08008040 comp rx=0 tx=0).stop 2024-04-28T13:32:58.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.534+0000 7f5e1d800640 1 -- 172.21.15.196:0/3148685493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e18073040 msgr2=0x7f5e18147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:58.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.534+0000 7f5e1d800640 1 --2- 172.21.15.196:0/3148685493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e18073040 0x7f5e18147c30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400c410 tx=0x7f5e0400c8d0 comp rx=0 tx=0).stop 2024-04-28T13:32:58.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.534+0000 7f5e1d800640 1 -- 172.21.15.196:0/3148685493 shutdown_connections 2024-04-28T13:32:58.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.534+0000 7f5e1d800640 1 --2- 172.21.15.196:0/3148685493 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5df806c9b0 0x7f5df806ee70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:58.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.534+0000 7f5e1d800640 1 --2- 172.21.15.196:0/3148685493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e1815b6f0 0x7f5e18148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:58.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.534+0000 7f5e1d800640 1 --2- 172.21.15.196:0/3148685493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e18073040 0x7f5e18147c30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:58.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.534+0000 7f5e1d800640 1 --2- 172.21.15.196:0/3148685493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e1806e330 0x7f5e18155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:58.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.534+0000 7f5e1d800640 1 -- 172.21.15.196:0/3148685493 >> 172.21.15.196:0/3148685493 conn(0x7f5e180cd830 msgr2=0x7f5e180cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:58.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.534+0000 7f5e1d800640 1 -- 172.21.15.196:0/3148685493 shutdown_connections 2024-04-28T13:32:58.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.534+0000 7f5e1d800640 1 -- 172.21.15.196:0/3148685493 wait complete. 2024-04-28T13:32:58.554 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-28T13:32:58.655 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-28T13:32:58.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.758+0000 7f4bb3e00640 1 Processor -- start 2024-04-28T13:32:58.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.758+0000 7f4bb3e00640 1 -- start start 2024-04-28T13:32:58.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.760+0000 7f4bb3e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4bac073a30 0x7f4bac06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:58.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.760+0000 7f4bb3e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bac073e60 con 0x7f4bac06c2a0 2024-04-28T13:32:58.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.760+0000 7f4bb3e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bac0d09a0 con 0x7f4bac1400c0 2024-04-28T13:32:58.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.760+0000 7f4bb3e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4bac0d0770 con 0x7f4bac073a30 2024-04-28T13:32:58.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.761+0000 7f4bb0a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f4bac06c2a0 0x7f4bac073320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:45462/0 (socket says 172.21.15.196:45462) 2024-04-28T13:32:58.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.761+0000 7f4bb0a00640 1 -- 172.21.15.196:0/360426386 learned_addr learned my addr 172.21.15.196:0/360426386 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:32:58.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.761+0000 7f4bb1e00640 1 --2- 172.21.15.196:0/360426386 >> v2:172.21.15.196:3300/0 conn(0x7f4bac073a30 0x7f4bac06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:58.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.761+0000 7f4babe00640 1 -- 172.21.15.196:0/360426386 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2831494168 0 0) 0x7f4bac073e60 con 0x7f4bac06c2a0 2024-04-28T13:32:58.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.761+0000 7f4babe00640 1 -- 172.21.15.196:0/360426386 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b90003520 con 0x7f4bac06c2a0 2024-04-28T13:32:58.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.761+0000 7f4bb1e00640 1 -- 172.21.15.196:0/360426386 >> v1:172.21.15.203:6789/0 conn(0x7f4bac1400c0 legacy=0x7f4bac0d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:58.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.761+0000 7f4bb1e00640 1 -- 172.21.15.196:0/360426386 >> v1:172.21.15.196:6789/0 conn(0x7f4bac06c2a0 legacy=0x7f4bac073320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:58.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.761+0000 7f4bb1e00640 1 -- 172.21.15.196:0/360426386 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bac0d0c80 con 0x7f4bac073a30 2024-04-28T13:32:58.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.762+0000 7f4bb1e00640 1 --2- 172.21.15.196:0/360426386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bac073a30 0x7f4bac06c700 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40097a0 tx=0x7f4ba402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584f672da8c1be7b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:58.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.763+0000 7f4babe00640 1 -- 172.21.15.196:0/360426386 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4037070 con 0x7f4bac073a30 2024-04-28T13:32:58.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.763+0000 7f4babe00640 1 -- 172.21.15.196:0/360426386 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba4032440 con 0x7f4bac073a30 2024-04-28T13:32:58.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.763+0000 7f4babe00640 1 -- 172.21.15.196:0/360426386 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4032810 con 0x7f4bac073a30 2024-04-28T13:32:58.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.763+0000 7f4bb3e00640 1 -- 172.21.15.196:0/360426386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bac073a30 msgr2=0x7f4bac06c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:58.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.763+0000 7f4bb3e00640 1 --2- 172.21.15.196:0/360426386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bac073a30 0x7f4bac06c700 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40097a0 tx=0x7f4ba402a230 comp rx=0 tx=0).stop 2024-04-28T13:32:58.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.763+0000 7f4bb3e00640 1 -- 172.21.15.196:0/360426386 shutdown_connections 2024-04-28T13:32:58.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.763+0000 7f4bb3e00640 1 --2- 172.21.15.196:0/360426386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bac073a30 0x7f4bac06c700 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:58.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.763+0000 7f4bb3e00640 1 -- 172.21.15.196:0/360426386 >> 172.21.15.196:0/360426386 conn(0x7f4bac0d1e90 msgr2=0x7f4bac0d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:58.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.763+0000 7f4bb3e00640 1 -- 172.21.15.196:0/360426386 shutdown_connections 2024-04-28T13:32:58.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.763+0000 7f4bb3e00640 1 -- 172.21.15.196:0/360426386 wait complete. 2024-04-28T13:32:58.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.765+0000 7f4bb3e00640 1 Processor -- start 2024-04-28T13:32:58.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.765+0000 7f4bb3e00640 1 -- start start 2024-04-28T13:32:58.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.766+0000 7f4bb3e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bac06c2a0 0x7f4bac070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:58.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.766+0000 7f4bb3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bac073a30 0x7f4bac13a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:58.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.766+0000 7f4bb3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bac1400c0 0x7f4bac13a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:58.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.766+0000 7f4bb3e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac0740d0 con 0x7f4bac073a30 2024-04-28T13:32:58.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.766+0000 7f4bb3e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac0d1240 con 0x7f4bac1400c0 2024-04-28T13:32:58.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.766+0000 7f4bb3e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4bac140650 con 0x7f4bac06c2a0 2024-04-28T13:32:58.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.766+0000 7f4bb0a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bac073a30 0x7f4bac13a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:58.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.767+0000 7f4bb0a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bac073a30 0x7f4bac13a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41956/0 (socket says 172.21.15.196:41956) 2024-04-28T13:32:58.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.767+0000 7f4bb0a00640 1 -- 172.21.15.196:0/1075740760 learned_addr learned my addr 172.21.15.196:0/1075740760 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:58.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.767+0000 7f4bb1400640 1 --2- 172.21.15.196:0/1075740760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bac06c2a0 0x7f4bac070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:58.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.767+0000 7f4bb1e00640 1 --2- 172.21.15.196:0/1075740760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bac1400c0 0x7f4bac13a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:58.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.767+0000 7f4bb0a00640 1 -- 172.21.15.196:0/1075740760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bac06c2a0 msgr2=0x7f4bac070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:58.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.767+0000 7f4bb0a00640 1 --2- 172.21.15.196:0/1075740760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bac06c2a0 0x7f4bac070560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:58.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.767+0000 7f4bb0a00640 1 -- 172.21.15.196:0/1075740760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bac1400c0 msgr2=0x7f4bac13a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:58.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.767+0000 7f4bb0a00640 1 --2- 172.21.15.196:0/1075740760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bac1400c0 0x7f4bac13a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:58.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.767+0000 7f4bb0a00640 1 -- 172.21.15.196:0/1075740760 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bac154bb0 con 0x7f4bac073a30 2024-04-28T13:32:58.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.767+0000 7f4bb0a00640 1 --2- 172.21.15.196:0/1075740760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bac073a30 0x7f4bac13a460 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800cbc0 tx=0x7f4b9800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:58.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.768+0000 7f4ba9600640 1 -- 172.21.15.196:0/1075740760 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98017070 con 0x7f4bac073a30 2024-04-28T13:32:58.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.768+0000 7f4ba9600640 1 -- 172.21.15.196:0/1075740760 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b9800ad40 con 0x7f4bac073a30 2024-04-28T13:32:58.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.768+0000 7f4ba9600640 1 -- 172.21.15.196:0/1075740760 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98012600 con 0x7f4bac073a30 2024-04-28T13:32:58.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.768+0000 7f4bb3e00640 1 -- 172.21.15.196:0/1075740760 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bac154e90 con 0x7f4bac073a30 2024-04-28T13:32:58.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.768+0000 7f4bb3e00640 1 -- 172.21.15.196:0/1075740760 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bac155970 con 0x7f4bac073a30 2024-04-28T13:32:58.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.769+0000 7f4bb3e00640 1 -- 172.21.15.196:0/1075740760 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bac075ba0 con 0x7f4bac073a30 2024-04-28T13:32:58.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.770+0000 7f4ba9600640 1 -- 172.21.15.196:0/1075740760 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4b9801a070 con 0x7f4bac073a30 2024-04-28T13:32:58.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.771+0000 7f4ba9600640 1 --2- 172.21.15.196:0/1075740760 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4b8c06c9b0 0x7f4b8c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:58.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.771+0000 7f4ba9600640 1 -- 172.21.15.196:0/1075740760 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f4b9808ba90 con 0x7f4bac073a30 2024-04-28T13:32:58.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.772+0000 7f4bb1400640 1 --2- 172.21.15.196:0/1075740760 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4b8c06c9b0 0x7f4b8c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:58.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.773+0000 7f4bb1400640 1 --2- 172.21.15.196:0/1075740760 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4b8c06c9b0 0x7f4b8c06ee70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c001500 tx=0x7f4b9c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:58.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:58.775+0000 7f4ba9600640 1 -- 172.21.15.196:0/1075740760 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4b98059720 con 0x7f4bac073a30 2024-04-28T13:32:59.018 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.018+0000 7f4bb3e00640 1 -- 172.21.15.196:0/1075740760 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f4bac069cf0 con 0x7f4bac073a30 2024-04-28T13:32:59.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.534+0000 7f4ba9600640 1 -- 172.21.15.196:0/1075740760 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v18) ==== 162+0+0 (secure 0 0 0) 0x7f4b9805da20 con 0x7f4bac073a30 2024-04-28T13:32:59.535 INFO:teuthology.orchestra.run.smithi196.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-28T13:32:59.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.537+0000 7f4bb3e00640 1 -- 172.21.15.196:0/1075740760 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4b8c06c9b0 msgr2=0x7f4b8c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:59.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.537+0000 7f4bb3e00640 1 --2- 172.21.15.196:0/1075740760 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4b8c06c9b0 0x7f4b8c06ee70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c001500 tx=0x7f4b9c009040 comp rx=0 tx=0).stop 2024-04-28T13:32:59.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.538+0000 7f4bb3e00640 1 -- 172.21.15.196:0/1075740760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bac073a30 msgr2=0x7f4bac13a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:59.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.538+0000 7f4bb3e00640 1 --2- 172.21.15.196:0/1075740760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bac073a30 0x7f4bac13a460 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800cbc0 tx=0x7f4b9800a430 comp rx=0 tx=0).stop 2024-04-28T13:32:59.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.538+0000 7f4bb3e00640 1 -- 172.21.15.196:0/1075740760 shutdown_connections 2024-04-28T13:32:59.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.538+0000 7f4bb3e00640 1 --2- 172.21.15.196:0/1075740760 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4b8c06c9b0 0x7f4b8c06ee70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:59.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.538+0000 7f4bb3e00640 1 --2- 172.21.15.196:0/1075740760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bac1400c0 0x7f4bac13a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:59.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.538+0000 7f4bb3e00640 1 --2- 172.21.15.196:0/1075740760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bac073a30 0x7f4bac13a460 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:59.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.538+0000 7f4bb3e00640 1 --2- 172.21.15.196:0/1075740760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bac06c2a0 0x7f4bac070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:59.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.538+0000 7f4bb3e00640 1 -- 172.21.15.196:0/1075740760 >> 172.21.15.196:0/1075740760 conn(0x7f4bac0d1e90 msgr2=0x7f4bac073150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:59.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.538+0000 7f4bb3e00640 1 -- 172.21.15.196:0/1075740760 shutdown_connections 2024-04-28T13:32:59.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.538+0000 7f4bb3e00640 1 -- 172.21.15.196:0/1075740760 wait complete. 2024-04-28T13:32:59.559 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:32:59.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.752+0000 7fcae9e00640 1 Processor -- start 2024-04-28T13:32:59.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.752+0000 7fcae9e00640 1 -- start start 2024-04-28T13:32:59.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.754+0000 7fcae9e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae41469f0 0x7fcae40ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.754+0000 7fcae9e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcae40cd420 0x7fcae40d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.754+0000 7fcae9e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcae40d1240 0x7fcae406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.754+0000 7fcae9e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcae40b5960 con 0x7fcae40cd420 2024-04-28T13:32:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.754+0000 7fcae9e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fcae4161ac0 con 0x7fcae40d1240 2024-04-28T13:32:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.754+0000 7fcae9e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcae4160f40 con 0x7fcae41469f0 2024-04-28T13:32:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.754+0000 7fcae2000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcae40cd420 0x7fcae40d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.754+0000 7fcae2000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcae40cd420 0x7fcae40d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41966/0 (socket says 172.21.15.196:41966) 2024-04-28T13:32:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.754+0000 7fcae2000640 1 -- 172.21.15.196:0/2799895036 learned_addr learned my addr 172.21.15.196:0/2799895036 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.755+0000 7fcae3400640 1 --2- 172.21.15.196:0/2799895036 >> v2:172.21.15.203:3300/0 conn(0x7fcae40d1240 0x7fcae406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.755+0000 7fcae2a00640 1 --2- 172.21.15.196:0/2799895036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae41469f0 0x7fcae40ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.755+0000 7fcae2000640 1 -- 172.21.15.196:0/2799895036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae41469f0 msgr2=0x7fcae40ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:59.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.755+0000 7fcae2000640 1 --2- 172.21.15.196:0/2799895036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae41469f0 0x7fcae40ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:59.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.755+0000 7fcae2000640 1 -- 172.21.15.196:0/2799895036 >> v2:172.21.15.203:3300/0 conn(0x7fcae40d1240 msgr2=0x7fcae406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:32:59.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.755+0000 7fcae2000640 1 --2- 172.21.15.196:0/2799895036 >> v2:172.21.15.203:3300/0 conn(0x7fcae40d1240 0x7fcae406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:59.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.755+0000 7fcae2000640 1 -- 172.21.15.196:0/2799895036 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae4161220 con 0x7fcae40cd420 2024-04-28T13:32:59.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.756+0000 7fcae2000640 1 --2- 172.21.15.196:0/2799895036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcae40cd420 0x7fcae40d0d00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcae40d1de0 tx=0x7fcad002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf05d9e33639a0a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:59.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.757+0000 7fcae1600640 1 -- 172.21.15.196:0/2799895036 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0037070 con 0x7fcae40cd420 2024-04-28T13:32:59.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.757+0000 7fcae1600640 1 -- 172.21.15.196:0/2799895036 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad0032440 con 0x7fcae40cd420 2024-04-28T13:32:59.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.757+0000 7fcae1600640 1 -- 172.21.15.196:0/2799895036 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0032810 con 0x7fcae40cd420 2024-04-28T13:32:59.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.757+0000 7fcae9e00640 1 -- 172.21.15.196:0/2799895036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcae40cd420 msgr2=0x7fcae40d0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:59.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.757+0000 7fcae9e00640 1 --2- 172.21.15.196:0/2799895036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcae40cd420 0x7fcae40d0d00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcae40d1de0 tx=0x7fcad002a1e0 comp rx=0 tx=0).stop 2024-04-28T13:32:59.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.757+0000 7fcae9e00640 1 -- 172.21.15.196:0/2799895036 shutdown_connections 2024-04-28T13:32:59.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.757+0000 7fcae9e00640 1 --2- 172.21.15.196:0/2799895036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcae40cd420 0x7fcae40d0d00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:59.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.757+0000 7fcae9e00640 1 --2- 172.21.15.196:0/2799895036 >> v2:172.21.15.203:3300/0 conn(0x7fcae40d1240 0x7fcae406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:59.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.757+0000 7fcae9e00640 1 --2- 172.21.15.196:0/2799895036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae41469f0 0x7fcae40ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:59.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.757+0000 7fcae9e00640 1 -- 172.21.15.196:0/2799895036 >> 172.21.15.196:0/2799895036 conn(0x7fcae40cd830 msgr2=0x7fcae40cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:32:59.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.758+0000 7fcae9e00640 1 -- 172.21.15.196:0/2799895036 shutdown_connections 2024-04-28T13:32:59.758 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.758+0000 7fcae9e00640 1 -- 172.21.15.196:0/2799895036 wait complete. 2024-04-28T13:32:59.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.759+0000 7fcae9e00640 1 Processor -- start 2024-04-28T13:32:59.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.759+0000 7fcae9e00640 1 -- start start 2024-04-28T13:32:59.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.760+0000 7fcae9e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae40cd420 0x7fcae40b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:59.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.760+0000 7fcae9e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae40d1240 0x7fcae415ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:59.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.760+0000 7fcae9e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcae41469f0 0x7fcae4139720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:59.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.760+0000 7fcae9e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcae41617e0 con 0x7fcae41469f0 2024-04-28T13:32:59.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.760+0000 7fcae9e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcae40d1650 con 0x7fcae40d1240 2024-04-28T13:32:59.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.760+0000 7fcae9e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcae4072020 con 0x7fcae40cd420 2024-04-28T13:32:59.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.761+0000 7fcae3400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcae41469f0 0x7fcae4139720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:59.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.761+0000 7fcae3400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcae41469f0 0x7fcae4139720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41970/0 (socket says 172.21.15.196:41970) 2024-04-28T13:32:59.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.761+0000 7fcae3400640 1 -- 172.21.15.196:0/1073348631 learned_addr learned my addr 172.21.15.196:0/1073348631 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:32:59.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.761+0000 7fcae2000640 1 --2- 172.21.15.196:0/1073348631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae40d1240 0x7fcae415ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:59.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.761+0000 7fcae3400640 1 -- 172.21.15.196:0/1073348631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae40cd420 msgr2=0x7fcae40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:59.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.761+0000 7fcae2a00640 1 --2- 172.21.15.196:0/1073348631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae40cd420 0x7fcae40b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:59.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.761+0000 7fcae3400640 1 --2- 172.21.15.196:0/1073348631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae40cd420 0x7fcae40b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:59.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.761+0000 7fcae3400640 1 -- 172.21.15.196:0/1073348631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae40d1240 msgr2=0x7fcae415ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:32:59.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.761+0000 7fcae3400640 1 --2- 172.21.15.196:0/1073348631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae40d1240 0x7fcae415ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:32:59.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.761+0000 7fcae3400640 1 -- 172.21.15.196:0/1073348631 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae40b1890 con 0x7fcae41469f0 2024-04-28T13:32:59.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.761+0000 7fcae2000640 1 --2- 172.21.15.196:0/1073348631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae40d1240 0x7fcae415ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:32:59.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.761+0000 7fcae3400640 1 --2- 172.21.15.196:0/1073348631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcae41469f0 0x7fcae4139720 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcadc007d90 tx=0x7fcadc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:59.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.762+0000 7fcadb400640 1 -- 172.21.15.196:0/1073348631 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc014420 con 0x7fcae41469f0 2024-04-28T13:32:59.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.762+0000 7fcadb400640 1 -- 172.21.15.196:0/1073348631 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcadc019070 con 0x7fcae41469f0 2024-04-28T13:32:59.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.762+0000 7fcadb400640 1 -- 172.21.15.196:0/1073348631 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc014820 con 0x7fcae41469f0 2024-04-28T13:32:59.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.762+0000 7fcae9e00640 1 -- 172.21.15.196:0/1073348631 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae40b1b70 con 0x7fcae41469f0 2024-04-28T13:32:59.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.762+0000 7fcae9e00640 1 -- 172.21.15.196:0/1073348631 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae40b2130 con 0x7fcae41469f0 2024-04-28T13:32:59.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.763+0000 7fcae9e00640 1 -- 172.21.15.196:0/1073348631 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae406a290 con 0x7fcae41469f0 2024-04-28T13:32:59.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.764+0000 7fcadb400640 1 -- 172.21.15.196:0/1073348631 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fcadc0149c0 con 0x7fcae41469f0 2024-04-28T13:32:59.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.765+0000 7fcadb400640 1 --2- 172.21.15.196:0/1073348631 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcac406c9b0 0x7fcac406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:32:59.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.765+0000 7fcadb400640 1 -- 172.21.15.196:0/1073348631 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fcadc08c440 con 0x7fcae41469f0 2024-04-28T13:32:59.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.765+0000 7fcae2a00640 1 --2- 172.21.15.196:0/1073348631 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcac406c9b0 0x7fcac406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:32:59.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.767+0000 7fcae2a00640 1 --2- 172.21.15.196:0/1073348631 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcac406c9b0 0x7fcac406ee70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcad4006fd0 tx=0x7fcad4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:32:59.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:32:59.769+0000 7fcadb400640 1 -- 172.21.15.196:0/1073348631 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcadc05a7b0 con 0x7fcae41469f0 2024-04-28T13:33:00.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:00.010+0000 7fcae9e00640 1 -- 172.21.15.196:0/1073348631 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcae415d040 con 0x7fcae41469f0 2024-04-28T13:33:00.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:00.011+0000 7fcadb400640 1 -- 172.21.15.196:0/1073348631 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5184 (secure 0 0 0) 0x7fcadc05a0b0 con 0x7fcae41469f0 2024-04-28T13:33:00.011 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:00.011 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/931076983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":931076983},{"type":"v1","addr":"172.21.15.196:6834","nonce":931076983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1114474485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1114474485},{"type":"v1","addr":"172.21.15.196:6835","nonce":1114474485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2070123371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2070123371},{"type":"v1","addr":"172.21.15.203:6837","nonce":2070123371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:32:31.669472+0000","modified":"2024-04-28T13:32:59.540186+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4694},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4694":{"gid":4694,"name":"b","rank":0,"incarnation":19,"state":"up:replay","state_seq":1,"addr":"172.21.15.203:6836/1998460400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1998460400},{"type":"v1","addr":"172.21.15.203:6836","nonce":1998460400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-28T13:33:00.011 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 19 2024-04-28T13:33:00.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:00.014+0000 7fcae9e00640 1 -- 172.21.15.196:0/1073348631 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcac406c9b0 msgr2=0x7fcac406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:00.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:00.014+0000 7fcae9e00640 1 --2- 172.21.15.196:0/1073348631 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcac406c9b0 0x7fcac406ee70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcad4006fd0 tx=0x7fcad4008040 comp rx=0 tx=0).stop 2024-04-28T13:33:00.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:00.014+0000 7fcae9e00640 1 -- 172.21.15.196:0/1073348631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcae41469f0 msgr2=0x7fcae4139720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:00.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:00.014+0000 7fcae9e00640 1 --2- 172.21.15.196:0/1073348631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcae41469f0 0x7fcae4139720 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcadc007d90 tx=0x7fcadc00e4b0 comp rx=0 tx=0).stop 2024-04-28T13:33:00.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:00.014+0000 7fcae9e00640 1 -- 172.21.15.196:0/1073348631 shutdown_connections 2024-04-28T13:33:00.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:00.014+0000 7fcae9e00640 1 --2- 172.21.15.196:0/1073348631 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcac406c9b0 0x7fcac406ee70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:00.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:00.014+0000 7fcae9e00640 1 --2- 172.21.15.196:0/1073348631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcae41469f0 0x7fcae4139720 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:00.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:00.014+0000 7fcae9e00640 1 --2- 172.21.15.196:0/1073348631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae40d1240 0x7fcae415ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:00.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:00.014+0000 7fcae9e00640 1 --2- 172.21.15.196:0/1073348631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae40cd420 0x7fcae40b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:00.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:00.014+0000 7fcae9e00640 1 -- 172.21.15.196:0/1073348631 >> 172.21.15.196:0/1073348631 conn(0x7fcae40cd830 msgr2=0x7fcae4072230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:00.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:00.014+0000 7fcae9e00640 1 -- 172.21.15.196:0/1073348631 shutdown_connections 2024-04-28T13:33:00.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:00.014+0000 7fcae9e00640 1 -- 172.21.15.196:0/1073348631 wait complete. 2024-04-28T13:33:00.435 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-28T13:32:31.669472+0000', 'modified': '2024-04-28T13:32:59.540186+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 34, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4694}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4694': {'gid': 4694, 'name': 'b', 'rank': 0, 'incarnation': 19, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.203:6836/1998460400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6835', 'nonce': 1998460400}, {'type': 'v1', 'addr': '172.21.15.203:6836', 'nonce': 1998460400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-28T13:33:00.436 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4694:up:replay 2024-04-28T13:33:01.436 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:33:01.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.645+0000 7f5ff3200640 1 Processor -- start 2024-04-28T13:33:01.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.645+0000 7f5ff3200640 1 -- start start 2024-04-28T13:33:01.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.647+0000 7f5ff3200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5fec1469f0 0x7f5fec0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:01.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.647+0000 7f5ff3200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fec0cd420 0x7f5fec0d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:01.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.647+0000 7f5ff3200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5fec0d1240 0x7f5fec06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:01.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.647+0000 7f5ff3200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5fec0b5960 con 0x7f5fec0d1240 2024-04-28T13:33:01.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.647+0000 7f5ff3200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f5fec161ac0 con 0x7f5fec1469f0 2024-04-28T13:33:01.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.647+0000 7f5ff3200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5fec160f40 con 0x7f5fec0cd420 2024-04-28T13:33:01.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.647+0000 7f5ff1200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5fec0d1240 0x7f5fec06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:01.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.647+0000 7f5ff1200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5fec0d1240 0x7f5fec06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41984/0 (socket says 172.21.15.196:41984) 2024-04-28T13:33:01.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.647+0000 7f5ff1200640 1 -- 172.21.15.196:0/2196618118 learned_addr learned my addr 172.21.15.196:0/2196618118 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:01.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.648+0000 7f5feb400640 1 --2- 172.21.15.196:0/2196618118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fec0cd420 0x7f5fec0d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:01.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.648+0000 7f5febe00640 1 --2- 172.21.15.196:0/2196618118 >> v2:172.21.15.203:3300/0 conn(0x7f5fec1469f0 0x7f5fec0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:01.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.648+0000 7f5ff1200640 1 -- 172.21.15.196:0/2196618118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fec0cd420 msgr2=0x7f5fec0d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:01.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.648+0000 7f5ff1200640 1 --2- 172.21.15.196:0/2196618118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fec0cd420 0x7f5fec0d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:01.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.648+0000 7f5ff1200640 1 -- 172.21.15.196:0/2196618118 >> v2:172.21.15.203:3300/0 conn(0x7f5fec1469f0 msgr2=0x7f5fec0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:01.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.648+0000 7f5ff1200640 1 --2- 172.21.15.196:0/2196618118 >> v2:172.21.15.203:3300/0 conn(0x7f5fec1469f0 0x7f5fec0ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:01.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.648+0000 7f5ff1200640 1 -- 172.21.15.196:0/2196618118 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fec161220 con 0x7f5fec0d1240 2024-04-28T13:33:01.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.648+0000 7f5febe00640 1 --2- 172.21.15.196:0/2196618118 >> v2:172.21.15.203:3300/0 conn(0x7f5fec1469f0 0x7f5fec0ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:01.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.649+0000 7f5ff1200640 1 --2- 172.21.15.196:0/2196618118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fec0d1240 0x7f5fec06e330 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5fec074680 tx=0x7f5fe402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caec8ec43572f14b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:01.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.649+0000 7f5feaa00640 1 -- 172.21.15.196:0/2196618118 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4036070 con 0x7f5fec0d1240 2024-04-28T13:33:01.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.649+0000 7f5feaa00640 1 -- 172.21.15.196:0/2196618118 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe4031600 con 0x7f5fec0d1240 2024-04-28T13:33:01.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.649+0000 7f5feaa00640 1 -- 172.21.15.196:0/2196618118 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40319d0 con 0x7f5fec0d1240 2024-04-28T13:33:01.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.650+0000 7f5ff3200640 1 -- 172.21.15.196:0/2196618118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fec0d1240 msgr2=0x7f5fec06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:01.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.650+0000 7f5ff3200640 1 --2- 172.21.15.196:0/2196618118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fec0d1240 0x7f5fec06e330 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5fec074680 tx=0x7f5fe402a6a0 comp rx=0 tx=0).stop 2024-04-28T13:33:01.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.650+0000 7f5ff3200640 1 -- 172.21.15.196:0/2196618118 shutdown_connections 2024-04-28T13:33:01.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.650+0000 7f5ff3200640 1 --2- 172.21.15.196:0/2196618118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fec0d1240 0x7f5fec06e330 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:01.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.650+0000 7f5ff3200640 1 --2- 172.21.15.196:0/2196618118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fec0cd420 0x7f5fec0d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:01.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.650+0000 7f5ff3200640 1 --2- 172.21.15.196:0/2196618118 >> v2:172.21.15.203:3300/0 conn(0x7f5fec1469f0 0x7f5fec0ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:01.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.650+0000 7f5ff3200640 1 -- 172.21.15.196:0/2196618118 >> 172.21.15.196:0/2196618118 conn(0x7f5fec0cd830 msgr2=0x7f5fec0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:01.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.650+0000 7f5ff3200640 1 -- 172.21.15.196:0/2196618118 shutdown_connections 2024-04-28T13:33:01.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.650+0000 7f5ff3200640 1 -- 172.21.15.196:0/2196618118 wait complete. 2024-04-28T13:33:01.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.652+0000 7f5ff3200640 1 Processor -- start 2024-04-28T13:33:01.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.652+0000 7f5ff3200640 1 -- start start 2024-04-28T13:33:01.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.653+0000 7f5ff3200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fec0cd420 0x7f5fec0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:01.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.653+0000 7f5ff3200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fec0d1240 0x7f5fec15e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:01.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.653+0000 7f5ff3200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fec1469f0 0x7f5fec1392d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:01.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.653+0000 7f5ff3200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec072020 con 0x7f5fec0cd420 2024-04-28T13:33:01.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.653+0000 7f5ff3200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1617e0 con 0x7f5fec1469f0 2024-04-28T13:33:01.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.653+0000 7f5ff3200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5fec160b80 con 0x7f5fec0d1240 2024-04-28T13:33:01.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.653+0000 7f5febe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fec0cd420 0x7f5fec0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:01.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.653+0000 7f5febe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fec0cd420 0x7f5fec0b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41994/0 (socket says 172.21.15.196:41994) 2024-04-28T13:33:01.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.653+0000 7f5febe00640 1 -- 172.21.15.196:0/3272811359 learned_addr learned my addr 172.21.15.196:0/3272811359 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:01.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.654+0000 7f5feb400640 1 --2- 172.21.15.196:0/3272811359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fec0d1240 0x7f5fec15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:01.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.654+0000 7f5ff1200640 1 --2- 172.21.15.196:0/3272811359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fec1469f0 0x7f5fec1392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:01.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.654+0000 7f5febe00640 1 -- 172.21.15.196:0/3272811359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fec0d1240 msgr2=0x7f5fec15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:01.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.654+0000 7f5febe00640 1 --2- 172.21.15.196:0/3272811359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fec0d1240 0x7f5fec15e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:01.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.654+0000 7f5febe00640 1 -- 172.21.15.196:0/3272811359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fec1469f0 msgr2=0x7f5fec1392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:01.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.654+0000 7f5febe00640 1 --2- 172.21.15.196:0/3272811359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fec1469f0 0x7f5fec1392d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:01.656 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.654+0000 7f5febe00640 1 -- 172.21.15.196:0/3272811359 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fec0b1ad0 con 0x7f5fec0cd420 2024-04-28T13:33:01.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.654+0000 7f5ff1200640 1 --2- 172.21.15.196:0/3272811359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fec1469f0 0x7f5fec1392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:01.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.654+0000 7f5febe00640 1 --2- 172.21.15.196:0/3272811359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fec0cd420 0x7f5fec0b25d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c7f0 tx=0x7f5fdc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:01.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.655+0000 7f5fe3e00640 1 -- 172.21.15.196:0/3272811359 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc015070 con 0x7f5fec0cd420 2024-04-28T13:33:01.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.655+0000 7f5fe3e00640 1 -- 172.21.15.196:0/3272811359 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fdc007a30 con 0x7f5fec0cd420 2024-04-28T13:33:01.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.655+0000 7f5ff3200640 1 -- 172.21.15.196:0/3272811359 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fec139810 con 0x7f5fec0cd420 2024-04-28T13:33:01.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.655+0000 7f5fe3e00640 1 -- 172.21.15.196:0/3272811359 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc007e00 con 0x7f5fec0cd420 2024-04-28T13:33:01.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.655+0000 7f5ff3200640 1 -- 172.21.15.196:0/3272811359 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fec15d950 con 0x7f5fec0cd420 2024-04-28T13:33:01.657 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.656+0000 7f5ff3200640 1 -- 172.21.15.196:0/3272811359 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fec06a290 con 0x7f5fec0cd420 2024-04-28T13:33:01.658 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.657+0000 7f5fe3e00640 1 -- 172.21.15.196:0/3272811359 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f5fdc018070 con 0x7f5fec0cd420 2024-04-28T13:33:01.658 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.658+0000 7f5fe3e00640 1 --2- 172.21.15.196:0/3272811359 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5fcc06c980 0x7f5fcc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:01.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.658+0000 7f5fe3e00640 1 -- 172.21.15.196:0/3272811359 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f5fdc08bbd0 con 0x7f5fec0cd420 2024-04-28T13:33:01.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.658+0000 7f5feb400640 1 --2- 172.21.15.196:0/3272811359 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5fcc06c980 0x7f5fcc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:01.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.660+0000 7f5feb400640 1 --2- 172.21.15.196:0/3272811359 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5fcc06c980 0x7f5fcc06ee40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5fec139e60 tx=0x7f5fd8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:01.663 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.662+0000 7f5fe3e00640 1 -- 172.21.15.196:0/3272811359 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5fdc059f30 con 0x7f5fec0cd420 2024-04-28T13:33:01.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.908+0000 7f5ff3200640 1 -- 172.21.15.196:0/3272811359 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fec0b6380 con 0x7f5fec0cd420 2024-04-28T13:33:01.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.909+0000 7f5fe3e00640 1 -- 172.21.15.196:0/3272811359 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5184 (secure 0 0 0) 0x7f5fdc059830 con 0x7f5fec0cd420 2024-04-28T13:33:01.910 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:01.911 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/931076983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":931076983},{"type":"v1","addr":"172.21.15.196:6834","nonce":931076983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1114474485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1114474485},{"type":"v1","addr":"172.21.15.196:6835","nonce":1114474485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2070123371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2070123371},{"type":"v1","addr":"172.21.15.203:6837","nonce":2070123371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:32:31.669472+0000","modified":"2024-04-28T13:33:00.574339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4694},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4694":{"gid":4694,"name":"b","rank":0,"incarnation":19,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.203:6836/1998460400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1998460400},{"type":"v1","addr":"172.21.15.203:6836","nonce":1998460400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-28T13:33:01.911 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 21 2024-04-28T13:33:01.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.912+0000 7f5ff3200640 1 -- 172.21.15.196:0/3272811359 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5fcc06c980 msgr2=0x7f5fcc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:01.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.912+0000 7f5ff3200640 1 --2- 172.21.15.196:0/3272811359 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5fcc06c980 0x7f5fcc06ee40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5fec139e60 tx=0x7f5fd8008040 comp rx=0 tx=0).stop 2024-04-28T13:33:01.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.912+0000 7f5ff3200640 1 -- 172.21.15.196:0/3272811359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fec0cd420 msgr2=0x7f5fec0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:01.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.912+0000 7f5ff3200640 1 --2- 172.21.15.196:0/3272811359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fec0cd420 0x7f5fec0b25d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c7f0 tx=0x7f5fdc00cbf0 comp rx=0 tx=0).stop 2024-04-28T13:33:01.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.913+0000 7f5ff3200640 1 -- 172.21.15.196:0/3272811359 shutdown_connections 2024-04-28T13:33:01.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.913+0000 7f5ff3200640 1 --2- 172.21.15.196:0/3272811359 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5fcc06c980 0x7f5fcc06ee40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:01.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.913+0000 7f5ff3200640 1 --2- 172.21.15.196:0/3272811359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fec1469f0 0x7f5fec1392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:01.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.913+0000 7f5ff3200640 1 --2- 172.21.15.196:0/3272811359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fec0d1240 0x7f5fec15e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:01.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.913+0000 7f5ff3200640 1 --2- 172.21.15.196:0/3272811359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fec0cd420 0x7f5fec0b25d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:01.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.913+0000 7f5ff3200640 1 -- 172.21.15.196:0/3272811359 >> 172.21.15.196:0/3272811359 conn(0x7f5fec0cd830 msgr2=0x7f5fec15c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:01.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.913+0000 7f5ff3200640 1 -- 172.21.15.196:0/3272811359 shutdown_connections 2024-04-28T13:33:01.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:01.913+0000 7f5ff3200640 1 -- 172.21.15.196:0/3272811359 wait complete. 2024-04-28T13:33:02.336 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-28T13:32:31.669472+0000', 'modified': '2024-04-28T13:33:00.574339+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 34, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4694}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4694': {'gid': 4694, 'name': 'b', 'rank': 0, 'incarnation': 19, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.203:6836/1998460400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6835', 'nonce': 1998460400}, {'type': 'v1', 'addr': '172.21.15.203:6836', 'nonce': 1998460400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-28T13:33:02.336 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4694:up:rejoin 2024-04-28T13:33:03.337 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:33:03.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.537+0000 7fa816200640 1 Processor -- start 2024-04-28T13:33:03.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.538+0000 7fa816200640 1 -- start start 2024-04-28T13:33:03.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.539+0000 7fa816200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa8101629c0 0x7fa8100d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:03.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.539+0000 7fa816200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa810163bf0 0x7fa810163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:03.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.539+0000 7fa816200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa810162da0 con 0x7fa810163570 2024-04-28T13:33:03.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.539+0000 7fa816200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa8100b6190 con 0x7fa810163bf0 2024-04-28T13:33:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.539+0000 7fa816200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa8100b6470 con 0x7fa8101629c0 2024-04-28T13:33:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.539+0000 7fa80fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa810163bf0 0x7fa810163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.539+0000 7fa80fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa810163bf0 0x7fa810163030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54426/0 (socket says 172.21.15.196:54426) 2024-04-28T13:33:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.540+0000 7fa80fe00640 1 -- 172.21.15.196:0/2007695057 learned_addr learned my addr 172.21.15.196:0/2007695057 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.540+0000 7fa814e00640 1 --2- 172.21.15.196:0/2007695057 >> v2:172.21.15.203:3300/0 conn(0x7fa8101629c0 0x7fa8100d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.540+0000 7fa80fe00640 1 -- 172.21.15.196:0/2007695057 >> v2:172.21.15.203:3300/0 conn(0x7fa8101629c0 msgr2=0x7fa8100d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.540+0000 7fa80fe00640 1 --2- 172.21.15.196:0/2007695057 >> v2:172.21.15.203:3300/0 conn(0x7fa8101629c0 0x7fa8100d3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.540+0000 7fa80fe00640 1 -- 172.21.15.196:0/2007695057 >> v1:172.21.15.203:6789/0 conn(0x7fa810163570 legacy=0x7fa8100b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:03.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.540+0000 7fa80fe00640 1 -- 172.21.15.196:0/2007695057 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8100b6750 con 0x7fa810163bf0 2024-04-28T13:33:03.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.541+0000 7fa80fe00640 1 --2- 172.21.15.196:0/2007695057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa810163bf0 0x7fa810163030 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa81008f8a0 tx=0x7fa80002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a6cd0cbf285553 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:03.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.541+0000 7fa80f400640 1 -- 172.21.15.196:0/2007695057 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800036070 con 0x7fa810163bf0 2024-04-28T13:33:03.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.541+0000 7fa80f400640 1 -- 172.21.15.196:0/2007695057 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa800031600 con 0x7fa810163bf0 2024-04-28T13:33:03.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.541+0000 7fa80f400640 1 -- 172.21.15.196:0/2007695057 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8000319d0 con 0x7fa810163bf0 2024-04-28T13:33:03.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.542+0000 7fa816200640 1 -- 172.21.15.196:0/2007695057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa810163bf0 msgr2=0x7fa810163030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:03.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.542+0000 7fa816200640 1 --2- 172.21.15.196:0/2007695057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa810163bf0 0x7fa810163030 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa81008f8a0 tx=0x7fa80002a6f0 comp rx=0 tx=0).stop 2024-04-28T13:33:03.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.542+0000 7fa816200640 1 -- 172.21.15.196:0/2007695057 shutdown_connections 2024-04-28T13:33:03.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.542+0000 7fa816200640 1 --2- 172.21.15.196:0/2007695057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa810163bf0 0x7fa810163030 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:03.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.542+0000 7fa816200640 1 --2- 172.21.15.196:0/2007695057 >> v2:172.21.15.203:3300/0 conn(0x7fa8101629c0 0x7fa8100d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:03.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.542+0000 7fa816200640 1 -- 172.21.15.196:0/2007695057 >> 172.21.15.196:0/2007695057 conn(0x7fa81009e570 msgr2=0x7fa8100d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:03.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.542+0000 7fa816200640 1 -- 172.21.15.196:0/2007695057 shutdown_connections 2024-04-28T13:33:03.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.542+0000 7fa816200640 1 -- 172.21.15.196:0/2007695057 wait complete. 2024-04-28T13:33:03.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.544+0000 7fa816200640 1 Processor -- start 2024-04-28T13:33:03.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.544+0000 7fa816200640 1 -- start start 2024-04-28T13:33:03.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.545+0000 7fa816200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8101629c0 0x7fa81015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:03.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.545+0000 7fa816200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa810163570 0x7fa810155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:03.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.545+0000 7fa816200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa810163bf0 0x7fa810155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:03.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.545+0000 7fa816200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8100d3c50 con 0x7fa810163bf0 2024-04-28T13:33:03.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.545+0000 7fa816200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa8100b1e60 con 0x7fa8101629c0 2024-04-28T13:33:03.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.545+0000 7fa816200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa8100d3dd0 con 0x7fa810163570 2024-04-28T13:33:03.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.545+0000 7fa815800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa810163bf0 0x7fa810155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:03.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.545+0000 7fa815800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa810163bf0 0x7fa810155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54428/0 (socket says 172.21.15.196:54428) 2024-04-28T13:33:03.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.545+0000 7fa815800640 1 -- 172.21.15.196:0/315775951 learned_addr learned my addr 172.21.15.196:0/315775951 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:03.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.546+0000 7fa814e00640 1 --2- 172.21.15.196:0/315775951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8101629c0 0x7fa81015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:03.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.546+0000 7fa80fe00640 1 --2- 172.21.15.196:0/315775951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa810163570 0x7fa810155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:03.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.546+0000 7fa815800640 1 -- 172.21.15.196:0/315775951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa810163570 msgr2=0x7fa810155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:03.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.546+0000 7fa815800640 1 --2- 172.21.15.196:0/315775951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa810163570 0x7fa810155780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:03.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.546+0000 7fa815800640 1 -- 172.21.15.196:0/315775951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8101629c0 msgr2=0x7fa81015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:03.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.546+0000 7fa815800640 1 --2- 172.21.15.196:0/315775951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8101629c0 0x7fa81015d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:03.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.546+0000 7fa815800640 1 -- 172.21.15.196:0/315775951 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8100d22e0 con 0x7fa810163bf0 2024-04-28T13:33:03.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.546+0000 7fa814e00640 1 --2- 172.21.15.196:0/315775951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8101629c0 0x7fa81015d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:03.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.546+0000 7fa815800640 1 --2- 172.21.15.196:0/315775951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa810163bf0 0x7fa810155cc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa80800c410 tx=0x7fa80800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:03.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.546+0000 7fa80cc00640 1 -- 172.21.15.196:0/315775951 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808011790 con 0x7fa810163bf0 2024-04-28T13:33:03.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.546+0000 7fa80cc00640 1 -- 172.21.15.196:0/315775951 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa808017070 con 0x7fa810163bf0 2024-04-28T13:33:03.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.546+0000 7fa80cc00640 1 -- 172.21.15.196:0/315775951 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808011bb0 con 0x7fa810163bf0 2024-04-28T13:33:03.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.546+0000 7fa816200640 1 -- 172.21.15.196:0/315775951 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8100d2000 con 0x7fa810163bf0 2024-04-28T13:33:03.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.547+0000 7fa816200640 1 -- 172.21.15.196:0/315775951 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8101f5490 con 0x7fa810163bf0 2024-04-28T13:33:03.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.548+0000 7fa816200640 1 -- 172.21.15.196:0/315775951 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa810163360 con 0x7fa810163bf0 2024-04-28T13:33:03.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.549+0000 7fa80cc00640 1 -- 172.21.15.196:0/315775951 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa80801c460 con 0x7fa810163bf0 2024-04-28T13:33:03.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.550+0000 7fa80cc00640 1 --2- 172.21.15.196:0/315775951 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa7ec06c980 0x7fa7ec06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:03.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.550+0000 7fa80cc00640 1 -- 172.21.15.196:0/315775951 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fa80808b840 con 0x7fa810163bf0 2024-04-28T13:33:03.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.550+0000 7fa814e00640 1 --2- 172.21.15.196:0/315775951 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa7ec06c980 0x7fa7ec06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:03.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.552+0000 7fa814e00640 1 --2- 172.21.15.196:0/315775951 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa7ec06c980 0x7fa7ec06ee40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8006fd0 tx=0x7fa7f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:03.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.554+0000 7fa80cc00640 1 -- 172.21.15.196:0/315775951 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa810163360 con 0x7fa810163bf0 2024-04-28T13:33:03.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.797+0000 7fa816200640 1 -- 172.21.15.196:0/315775951 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa81006a290 con 0x7fa810163bf0 2024-04-28T13:33:03.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.798+0000 7fa80cc00640 1 -- 172.21.15.196:0/315775951 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5191 (secure 0 0 0) 0x7fa808021020 con 0x7fa810163bf0 2024-04-28T13:33:03.799 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:03.800 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/931076983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":931076983},{"type":"v1","addr":"172.21.15.196:6834","nonce":931076983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1114474485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1114474485},{"type":"v1","addr":"172.21.15.196:6835","nonce":1114474485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2070123371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2070123371},{"type":"v1","addr":"172.21.15.203:6837","nonce":2070123371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:32:31.669472+0000","modified":"2024-04-28T13:33:02.550473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4694},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4694":{"gid":4694,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6836/1998460400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1998460400},{"type":"v1","addr":"172.21.15.203:6836","nonce":1998460400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4694,"qdb_cluster":[4694]},"id":2}]} 2024-04-28T13:33:03.800 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 22 2024-04-28T13:33:03.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.801+0000 7fa816200640 1 -- 172.21.15.196:0/315775951 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa7ec06c980 msgr2=0x7fa7ec06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:03.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.801+0000 7fa816200640 1 --2- 172.21.15.196:0/315775951 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa7ec06c980 0x7fa7ec06ee40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8006fd0 tx=0x7fa7f8008040 comp rx=0 tx=0).stop 2024-04-28T13:33:03.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.801+0000 7fa816200640 1 -- 172.21.15.196:0/315775951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa810163bf0 msgr2=0x7fa810155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:03.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.801+0000 7fa816200640 1 --2- 172.21.15.196:0/315775951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa810163bf0 0x7fa810155cc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa80800c410 tx=0x7fa80800c8d0 comp rx=0 tx=0).stop 2024-04-28T13:33:03.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.801+0000 7fa816200640 1 -- 172.21.15.196:0/315775951 shutdown_connections 2024-04-28T13:33:03.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.801+0000 7fa816200640 1 --2- 172.21.15.196:0/315775951 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa7ec06c980 0x7fa7ec06ee40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:03.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.801+0000 7fa816200640 1 --2- 172.21.15.196:0/315775951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa810163bf0 0x7fa810155cc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:03.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.801+0000 7fa816200640 1 --2- 172.21.15.196:0/315775951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa810163570 0x7fa810155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:03.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.801+0000 7fa816200640 1 --2- 172.21.15.196:0/315775951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8101629c0 0x7fa81015d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:03.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.801+0000 7fa816200640 1 -- 172.21.15.196:0/315775951 >> 172.21.15.196:0/315775951 conn(0x7fa81009e570 msgr2=0x7fa8100b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:03.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.801+0000 7fa816200640 1 -- 172.21.15.196:0/315775951 shutdown_connections 2024-04-28T13:33:03.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:03.801+0000 7fa816200640 1 -- 172.21.15.196:0/315775951 wait complete. 2024-04-28T13:33:04.223 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-28T13:32:31.669472+0000', 'modified': '2024-04-28T13:33:02.550473+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 34, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4694}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4694': {'gid': 4694, 'name': 'b', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.203:6836/1998460400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6835', 'nonce': 1998460400}, {'type': 'v1', 'addr': '172.21.15.203:6836', 'nonce': 1998460400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4694, 'qdb_cluster': [4694]} 2024-04-28T13:33:04.223 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-28T13:33:04.224 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-28T13:33:04.433 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.432+0000 7f4277200640 1 Processor -- start 2024-04-28T13:33:04.433 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.433+0000 7f4277200640 1 -- start start 2024-04-28T13:33:04.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.434+0000 7f4277200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4278163bf0 0x7f4278163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.434+0000 7f4277200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4278163570 0x7f42780b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.434+0000 7f4277200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4278162da0 con 0x7f42781629c0 2024-04-28T13:33:04.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.434+0000 7f4277200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f42780d3e20 con 0x7f4278163570 2024-04-28T13:33:04.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.434+0000 7f4277200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f42780b6190 con 0x7f4278163bf0 2024-04-28T13:33:04.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.434+0000 7f4275e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f42781629c0 0x7f42780d3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:33770/0 (socket says 172.21.15.196:33770) 2024-04-28T13:33:04.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.434+0000 7f4275e00640 1 -- 172.21.15.196:0/1164627757 learned_addr learned my addr 172.21.15.196:0/1164627757 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:04.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.435+0000 7f4276800640 1 --2- 172.21.15.196:0/1164627757 >> v2:172.21.15.196:3300/0 conn(0x7f4278163570 0x7f42780b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.435+0000 7f4275400640 1 --2- 172.21.15.196:0/1164627757 >> v2:172.21.15.203:3300/0 conn(0x7f4278163bf0 0x7f4278163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.435+0000 7f4274a00640 1 -- 172.21.15.196:0/1164627757 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2721030649 0 0) 0x7f4278162da0 con 0x7f42781629c0 2024-04-28T13:33:04.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.435+0000 7f4274a00640 1 -- 172.21.15.196:0/1164627757 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4258003520 con 0x7f42781629c0 2024-04-28T13:33:04.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.435+0000 7f4276800640 1 -- 172.21.15.196:0/1164627757 >> v2:172.21.15.203:3300/0 conn(0x7f4278163bf0 msgr2=0x7f4278163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:04.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.435+0000 7f4276800640 1 --2- 172.21.15.196:0/1164627757 >> v2:172.21.15.203:3300/0 conn(0x7f4278163bf0 0x7f4278163030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.435+0000 7f4276800640 1 -- 172.21.15.196:0/1164627757 >> v1:172.21.15.196:6789/0 conn(0x7f42781629c0 legacy=0x7f42780d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:04.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.435+0000 7f4276800640 1 -- 172.21.15.196:0/1164627757 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42780b6470 con 0x7f4278163570 2024-04-28T13:33:04.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.436+0000 7f4276800640 1 --2- 172.21.15.196:0/1164627757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4278163570 0x7f42780b2380 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f427808f920 tx=0x7f427002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f45ecb2d25873a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:04.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.436+0000 7f4274a00640 1 -- 172.21.15.196:0/1164627757 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270037070 con 0x7f4278163570 2024-04-28T13:33:04.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.436+0000 7f4274a00640 1 -- 172.21.15.196:0/1164627757 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4270032440 con 0x7f4278163570 2024-04-28T13:33:04.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.436+0000 7f4274a00640 1 -- 172.21.15.196:0/1164627757 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270032810 con 0x7f4278163570 2024-04-28T13:33:04.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.437+0000 7f4277200640 1 -- 172.21.15.196:0/1164627757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4278163570 msgr2=0x7f42780b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:04.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.437+0000 7f4277200640 1 --2- 172.21.15.196:0/1164627757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4278163570 0x7f42780b2380 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f427808f920 tx=0x7f427002a1a0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.437+0000 7f4277200640 1 -- 172.21.15.196:0/1164627757 shutdown_connections 2024-04-28T13:33:04.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.437+0000 7f4277200640 1 --2- 172.21.15.196:0/1164627757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4278163570 0x7f42780b2380 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.437+0000 7f4277200640 1 --2- 172.21.15.196:0/1164627757 >> v2:172.21.15.203:3300/0 conn(0x7f4278163bf0 0x7f4278163030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.437+0000 7f4277200640 1 -- 172.21.15.196:0/1164627757 >> 172.21.15.196:0/1164627757 conn(0x7f427809e570 msgr2=0x7f42780d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:04.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.437+0000 7f4277200640 1 -- 172.21.15.196:0/1164627757 shutdown_connections 2024-04-28T13:33:04.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.437+0000 7f4277200640 1 -- 172.21.15.196:0/1164627757 wait complete. 2024-04-28T13:33:04.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.439+0000 7f4277200640 1 Processor -- start 2024-04-28T13:33:04.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.439+0000 7f4277200640 1 -- start start 2024-04-28T13:33:04.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.440+0000 7f4277200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42781629c0 0x7f42780ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.440+0000 7f4277200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4278163570 0x7f427815b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.440+0000 7f4277200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4278163bf0 0x7f427806d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.440+0000 7f4277200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f42780b1e60 con 0x7f42781629c0 2024-04-28T13:33:04.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.440+0000 7f4277200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f42780b6a30 con 0x7f4278163570 2024-04-28T13:33:04.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.440+0000 7f4277200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f42780b28c0 con 0x7f4278163bf0 2024-04-28T13:33:04.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.440+0000 7f4275e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42781629c0 0x7f42780ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.440+0000 7f4275e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42781629c0 0x7f42780ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54452/0 (socket says 172.21.15.196:54452) 2024-04-28T13:33:04.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.440+0000 7f4275e00640 1 -- 172.21.15.196:0/2106789834 learned_addr learned my addr 172.21.15.196:0/2106789834 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:04.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.441+0000 7f4275400640 1 --2- 172.21.15.196:0/2106789834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4278163570 0x7f427815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.441+0000 7f4275e00640 1 -- 172.21.15.196:0/2106789834 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4278163bf0 msgr2=0x7f427806d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:04.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.441+0000 7f4275e00640 1 --2- 172.21.15.196:0/2106789834 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4278163bf0 0x7f427806d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.441+0000 7f4275e00640 1 -- 172.21.15.196:0/2106789834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4278163570 msgr2=0x7f427815b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:04.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.441+0000 7f4275e00640 1 --2- 172.21.15.196:0/2106789834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4278163570 0x7f427815b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.441+0000 7f4275e00640 1 -- 172.21.15.196:0/2106789834 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42780ce5b0 con 0x7f42781629c0 2024-04-28T13:33:04.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.441+0000 7f4275e00640 1 --2- 172.21.15.196:0/2106789834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42781629c0 0x7f42780ce070 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f426800c460 tx=0x7f426800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:04.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.441+0000 7f426e000640 1 -- 172.21.15.196:0/2106789834 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268017070 con 0x7f42781629c0 2024-04-28T13:33:04.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.441+0000 7f426e000640 1 -- 172.21.15.196:0/2106789834 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4268005ce0 con 0x7f42781629c0 2024-04-28T13:33:04.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.441+0000 7f426e000640 1 -- 172.21.15.196:0/2106789834 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268007290 con 0x7f42781629c0 2024-04-28T13:33:04.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.441+0000 7f4277200640 1 -- 172.21.15.196:0/2106789834 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42780d2630 con 0x7f42781629c0 2024-04-28T13:33:04.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.442+0000 7f4277200640 1 -- 172.21.15.196:0/2106789834 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42780cd870 con 0x7f42781629c0 2024-04-28T13:33:04.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.444+0000 7f426e000640 1 -- 172.21.15.196:0/2106789834 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f426801a070 con 0x7f42781629c0 2024-04-28T13:33:04.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.445+0000 7f426e000640 1 --2- 172.21.15.196:0/2106789834 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f425406c980 0x7f425406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.445+0000 7f426e000640 1 -- 172.21.15.196:0/2106789834 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f426808afe0 con 0x7f42781629c0 2024-04-28T13:33:04.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.445+0000 7f4275400640 1 --2- 172.21.15.196:0/2106789834 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f425406c980 0x7f425406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.445+0000 7f4280004740 1 Processor -- start 2024-04-28T13:33:04.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.446+0000 7f4280004740 1 -- start start 2024-04-28T13:33:04.448 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.446+0000 7f4280004740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637549177b0 0x563754919910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.446+0000 7f4280004740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754953e80 0x563754917b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.446+0000 7f4280004740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563754942270 0x5637549426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.446+0000 7f4280004740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56375487b600 con 0x563754953e80 2024-04-28T13:33:04.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.446+0000 7f4280004740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563754818100 con 0x5637549177b0 2024-04-28T13:33:04.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.446+0000 7f4280004740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5637548775c0 con 0x563754942270 2024-04-28T13:33:04.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.447+0000 7f4275400640 1 --2- 172.21.15.196:0/2106789834 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f425406c980 0x7f425406ee40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f427815e6a0 tx=0x7f426000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:04.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.447+0000 7f4275400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754953e80 0x563754917b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.447+0000 7f4275400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754953e80 0x563754917b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54466/0 (socket says 172.21.15.196:54466) 2024-04-28T13:33:04.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.447+0000 7f4275400640 1 -- 172.21.15.196:0/3531254293 learned_addr learned my addr 172.21.15.196:0/3531254293 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:04.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.447+0000 7f4275e00640 1 --2- 172.21.15.196:0/3531254293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637549177b0 0x563754919910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.447+0000 7f4275400640 1 -- 172.21.15.196:0/3531254293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563754942270 msgr2=0x5637549426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:04.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.447+0000 7f4276800640 1 --2- 172.21.15.196:0/3531254293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563754942270 0x5637549426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.447+0000 7f4275400640 1 --2- 172.21.15.196:0/3531254293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563754942270 0x5637549426f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.447+0000 7f4275400640 1 -- 172.21.15.196:0/3531254293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637549177b0 msgr2=0x563754919910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:04.449 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.447+0000 7f4275400640 1 --2- 172.21.15.196:0/3531254293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637549177b0 0x563754919910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.447+0000 7f4275400640 1 -- 172.21.15.196:0/3531254293 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56375458d2d0 con 0x563754953e80 2024-04-28T13:33:04.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.447+0000 7f4275e00640 1 --2- 172.21.15.196:0/3531254293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637549177b0 0x563754919910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:04.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.447+0000 7f4275400640 1 --2- 172.21.15.196:0/3531254293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754953e80 0x563754917b90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4260016710 tx=0x7f4260012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8980f77dbdef1b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:04.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.448+0000 7f4266a00640 1 -- 172.21.15.196:0/3531254293 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260026070 con 0x563754953e80 2024-04-28T13:33:04.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.448+0000 7f4266a00640 1 -- 172.21.15.196:0/3531254293 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4260014920 con 0x563754953e80 2024-04-28T13:33:04.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.448+0000 7f4266a00640 1 -- 172.21.15.196:0/3531254293 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260014cd0 con 0x563754953e80 2024-04-28T13:33:04.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.448+0000 7f4280004740 1 -- 172.21.15.196:0/3531254293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754953e80 msgr2=0x563754917b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:04.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.448+0000 7f4280004740 1 --2- 172.21.15.196:0/3531254293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754953e80 0x563754917b90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4260016710 tx=0x7f4260012000 comp rx=0 tx=0).stop 2024-04-28T13:33:04.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.449+0000 7f4280004740 1 -- 172.21.15.196:0/3531254293 shutdown_connections 2024-04-28T13:33:04.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.449+0000 7f4280004740 1 --2- 172.21.15.196:0/3531254293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563754942270 0x5637549426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.449+0000 7f4280004740 1 --2- 172.21.15.196:0/3531254293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754953e80 0x563754917b90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.449+0000 7f4280004740 1 --2- 172.21.15.196:0/3531254293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5637549177b0 0x563754919910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.449+0000 7f4280004740 1 -- 172.21.15.196:0/3531254293 >> 172.21.15.196:0/3531254293 conn(0x5637548abd10 msgr2=0x56375495d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:04.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.449+0000 7f4280004740 1 -- 172.21.15.196:0/3531254293 shutdown_connections 2024-04-28T13:33:04.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.449+0000 7f4280004740 1 -- 172.21.15.196:0/3531254293 wait complete. 2024-04-28T13:33:04.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.449+0000 7f4280004740 1 -- start start 2024-04-28T13:33:04.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.450+0000 7f4280004740 1 Processor -- start 2024-04-28T13:33:04.454 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4280004740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5637549177b0 0x563754944850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4280004740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563754942270 0x56375493ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4280004740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754953e80 0x56375493f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4280004740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563754802f70 con 0x563754953e80 2024-04-28T13:33:04.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4280004740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56375493fdc0 con 0x563754942270 2024-04-28T13:33:04.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4280004740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x56375495dcd0 con 0x5637549177b0 2024-04-28T13:33:04.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4280004740 10 client.0 _ll_register_callbacks cb 0x563754871480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T13:33:04.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4276800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754953e80 0x56375493f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4276800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754953e80 0x56375493f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54474/0 (socket says 172.21.15.196:54474) 2024-04-28T13:33:04.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4276800640 1 -- 172.21.15.196:0/3324397683 learned_addr learned my addr 172.21.15.196:0/3324397683 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:04.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4275400640 1 --2- 172.21.15.196:0/3324397683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563754942270 0x56375493ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4276800640 1 -- 172.21.15.196:0/3324397683 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5637549177b0 msgr2=0x563754944850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:04.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4276800640 1 --2- 172.21.15.196:0/3324397683 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5637549177b0 0x563754944850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4276800640 1 -- 172.21.15.196:0/3324397683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563754942270 msgr2=0x56375493ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:04.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4276800640 1 --2- 172.21.15.196:0/3324397683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563754942270 0x56375493ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4276800640 1 -- 172.21.15.196:0/3324397683 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637546576b0 con 0x563754953e80 2024-04-28T13:33:04.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.451+0000 7f4275400640 1 --2- 172.21.15.196:0/3324397683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563754942270 0x56375493ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:04.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.452+0000 7f4276800640 1 --2- 172.21.15.196:0/3324397683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754953e80 0x56375493f4c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f427002a1e0 tx=0x7f42700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b713c22188100b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:04.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.452+0000 7f425f400640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-04-28T13:33:04.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.452+0000 7f425f400640 1 -- 172.21.15.196:0/3324397683 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270041070 con 0x563754953e80 2024-04-28T13:33:04.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.452+0000 7f425f400640 1 -- 172.21.15.196:0/3324397683 <== mon.0 v2:172.21.15.196:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f427003c870 con 0x563754953e80 2024-04-28T13:33:04.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.452+0000 7f4280004740 1 -- 172.21.15.196:0/3324397683 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4260014920 con 0x563754953e80 2024-04-28T13:33:04.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.452+0000 7f425f400640 1 -- 172.21.15.196:0/3324397683 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f427004c050 con 0x563754953e80 2024-04-28T13:33:04.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.452+0000 7f4280004740 10 client.5463 fetch_fsmap learned FSMap version 22 2024-04-28T13:33:04.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.452+0000 7f4280004740 1 -- 172.21.15.196:0/3324397683 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x5637549454a0 con 0x563754953e80 2024-04-28T13:33:04.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.453+0000 7f425f400640 1 -- 172.21.15.196:0/3324397683 <== mon.0 v2:172.21.15.196:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4260014920 con 0x563754953e80 2024-04-28T13:33:04.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.453+0000 7f425f400640 1 -- 172.21.15.196:0/3324397683 <== mon.0 v2:172.21.15.196:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7f427004f020 con 0x563754953e80 2024-04-28T13:33:04.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.453+0000 7f4280004740 10 client.5463 fetch_fsmap finished waiting for FSMap version 22 2024-04-28T13:33:04.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.453+0000 7f4280004740 10 client.5463 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T13:33:04.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.453+0000 7f4280004740 20 client.5463 populate_metadata read hostname 'smithi196' 2024-04-28T13:33:04.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.453+0000 7f4280004740 1 --2- 172.21.15.196:0/3324397683 >> [v2:172.21.15.203:6835/1998460400,v1:172.21.15.203:6836/1998460400] conn(0x563754944d90 0x563754942af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.453+0000 7f4280004740 4 client.5463 mds_command: new command op to 4694 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T13:33:04.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.453+0000 7f4280004740 1 -- 172.21.15.196:0/3324397683 --> [v2:172.21.15.203:6835/1998460400,v1:172.21.15.203:6836/1998460400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563754889210 con 0x563754944d90 2024-04-28T13:33:04.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.453+0000 7f4275e00640 1 --2- 172.21.15.196:0/3324397683 >> [v2:172.21.15.203:6835/1998460400,v1:172.21.15.203:6836/1998460400] conn(0x563754944d90 0x563754942af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.455+0000 7f4275e00640 1 --2- 172.21.15.196:0/3324397683 >> [v2:172.21.15.203:6835/1998460400,v1:172.21.15.203:6836/1998460400] conn(0x563754944d90 0x563754942af0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad812ded41da4cab server_cookie=bbf22823be8cd462 in_seq=0 out_seq=0 2024-04-28T13:33:04.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.455+0000 7f425f400640 10 client.5463 ms_handle_connect on v2:172.21.15.203:6835/1998460400 2024-04-28T13:33:04.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.457+0000 7f425f400640 1 -- 172.21.15.196:0/3324397683 <== mds.0 v2:172.21.15.203:6835/1998460400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563754889210 con 0x563754944d90 2024-04-28T13:33:04.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.457+0000 7f425f400640 10 client.5463 handle_command_reply: tid=0 2024-04-28T13:33:04.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.457+0000 7f4280004740 1 client.5463 shutdown 2024-04-28T13:33:04.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.457+0000 7f4280004740 1 -- 172.21.15.196:0/3324397683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754953e80 msgr2=0x56375493f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:04.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.457+0000 7f4280004740 1 --2- 172.21.15.196:0/3324397683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754953e80 0x56375493f4c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f427002a1e0 tx=0x7f42700046c0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.457+0000 7f4280004740 1 -- 172.21.15.196:0/3324397683 shutdown_connections 2024-04-28T13:33:04.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.457+0000 7f4280004740 1 --2- 172.21.15.196:0/3324397683 >> [v2:172.21.15.203:6835/1998460400,v1:172.21.15.203:6836/1998460400] conn(0x563754944d90 0x563754942af0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.457+0000 7f4280004740 1 --2- 172.21.15.196:0/3324397683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754953e80 0x56375493f4c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.457+0000 7f4280004740 1 --2- 172.21.15.196:0/3324397683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563754942270 0x56375493ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.457+0000 7f4280004740 1 --2- 172.21.15.196:0/3324397683 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5637549177b0 0x563754944850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.457+0000 7f425f400640 0 client.5463 ms_handle_reset on v2:172.21.15.203:6835/1998460400 2024-04-28T13:33:04.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.457+0000 7f4280004740 1 -- 172.21.15.196:0/3324397683 >> 172.21.15.196:0/3324397683 conn(0x5637548abd10 msgr2=0x5637549401c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:04.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.457+0000 7f4280004740 1 -- 172.21.15.196:0/3324397683 shutdown_connections 2024-04-28T13:33:04.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.457+0000 7f4280004740 1 -- 172.21.15.196:0/3324397683 wait complete. 2024-04-28T13:33:04.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.458+0000 7f4280004740 20 client.5463 trim_cache size 0 max 16384 2024-04-28T13:33:04.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.477+0000 7f4280004740 1 Processor -- start 2024-04-28T13:33:04.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.477+0000 7f4280004740 1 -- start start 2024-04-28T13:33:04.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.478+0000 7f4280004740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5637549177b0 0x563754943110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.478+0000 7f4280004740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754942270 0x56375490e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.478+0000 7f4280004740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563754953e80 0x5637549458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.478+0000 7f4280004740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4278201510 con 0x563754942270 2024-04-28T13:33:04.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.478+0000 7f4280004740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f42780b9800 con 0x563754953e80 2024-04-28T13:33:04.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.478+0000 7f4280004740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f42780d0480 con 0x5637549177b0 2024-04-28T13:33:04.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.478+0000 7f4275400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754942270 0x56375490e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.478+0000 7f4275400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754942270 0x56375490e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54478/0 (socket says 172.21.15.196:54478) 2024-04-28T13:33:04.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.478+0000 7f4275400640 1 -- 172.21.15.196:0/3802950149 learned_addr learned my addr 172.21.15.196:0/3802950149 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:04.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.479+0000 7f4275e00640 1 --2- 172.21.15.196:0/3802950149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5637549177b0 0x563754943110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.479+0000 7f4276800640 1 --2- 172.21.15.196:0/3802950149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563754953e80 0x5637549458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.479+0000 7f4275400640 1 -- 172.21.15.196:0/3802950149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5637549177b0 msgr2=0x563754943110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:04.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.479+0000 7f4275400640 1 --2- 172.21.15.196:0/3802950149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5637549177b0 0x563754943110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.479+0000 7f4275400640 1 -- 172.21.15.196:0/3802950149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563754953e80 msgr2=0x5637549458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:04.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.479+0000 7f4275400640 1 --2- 172.21.15.196:0/3802950149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563754953e80 0x5637549458c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.479+0000 7f4275400640 1 -- 172.21.15.196:0/3802950149 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637549570d0 con 0x563754942270 2024-04-28T13:33:04.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.479+0000 7f4275400640 1 --2- 172.21.15.196:0/3802950149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754942270 0x56375490e0d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4260016fa0 tx=0x7f4260014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a6c9f84593a5f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:04.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.479+0000 7f4266000640 1 -- 172.21.15.196:0/3802950149 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260031070 con 0x563754942270 2024-04-28T13:33:04.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.479+0000 7f4266000640 1 -- 172.21.15.196:0/3802950149 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4260014a00 con 0x563754942270 2024-04-28T13:33:04.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.479+0000 7f4266000640 1 -- 172.21.15.196:0/3802950149 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260026040 con 0x563754942270 2024-04-28T13:33:04.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.480+0000 7f4280004740 1 -- 172.21.15.196:0/3802950149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754942270 msgr2=0x56375490e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:04.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.480+0000 7f4280004740 1 --2- 172.21.15.196:0/3802950149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754942270 0x56375490e0d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4260016fa0 tx=0x7f4260014000 comp rx=0 tx=0).stop 2024-04-28T13:33:04.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.480+0000 7f4280004740 1 -- 172.21.15.196:0/3802950149 shutdown_connections 2024-04-28T13:33:04.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.480+0000 7f4280004740 1 --2- 172.21.15.196:0/3802950149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563754953e80 0x5637549458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.480+0000 7f4280004740 1 --2- 172.21.15.196:0/3802950149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563754942270 0x56375490e0d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.480+0000 7f4280004740 1 --2- 172.21.15.196:0/3802950149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5637549177b0 0x563754943110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.480+0000 7f4280004740 1 -- 172.21.15.196:0/3802950149 >> 172.21.15.196:0/3802950149 conn(0x5637548abd10 msgr2=0x56375493c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:04.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.480+0000 7f4280004740 1 -- 172.21.15.196:0/3802950149 shutdown_connections 2024-04-28T13:33:04.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.480+0000 7f4280004740 1 -- 172.21.15.196:0/3802950149 wait complete. 2024-04-28T13:33:04.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.481+0000 7f4280004740 1 -- start start 2024-04-28T13:33:04.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.481+0000 7f4280004740 1 Processor -- start 2024-04-28T13:33:04.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.482+0000 7f4280004740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637549177b0 0x563754952c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.482+0000 7f4280004740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563754953e80 0x563754955a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.482+0000 7f4280004740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563754980a10 0x563754944190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.482+0000 7f4280004740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56375493d0d0 con 0x5637549177b0 2024-04-28T13:33:04.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.482+0000 7f4280004740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f42780b91b0 con 0x563754953e80 2024-04-28T13:33:04.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.482+0000 7f4280004740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x563754954790 con 0x563754980a10 2024-04-28T13:33:04.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.482+0000 7f4280004740 10 client.0 _ll_register_callbacks cb 0x7f4270035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T13:33:04.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.482+0000 7f4275e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637549177b0 0x563754952c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.483+0000 7f4275e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637549177b0 0x563754952c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54484/0 (socket says 172.21.15.196:54484) 2024-04-28T13:33:04.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.483+0000 7f4275e00640 1 -- 172.21.15.196:0/2920531926 learned_addr learned my addr 172.21.15.196:0/2920531926 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:04.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.483+0000 7f4275e00640 1 -- 172.21.15.196:0/2920531926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563754980a10 msgr2=0x563754944190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:04.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.483+0000 7f4275e00640 1 --2- 172.21.15.196:0/2920531926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563754980a10 0x563754944190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.483+0000 7f4275e00640 1 -- 172.21.15.196:0/2920531926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563754953e80 msgr2=0x563754955a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:04.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.483+0000 7f4275e00640 1 --2- 172.21.15.196:0/2920531926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563754953e80 0x563754955a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.483+0000 7f4275e00640 1 -- 172.21.15.196:0/2920531926 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56375493d9e0 con 0x5637549177b0 2024-04-28T13:33:04.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.483+0000 7f4275e00640 1 --2- 172.21.15.196:0/2920531926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637549177b0 0x563754952c00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f42680550a0 tx=0x7f426805d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3aeaaed31f76498 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:04.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.483+0000 7f425f400640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-04-28T13:33:04.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.483+0000 7f425f400640 1 -- 172.21.15.196:0/2920531926 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268056f90 con 0x5637549177b0 2024-04-28T13:33:04.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.483+0000 7f425f400640 1 -- 172.21.15.196:0/2920531926 <== mon.0 v2:172.21.15.196:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f4268089550 con 0x5637549177b0 2024-04-28T13:33:04.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.483+0000 7f4280004740 1 -- 172.21.15.196:0/2920531926 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4260014a00 con 0x5637549177b0 2024-04-28T13:33:04.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.484+0000 7f425f400640 1 -- 172.21.15.196:0/2920531926 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f42680890d0 con 0x5637549177b0 2024-04-28T13:33:04.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.484+0000 7f4280004740 10 client.5469 fetch_fsmap learned FSMap version 22 2024-04-28T13:33:04.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.484+0000 7f4280004740 1 -- 172.21.15.196:0/2920531926 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x563754955fb0 con 0x5637549177b0 2024-04-28T13:33:04.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.484+0000 7f425f400640 1 -- 172.21.15.196:0/2920531926 <== mon.0 v2:172.21.15.196:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4260014a00 con 0x5637549177b0 2024-04-28T13:33:04.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.484+0000 7f425f400640 1 -- 172.21.15.196:0/2920531926 <== mon.0 v2:172.21.15.196:3300/0 5 ==== fsmap(e 22) ==== 3494+0+0 (secure 0 0 0) 0x7f4268055300 con 0x5637549177b0 2024-04-28T13:33:04.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.484+0000 7f4280004740 10 client.5469 fetch_fsmap finished waiting for FSMap version 22 2024-04-28T13:33:04.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.484+0000 7f4280004740 10 client.5469 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T13:33:04.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.484+0000 7f4280004740 20 client.5469 populate_metadata read hostname 'smithi196' 2024-04-28T13:33:04.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.484+0000 7f4280004740 1 --2- 172.21.15.196:0/2920531926 >> [v2:172.21.15.203:6835/1998460400,v1:172.21.15.203:6836/1998460400] conn(0x56375496bbe0 0x56375496bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:04.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.484+0000 7f4280004740 4 client.5469 mds_command: new command op to 4694 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T13:33:04.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.484+0000 7f4280004740 1 -- 172.21.15.196:0/2920531926 --> [v2:172.21.15.203:6835/1998460400,v1:172.21.15.203:6836/1998460400] -- command(tid 0: {"prefix": "status"}) -- 0x563754883080 con 0x56375496bbe0 2024-04-28T13:33:04.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.485+0000 7f4275400640 1 --2- 172.21.15.196:0/2920531926 >> [v2:172.21.15.203:6835/1998460400,v1:172.21.15.203:6836/1998460400] conn(0x56375496bbe0 0x56375496bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:04.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.486+0000 7f4275400640 1 --2- 172.21.15.196:0/2920531926 >> [v2:172.21.15.203:6835/1998460400,v1:172.21.15.203:6836/1998460400] conn(0x56375496bbe0 0x56375496bfc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d0c59a71773b4e7 server_cookie=283794d4113a2dd5 in_seq=0 out_seq=0 2024-04-28T13:33:04.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.486+0000 7f425f400640 10 client.5469 ms_handle_connect on v2:172.21.15.203:6835/1998460400 2024-04-28T13:33:04.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.486+0000 7f425f400640 1 -- 172.21.15.196:0/2920531926 <== mds.0 v2:172.21.15.203:6835/1998460400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x563754883080 con 0x56375496bbe0 2024-04-28T13:33:04.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.486+0000 7f425f400640 10 client.5469 handle_command_reply: tid=0 2024-04-28T13:33:04.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.486+0000 7f4280004740 1 client.5469 shutdown 2024-04-28T13:33:04.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.486+0000 7f4280004740 1 -- 172.21.15.196:0/2920531926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637549177b0 msgr2=0x563754952c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:04.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.486+0000 7f4280004740 1 --2- 172.21.15.196:0/2920531926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637549177b0 0x563754952c00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f42680550a0 tx=0x7f426805d040 comp rx=0 tx=0).stop 2024-04-28T13:33:04.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.486+0000 7f4280004740 1 -- 172.21.15.196:0/2920531926 shutdown_connections 2024-04-28T13:33:04.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.486+0000 7f4280004740 1 --2- 172.21.15.196:0/2920531926 >> [v2:172.21.15.203:6835/1998460400,v1:172.21.15.203:6836/1998460400] conn(0x56375496bbe0 0x56375496bfc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.486+0000 7f4280004740 1 --2- 172.21.15.196:0/2920531926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563754980a10 0x563754944190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.486+0000 7f4280004740 1 --2- 172.21.15.196:0/2920531926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563754953e80 0x563754955a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.486+0000 7f4280004740 1 --2- 172.21.15.196:0/2920531926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637549177b0 0x563754952c00 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.486+0000 7f425f400640 0 client.5469 ms_handle_reset on v2:172.21.15.203:6835/1998460400 2024-04-28T13:33:04.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.486+0000 7f4280004740 1 -- 172.21.15.196:0/2920531926 >> 172.21.15.196:0/2920531926 conn(0x5637548abd10 msgr2=0x5637549812d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:04.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.487+0000 7f4280004740 1 -- 172.21.15.196:0/2920531926 shutdown_connections 2024-04-28T13:33:04.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.487+0000 7f4280004740 1 -- 172.21.15.196:0/2920531926 wait complete. 2024-04-28T13:33:04.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.487+0000 7f4280004740 20 client.5469 trim_cache size 0 max 16384 2024-04-28T13:33:04.490 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:33:04.490 INFO:teuthology.orchestra.run.smithi196.stdout: "cluster_fsid": "270aea32-2e16-4a97-85f7-aeae39bc8c76", 2024-04-28T13:33:04.490 INFO:teuthology.orchestra.run.smithi196.stdout: "whoami": 0, 2024-04-28T13:33:04.490 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 4694, 2024-04-28T13:33:04.490 INFO:teuthology.orchestra.run.smithi196.stdout: "want_state": "up:active", 2024-04-28T13:33:04.490 INFO:teuthology.orchestra.run.smithi196.stdout: "state": "up:active", 2024-04-28T13:33:04.490 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs", 2024-04-28T13:33:04.491 INFO:teuthology.orchestra.run.smithi196.stdout: "rank_uptime": 4.935014389, 2024-04-28T13:33:04.491 INFO:teuthology.orchestra.run.smithi196.stdout: "mdsmap_epoch": 22, 2024-04-28T13:33:04.491 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_epoch": 34, 2024-04-28T13:33:04.491 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_epoch_barrier": 34, 2024-04-28T13:33:04.491 INFO:teuthology.orchestra.run.smithi196.stdout: "uptime": 5.8272455169999997 2024-04-28T13:33:04.491 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:33:04.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.487+0000 7f4277200640 1 -- 172.21.15.196:0/2106789834 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f425406c980 msgr2=0x7f425406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:04.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.487+0000 7f4277200640 1 --2- 172.21.15.196:0/2106789834 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f425406c980 0x7f425406ee40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f427815e6a0 tx=0x7f426000b040 comp rx=0 tx=0).stop 2024-04-28T13:33:04.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.487+0000 7f4277200640 1 -- 172.21.15.196:0/2106789834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42781629c0 msgr2=0x7f42780ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:04.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.487+0000 7f4277200640 1 --2- 172.21.15.196:0/2106789834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42781629c0 0x7f42780ce070 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f426800c460 tx=0x7f426800c920 comp rx=0 tx=0).stop 2024-04-28T13:33:04.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.487+0000 7f4277200640 1 -- 172.21.15.196:0/2106789834 shutdown_connections 2024-04-28T13:33:04.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.487+0000 7f4277200640 1 --2- 172.21.15.196:0/2106789834 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f425406c980 0x7f425406ee40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.487+0000 7f4277200640 1 --2- 172.21.15.196:0/2106789834 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4278163bf0 0x7f427806d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.487+0000 7f4277200640 1 --2- 172.21.15.196:0/2106789834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4278163570 0x7f427815b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.487+0000 7f4277200640 1 --2- 172.21.15.196:0/2106789834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42781629c0 0x7f42780ce070 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:04.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.487+0000 7f4277200640 1 -- 172.21.15.196:0/2106789834 >> 172.21.15.196:0/2106789834 conn(0x7f427809e570 msgr2=0x7f427806e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:04.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.488+0000 7f4277200640 1 -- 172.21.15.196:0/2106789834 shutdown_connections 2024-04-28T13:33:04.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:04.488+0000 7f4277200640 1 -- 172.21.15.196:0/2106789834 wait complete. 2024-04-28T13:33:04.506 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-28T13:33:04.506 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-28T13:33:04.614 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-28T13:33:04.728 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-28T13:33:04.728 INFO:tasks.cephfs.mount:Terminating background process 2024-04-28T13:33:04.728 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:33:04.728 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:33:04.785 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:33:04.785 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:33:04.836 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:33:05.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.033+0000 7f1f4f200640 1 Processor -- start 2024-04-28T13:33:05.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.033+0000 7f1f4f200640 1 -- start start 2024-04-28T13:33:05.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.034+0000 7f1f4f200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1f500b5ee0 0x7f1f500b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.035+0000 7f1f4f200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1f5009ec80 0x7f1f500d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.035+0000 7f1f4f200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f5015e380 con 0x7f1f500add70 2024-04-28T13:33:05.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.035+0000 7f1f4f200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1f500b4b20 con 0x7f1f5009ec80 2024-04-28T13:33:05.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.035+0000 7f1f4f200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f1f500b3f30 con 0x7f1f500b5ee0 2024-04-28T13:33:05.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.035+0000 7f1f45400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1f5009ec80 0x7f1f500d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.035+0000 7f1f45400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1f5009ec80 0x7f1f500d1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54486/0 (socket says 172.21.15.196:54486) 2024-04-28T13:33:05.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.035+0000 7f1f45400640 1 -- 172.21.15.196:0/1399012810 learned_addr learned my addr 172.21.15.196:0/1399012810 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:05.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.035+0000 7f1f4de00640 1 --2- 172.21.15.196:0/1399012810 >> v2:172.21.15.203:3300/0 conn(0x7f1f500b5ee0 0x7f1f500b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.036+0000 7f1f45400640 1 -- 172.21.15.196:0/1399012810 >> v2:172.21.15.203:3300/0 conn(0x7f1f500b5ee0 msgr2=0x7f1f500b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:05.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.036+0000 7f1f45400640 1 --2- 172.21.15.196:0/1399012810 >> v2:172.21.15.203:3300/0 conn(0x7f1f500b5ee0 0x7f1f500b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.036+0000 7f1f45400640 1 -- 172.21.15.196:0/1399012810 >> v1:172.21.15.203:6789/0 conn(0x7f1f500add70 legacy=0x7f1f5006df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:05.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.036+0000 7f1f45400640 1 -- 172.21.15.196:0/1399012810 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f500b41f0 con 0x7f1f5009ec80 2024-04-28T13:33:05.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.037+0000 7f1f45400640 1 --2- 172.21.15.196:0/1399012810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f5009ec80 0x7f1f500d1050 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1f5006eda0 tx=0x7f1f3802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7fe2c17a03f0a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:05.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.037+0000 7f1f4d400640 1 -- 172.21.15.196:0/1399012810 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38037070 con 0x7f1f5009ec80 2024-04-28T13:33:05.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.037+0000 7f1f4d400640 1 -- 172.21.15.196:0/1399012810 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f38032440 con 0x7f1f5009ec80 2024-04-28T13:33:05.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.037+0000 7f1f4d400640 1 -- 172.21.15.196:0/1399012810 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38032810 con 0x7f1f5009ec80 2024-04-28T13:33:05.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.038+0000 7f1f4f200640 1 -- 172.21.15.196:0/1399012810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f5009ec80 msgr2=0x7f1f500d1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:05.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.038+0000 7f1f4f200640 1 --2- 172.21.15.196:0/1399012810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f5009ec80 0x7f1f500d1050 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1f5006eda0 tx=0x7f1f3802a230 comp rx=0 tx=0).stop 2024-04-28T13:33:05.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.038+0000 7f1f4f200640 1 -- 172.21.15.196:0/1399012810 shutdown_connections 2024-04-28T13:33:05.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.038+0000 7f1f4f200640 1 --2- 172.21.15.196:0/1399012810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f5009ec80 0x7f1f500d1050 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.038+0000 7f1f4f200640 1 --2- 172.21.15.196:0/1399012810 >> v2:172.21.15.203:3300/0 conn(0x7f1f500b5ee0 0x7f1f500b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.038+0000 7f1f4f200640 1 -- 172.21.15.196:0/1399012810 >> 172.21.15.196:0/1399012810 conn(0x7f1f5006c6c0 msgr2=0x7f1f5006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:05.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.038+0000 7f1f4f200640 1 -- 172.21.15.196:0/1399012810 shutdown_connections 2024-04-28T13:33:05.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.038+0000 7f1f4f200640 1 -- 172.21.15.196:0/1399012810 wait complete. 2024-04-28T13:33:05.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.040+0000 7f1f4f200640 1 Processor -- start 2024-04-28T13:33:05.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.040+0000 7f1f4f200640 1 -- start start 2024-04-28T13:33:05.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.041+0000 7f1f4f200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f5009ec80 0x7f1f501f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.041+0000 7f1f4f200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f500add70 0x7f1f500b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.041+0000 7f1f4f200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f500b5ee0 0x7f1f5009ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.041+0000 7f1f4f200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f500b3c20 con 0x7f1f500add70 2024-04-28T13:33:05.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.041+0000 7f1f4f200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1f500b47b0 con 0x7f1f5009ec80 2024-04-28T13:33:05.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.041+0000 7f1f4f200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1f500d1590 con 0x7f1f500b5ee0 2024-04-28T13:33:05.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.041+0000 7f1f45400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f500add70 0x7f1f500b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.041+0000 7f1f45400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f500add70 0x7f1f500b1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54496/0 (socket says 172.21.15.196:54496) 2024-04-28T13:33:05.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.041+0000 7f1f45400640 1 -- 172.21.15.196:0/1023392555 learned_addr learned my addr 172.21.15.196:0/1023392555 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:05.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.041+0000 7f1f4de00640 1 --2- 172.21.15.196:0/1023392555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f5009ec80 0x7f1f501f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.041+0000 7f1f4e800640 1 --2- 172.21.15.196:0/1023392555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f500b5ee0 0x7f1f5009ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.041+0000 7f1f45400640 1 -- 172.21.15.196:0/1023392555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f500b5ee0 msgr2=0x7f1f5009ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:05.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.041+0000 7f1f45400640 1 --2- 172.21.15.196:0/1023392555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f500b5ee0 0x7f1f5009ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.042+0000 7f1f45400640 1 -- 172.21.15.196:0/1023392555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f5009ec80 msgr2=0x7f1f501f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:05.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.042+0000 7f1f45400640 1 --2- 172.21.15.196:0/1023392555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f5009ec80 0x7f1f501f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.042+0000 7f1f45400640 1 -- 172.21.15.196:0/1023392555 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f50150e00 con 0x7f1f500add70 2024-04-28T13:33:05.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.042+0000 7f1f4de00640 1 --2- 172.21.15.196:0/1023392555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f5009ec80 0x7f1f501f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:05.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.042+0000 7f1f45400640 1 --2- 172.21.15.196:0/1023392555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f500add70 0x7f1f500b1e90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1f38000c00 tx=0x7f1f380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:05.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.042+0000 7f1f46a00640 1 -- 172.21.15.196:0/1023392555 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38041020 con 0x7f1f500add70 2024-04-28T13:33:05.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.042+0000 7f1f46a00640 1 -- 172.21.15.196:0/1023392555 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f380306c0 con 0x7f1f500add70 2024-04-28T13:33:05.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.042+0000 7f1f46a00640 1 -- 172.21.15.196:0/1023392555 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38037040 con 0x7f1f500add70 2024-04-28T13:33:05.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.042+0000 7f1f4f200640 1 -- 172.21.15.196:0/1023392555 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f501f4ec0 con 0x7f1f500add70 2024-04-28T13:33:05.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.042+0000 7f1f4f200640 1 -- 172.21.15.196:0/1023392555 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f500b23d0 con 0x7f1f500add70 2024-04-28T13:33:05.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.043+0000 7f1f4f200640 1 -- 172.21.15.196:0/1023392555 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f5006a290 con 0x7f1f500add70 2024-04-28T13:33:05.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.044+0000 7f1f46a00640 1 -- 172.21.15.196:0/1023392555 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f1f38044020 con 0x7f1f500add70 2024-04-28T13:33:05.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.045+0000 7f1f46a00640 1 --2- 172.21.15.196:0/1023392555 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1f2c06c980 0x7f1f2c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.045+0000 7f1f46a00640 1 -- 172.21.15.196:0/1023392555 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f1f380ac540 con 0x7f1f500add70 2024-04-28T13:33:05.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.046+0000 7f1f4de00640 1 --2- 172.21.15.196:0/1023392555 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1f2c06c980 0x7f1f2c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.047+0000 7f1f4de00640 1 --2- 172.21.15.196:0/1023392555 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1f2c06c980 0x7f1f2c06ee40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f40006fd0 tx=0x7f1f40008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:05.050 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.050+0000 7f1f46a00640 1 -- 172.21.15.196:0/1023392555 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f3804b030 con 0x7f1f500add70 2024-04-28T13:33:05.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.253+0000 7f1f4f200640 1 -- 172.21.15.196:0/1023392555 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f5006ee60 con 0x7f1f500add70 2024-04-28T13:33:05.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.256+0000 7f1f46a00640 1 -- 172.21.15.196:0/1023392555 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16834 (secure 0 0 0) 0x7f1f3807a820 con 0x7f1f500add70 2024-04-28T13:33:05.258 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:05.258 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":34,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:58.527505+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:32:29.352348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-28T13:32:30.637498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6836/459640695":"2024-04-29T13:32:58.527451+0000","172.21.15.203:6835/459640695":"2024-04-29T13:32:58.527451+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:33:05.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.259+0000 7f1f4f200640 1 -- 172.21.15.196:0/1023392555 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1f2c06c980 msgr2=0x7f1f2c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:05.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.259+0000 7f1f4f200640 1 --2- 172.21.15.196:0/1023392555 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1f2c06c980 0x7f1f2c06ee40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f40006fd0 tx=0x7f1f40008040 comp rx=0 tx=0).stop 2024-04-28T13:33:05.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.259+0000 7f1f4f200640 1 -- 172.21.15.196:0/1023392555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f500add70 msgr2=0x7f1f500b1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:05.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.259+0000 7f1f4f200640 1 --2- 172.21.15.196:0/1023392555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f500add70 0x7f1f500b1e90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1f38000c00 tx=0x7f1f380046c0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.259+0000 7f1f4f200640 1 -- 172.21.15.196:0/1023392555 shutdown_connections 2024-04-28T13:33:05.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.259+0000 7f1f4f200640 1 --2- 172.21.15.196:0/1023392555 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1f2c06c980 0x7f1f2c06ee40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.259+0000 7f1f4f200640 1 --2- 172.21.15.196:0/1023392555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f500b5ee0 0x7f1f5009ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.259+0000 7f1f4f200640 1 --2- 172.21.15.196:0/1023392555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f500add70 0x7f1f500b1e90 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.259+0000 7f1f4f200640 1 --2- 172.21.15.196:0/1023392555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f5009ec80 0x7f1f501f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.259+0000 7f1f4f200640 1 -- 172.21.15.196:0/1023392555 >> 172.21.15.196:0/1023392555 conn(0x7f1f5006c6c0 msgr2=0x7f1f500cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:05.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.259+0000 7f1f4f200640 1 -- 172.21.15.196:0/1023392555 shutdown_connections 2024-04-28T13:33:05.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.259+0000 7f1f4f200640 1 -- 172.21.15.196:0/1023392555 wait complete. 2024-04-28T13:33:05.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:32:29.352348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:32:30.637498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:33:05.279 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:33:05.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.478+0000 7fccfde00640 1 Processor -- start 2024-04-28T13:33:05.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.479+0000 7fccfde00640 1 -- start start 2024-04-28T13:33:05.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.480+0000 7fccfde00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fccf80d3830 0x7fccf80d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.480+0000 7fccfde00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fccf80d3c10 0x7fccf80b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.480+0000 7fccfde00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fccf80b8370 0x7fccf80d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.480+0000 7fccfde00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fccf80cfb80 con 0x7fccf80d3830 2024-04-28T13:33:05.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.480+0000 7fccfde00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fccf80d2ee0 con 0x7fccf80d3c10 2024-04-28T13:33:05.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.480+0000 7fccfde00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fccf80d31c0 con 0x7fccf80b8370 2024-04-28T13:33:05.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.481+0000 7fccfca00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fccf80d3830 0x7fccf80d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.481+0000 7fccfca00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fccf80d3830 0x7fccf80d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54498/0 (socket says 172.21.15.196:54498) 2024-04-28T13:33:05.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.481+0000 7fccfca00640 1 -- 172.21.15.196:0/1324648052 learned_addr learned my addr 172.21.15.196:0/1324648052 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:05.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.481+0000 7fccf7e00640 1 --2- 172.21.15.196:0/1324648052 >> v2:172.21.15.203:3300/0 conn(0x7fccf80d3c10 0x7fccf80b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.481+0000 7fccfd400640 1 --2- 172.21.15.196:0/1324648052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fccf80b8370 0x7fccf80d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.481+0000 7fccfca00640 1 -- 172.21.15.196:0/1324648052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fccf80b8370 msgr2=0x7fccf80d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:05.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.481+0000 7fccfca00640 1 --2- 172.21.15.196:0/1324648052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fccf80b8370 0x7fccf80d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.481+0000 7fccfca00640 1 -- 172.21.15.196:0/1324648052 >> v2:172.21.15.203:3300/0 conn(0x7fccf80d3c10 msgr2=0x7fccf80b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:05.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.481+0000 7fccfca00640 1 --2- 172.21.15.196:0/1324648052 >> v2:172.21.15.203:3300/0 conn(0x7fccf80d3c10 0x7fccf80b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.481+0000 7fccfca00640 1 -- 172.21.15.196:0/1324648052 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf80cfe00 con 0x7fccf80d3830 2024-04-28T13:33:05.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.481+0000 7fccfd400640 1 --2- 172.21.15.196:0/1324648052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fccf80b8370 0x7fccf80d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:05.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.482+0000 7fccfca00640 1 --2- 172.21.15.196:0/1324648052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf80d3830 0x7fccf80d2400 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fccf806f510 tx=0x7fcce802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad52465133c68563 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:05.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.482+0000 7fccf7400640 1 -- 172.21.15.196:0/1324648052 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8037070 con 0x7fccf80d3830 2024-04-28T13:33:05.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.482+0000 7fccf7400640 1 -- 172.21.15.196:0/1324648052 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce8032440 con 0x7fccf80d3830 2024-04-28T13:33:05.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.482+0000 7fccf7400640 1 -- 172.21.15.196:0/1324648052 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8032810 con 0x7fccf80d3830 2024-04-28T13:33:05.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.483+0000 7fccfde00640 1 -- 172.21.15.196:0/1324648052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf80d3830 msgr2=0x7fccf80d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:05.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.483+0000 7fccfde00640 1 --2- 172.21.15.196:0/1324648052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf80d3830 0x7fccf80d2400 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fccf806f510 tx=0x7fcce802a1e0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.483+0000 7fccfde00640 1 -- 172.21.15.196:0/1324648052 shutdown_connections 2024-04-28T13:33:05.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.483+0000 7fccfde00640 1 --2- 172.21.15.196:0/1324648052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fccf80b8370 0x7fccf80d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.483+0000 7fccfde00640 1 --2- 172.21.15.196:0/1324648052 >> v2:172.21.15.203:3300/0 conn(0x7fccf80d3c10 0x7fccf80b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.483+0000 7fccfde00640 1 --2- 172.21.15.196:0/1324648052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf80d3830 0x7fccf80d2400 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.483+0000 7fccfde00640 1 -- 172.21.15.196:0/1324648052 >> 172.21.15.196:0/1324648052 conn(0x7fccf80d4fe0 msgr2=0x7fccf80d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:05.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.483+0000 7fccfde00640 1 -- 172.21.15.196:0/1324648052 shutdown_connections 2024-04-28T13:33:05.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.483+0000 7fccfde00640 1 -- 172.21.15.196:0/1324648052 wait complete. 2024-04-28T13:33:05.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.485+0000 7fccfde00640 1 Processor -- start 2024-04-28T13:33:05.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.485+0000 7fccfde00640 1 -- start start 2024-04-28T13:33:05.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.486+0000 7fccfde00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf80b8370 0x7fccf80c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.486+0000 7fccfde00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fccf80d3830 0x7fccf814f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.486+0000 7fccfde00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccf80d3c10 0x7fccf80c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.486+0000 7fccfde00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccf80d5d90 con 0x7fccf80b8370 2024-04-28T13:33:05.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.486+0000 7fccfde00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fccf80d2940 con 0x7fccf80d3c10 2024-04-28T13:33:05.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.486+0000 7fccfde00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fccf80d2bb0 con 0x7fccf80d3830 2024-04-28T13:33:05.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.486+0000 7fccfca00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf80b8370 0x7fccf80c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.486+0000 7fccfca00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf80b8370 0x7fccf80c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54500/0 (socket says 172.21.15.196:54500) 2024-04-28T13:33:05.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.486+0000 7fccfca00640 1 -- 172.21.15.196:0/463733899 learned_addr learned my addr 172.21.15.196:0/463733899 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:05.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.487+0000 7fccf7e00640 1 --2- 172.21.15.196:0/463733899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fccf80d3830 0x7fccf814f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.487+0000 7fccfd400640 1 --2- 172.21.15.196:0/463733899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccf80d3c10 0x7fccf80c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.487+0000 7fccfca00640 1 -- 172.21.15.196:0/463733899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fccf80d3830 msgr2=0x7fccf814f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:05.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.487+0000 7fccfca00640 1 --2- 172.21.15.196:0/463733899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fccf80d3830 0x7fccf814f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.487+0000 7fccfca00640 1 -- 172.21.15.196:0/463733899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccf80d3c10 msgr2=0x7fccf80c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:05.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.487+0000 7fccfca00640 1 --2- 172.21.15.196:0/463733899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccf80d3c10 0x7fccf80c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.487+0000 7fccfca00640 1 -- 172.21.15.196:0/463733899 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf81f55d0 con 0x7fccf80b8370 2024-04-28T13:33:05.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.487+0000 7fccfd400640 1 --2- 172.21.15.196:0/463733899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccf80d3c10 0x7fccf80c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:05.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.487+0000 7fccfca00640 1 --2- 172.21.15.196:0/463733899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf80b8370 0x7fccf80c8b10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcce8000c00 tx=0x7fcce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:05.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.488+0000 7fccf4c00640 1 -- 172.21.15.196:0/463733899 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8041020 con 0x7fccf80b8370 2024-04-28T13:33:05.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.488+0000 7fccf4c00640 1 -- 172.21.15.196:0/463733899 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce80306c0 con 0x7fccf80b8370 2024-04-28T13:33:05.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.488+0000 7fccf4c00640 1 -- 172.21.15.196:0/463733899 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8037040 con 0x7fccf80b8370 2024-04-28T13:33:05.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.488+0000 7fccfde00640 1 -- 172.21.15.196:0/463733899 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf81f58b0 con 0x7fccf80b8370 2024-04-28T13:33:05.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.488+0000 7fccfde00640 1 -- 172.21.15.196:0/463733899 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf81f5e70 con 0x7fccf80b8370 2024-04-28T13:33:05.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.489+0000 7fccfde00640 1 -- 172.21.15.196:0/463733899 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf8050730 con 0x7fccf80b8370 2024-04-28T13:33:05.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.490+0000 7fccf4c00640 1 -- 172.21.15.196:0/463733899 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fcce8044020 con 0x7fccf80b8370 2024-04-28T13:33:05.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.491+0000 7fccf4c00640 1 --2- 172.21.15.196:0/463733899 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fccd406c980 0x7fccd406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.491+0000 7fccf4c00640 1 -- 172.21.15.196:0/463733899 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fcce80ac430 con 0x7fccf80b8370 2024-04-28T13:33:05.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.492+0000 7fccf7e00640 1 --2- 172.21.15.196:0/463733899 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fccd406c980 0x7fccd406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.493+0000 7fccf7e00640 1 --2- 172.21.15.196:0/463733899 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fccd406c980 0x7fccd406ee40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcce0006fd0 tx=0x7fcce0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:05.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.495+0000 7fccf4c00640 1 -- 172.21.15.196:0/463733899 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcce804b030 con 0x7fccf80b8370 2024-04-28T13:33:05.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.695+0000 7fccfde00640 1 -- 172.21.15.196:0/463733899 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fccf806a590 con 0x7fccf80b8370 2024-04-28T13:33:05.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.696+0000 7fccf4c00640 1 -- 172.21.15.196:0/463733899 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7fcce807a810 con 0x7fccf80b8370 2024-04-28T13:33:05.696 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:33:05.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.698+0000 7fccfde00640 1 -- 172.21.15.196:0/463733899 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fccd406c980 msgr2=0x7fccd406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:05.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.698+0000 7fccfde00640 1 --2- 172.21.15.196:0/463733899 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fccd406c980 0x7fccd406ee40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcce0006fd0 tx=0x7fcce0008040 comp rx=0 tx=0).stop 2024-04-28T13:33:05.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.699+0000 7fccfde00640 1 -- 172.21.15.196:0/463733899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf80b8370 msgr2=0x7fccf80c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:05.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.699+0000 7fccfde00640 1 --2- 172.21.15.196:0/463733899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf80b8370 0x7fccf80c8b10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcce8000c00 tx=0x7fcce80046c0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.699+0000 7fccfde00640 1 -- 172.21.15.196:0/463733899 shutdown_connections 2024-04-28T13:33:05.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.699+0000 7fccfde00640 1 --2- 172.21.15.196:0/463733899 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fccd406c980 0x7fccd406ee40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.699+0000 7fccfde00640 1 --2- 172.21.15.196:0/463733899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccf80d3c10 0x7fccf80c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.699+0000 7fccfde00640 1 --2- 172.21.15.196:0/463733899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fccf80d3830 0x7fccf814f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.699+0000 7fccfde00640 1 --2- 172.21.15.196:0/463733899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccf80b8370 0x7fccf80c8b10 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.699+0000 7fccfde00640 1 -- 172.21.15.196:0/463733899 >> 172.21.15.196:0/463733899 conn(0x7fccf80d4fe0 msgr2=0x7fccf80d1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:05.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.699+0000 7fccfde00640 1 -- 172.21.15.196:0/463733899 shutdown_connections 2024-04-28T13:33:05.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.699+0000 7fccfde00640 1 -- 172.21.15.196:0/463733899 wait complete. 2024-04-28T13:33:05.719 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:33:05.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.930+0000 7f1d42400640 1 Processor -- start 2024-04-28T13:33:05.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.930+0000 7f1d42400640 1 -- start start 2024-04-28T13:33:05.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.932+0000 7f1d42400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1d3c06f340 0x7f1d3c0b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.932+0000 7f1d42400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d3c0b3e50 0x7f1d3c0720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.932+0000 7f1d42400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d3c0b46a0 con 0x7f1d3c0d0300 2024-04-28T13:33:05.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.932+0000 7f1d42400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1d3c0b5db0 con 0x7f1d3c06f340 2024-04-28T13:33:05.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.932+0000 7f1d42400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3c0b6090 con 0x7f1d3c0b3e50 2024-04-28T13:33:05.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.932+0000 7f1d3b400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1d3c06f340 0x7f1d3c0b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.932+0000 7f1d3b400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1d3c06f340 0x7f1d3c0b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54510/0 (socket says 172.21.15.196:54510) 2024-04-28T13:33:05.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.932+0000 7f1d3b400640 1 -- 172.21.15.196:0/1009454769 learned_addr learned my addr 172.21.15.196:0/1009454769 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:05.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.933+0000 7f1d3be00640 1 --2- 172.21.15.196:0/1009454769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d3c0b3e50 0x7f1d3c0720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.933+0000 7f1d3b400640 1 -- 172.21.15.196:0/1009454769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d3c0b3e50 msgr2=0x7f1d3c0720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:05.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.933+0000 7f1d3b400640 1 --2- 172.21.15.196:0/1009454769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d3c0b3e50 0x7f1d3c0720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.933+0000 7f1d3b400640 1 -- 172.21.15.196:0/1009454769 >> v1:172.21.15.203:6789/0 conn(0x7f1d3c0d0300 legacy=0x7f1d3c06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:05.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.933+0000 7f1d3b400640 1 -- 172.21.15.196:0/1009454769 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c0b6370 con 0x7f1d3c06f340 2024-04-28T13:33:05.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.934+0000 7f1d3b400640 1 --2- 172.21.15.196:0/1009454769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c06f340 0x7f1d3c0b6b70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009870 tx=0x7f1d2c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a187093ecc5311 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:05.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.934+0000 7f1d3a000640 1 -- 172.21.15.196:0/1009454769 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c037070 con 0x7f1d3c06f340 2024-04-28T13:33:05.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.934+0000 7f1d3a000640 1 -- 172.21.15.196:0/1009454769 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d2c02ae10 con 0x7f1d3c06f340 2024-04-28T13:33:05.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.934+0000 7f1d3a000640 1 -- 172.21.15.196:0/1009454769 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c032660 con 0x7f1d3c06f340 2024-04-28T13:33:05.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.935+0000 7f1d42400640 1 -- 172.21.15.196:0/1009454769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c06f340 msgr2=0x7f1d3c0b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:05.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.935+0000 7f1d42400640 1 --2- 172.21.15.196:0/1009454769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c06f340 0x7f1d3c0b6b70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009870 tx=0x7f1d2c02a100 comp rx=0 tx=0).stop 2024-04-28T13:33:05.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.935+0000 7f1d42400640 1 -- 172.21.15.196:0/1009454769 shutdown_connections 2024-04-28T13:33:05.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.935+0000 7f1d42400640 1 --2- 172.21.15.196:0/1009454769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d3c0b3e50 0x7f1d3c0720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.935+0000 7f1d42400640 1 --2- 172.21.15.196:0/1009454769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c06f340 0x7f1d3c0b6b70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.935+0000 7f1d42400640 1 -- 172.21.15.196:0/1009454769 >> 172.21.15.196:0/1009454769 conn(0x7f1d3c0cd8a0 msgr2=0x7f1d3c0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:05.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.935+0000 7f1d42400640 1 -- 172.21.15.196:0/1009454769 shutdown_connections 2024-04-28T13:33:05.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.935+0000 7f1d42400640 1 -- 172.21.15.196:0/1009454769 wait complete. 2024-04-28T13:33:05.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.937+0000 7f1d42400640 1 Processor -- start 2024-04-28T13:33:05.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.937+0000 7f1d42400640 1 -- start start 2024-04-28T13:33:05.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.938+0000 7f1d42400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d3c06f340 0x7f1d3c13d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.938+0000 7f1d42400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d3c0b3e50 0x7f1d3c15d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.938+0000 7f1d42400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c0d0300 0x7f1d3c15dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.938+0000 7f1d42400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c0b4460 con 0x7f1d3c0d0300 2024-04-28T13:33:05.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.938+0000 7f1d42400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c0b42e0 con 0x7f1d3c06f340 2024-04-28T13:33:05.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.938+0000 7f1d42400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3c142a30 con 0x7f1d3c0b3e50 2024-04-28T13:33:05.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.938+0000 7f1d3be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c0d0300 0x7f1d3c15dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.938+0000 7f1d3be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c0d0300 0x7f1d3c15dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54512/0 (socket says 172.21.15.196:54512) 2024-04-28T13:33:05.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.938+0000 7f1d3be00640 1 -- 172.21.15.196:0/988588075 learned_addr learned my addr 172.21.15.196:0/988588075 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:05.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.939+0000 7f1d3aa00640 1 --2- 172.21.15.196:0/988588075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d3c0b3e50 0x7f1d3c15d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.939+0000 7f1d3b400640 1 --2- 172.21.15.196:0/988588075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d3c06f340 0x7f1d3c13d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.939+0000 7f1d3be00640 1 -- 172.21.15.196:0/988588075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d3c0b3e50 msgr2=0x7f1d3c15d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:05.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.939+0000 7f1d3be00640 1 --2- 172.21.15.196:0/988588075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d3c0b3e50 0x7f1d3c15d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.939+0000 7f1d3be00640 1 -- 172.21.15.196:0/988588075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d3c06f340 msgr2=0x7f1d3c13d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:05.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.939+0000 7f1d3be00640 1 --2- 172.21.15.196:0/988588075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d3c06f340 0x7f1d3c13d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:05.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.939+0000 7f1d3be00640 1 -- 172.21.15.196:0/988588075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c155230 con 0x7f1d3c0d0300 2024-04-28T13:33:05.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.939+0000 7f1d3be00640 1 --2- 172.21.15.196:0/988588075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c0d0300 0x7f1d3c15dc70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1d34007e40 tx=0x7f1d3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:05.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.939+0000 7f1d33400640 1 -- 172.21.15.196:0/988588075 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34014620 con 0x7f1d3c0d0300 2024-04-28T13:33:05.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.939+0000 7f1d33400640 1 -- 172.21.15.196:0/988588075 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d34019070 con 0x7f1d3c0d0300 2024-04-28T13:33:05.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.939+0000 7f1d33400640 1 -- 172.21.15.196:0/988588075 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34014a20 con 0x7f1d3c0d0300 2024-04-28T13:33:05.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.939+0000 7f1d42400640 1 -- 172.21.15.196:0/988588075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3c158100 con 0x7f1d3c0d0300 2024-04-28T13:33:05.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.940+0000 7f1d42400640 1 -- 172.21.15.196:0/988588075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d3c1554d0 con 0x7f1d3c0d0300 2024-04-28T13:33:05.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.941+0000 7f1d42400640 1 -- 172.21.15.196:0/988588075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d3c152730 con 0x7f1d3c0d0300 2024-04-28T13:33:05.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.942+0000 7f1d33400640 1 -- 172.21.15.196:0/988588075 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f1d34014bc0 con 0x7f1d3c0d0300 2024-04-28T13:33:05.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.942+0000 7f1d33400640 1 --2- 172.21.15.196:0/988588075 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1d1c06c980 0x7f1d1c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:05.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.943+0000 7f1d33400640 1 -- 172.21.15.196:0/988588075 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f1d3408c590 con 0x7f1d3c0d0300 2024-04-28T13:33:05.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.943+0000 7f1d3b400640 1 --2- 172.21.15.196:0/988588075 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1d1c06c980 0x7f1d1c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:05.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.944+0000 7f1d3b400640 1 --2- 172.21.15.196:0/988588075 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1d1c06c980 0x7f1d1c06ee40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c034000 tx=0x7f1d2c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:05.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:05.947+0000 7f1d33400640 1 -- 172.21.15.196:0/988588075 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d3c152730 con 0x7f1d3c0d0300 2024-04-28T13:33:06.146 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.145+0000 7f1d42400640 1 -- 172.21.15.196:0/988588075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1d3c06a290 con 0x7f1d3c0d0300 2024-04-28T13:33:06.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.146+0000 7f1d33400640 1 -- 172.21.15.196:0/988588075 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f1d3405a900 con 0x7f1d3c0d0300 2024-04-28T13:33:06.147 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:33:06.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.149+0000 7f1d42400640 1 -- 172.21.15.196:0/988588075 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1d1c06c980 msgr2=0x7f1d1c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:06.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.149+0000 7f1d42400640 1 --2- 172.21.15.196:0/988588075 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1d1c06c980 0x7f1d1c06ee40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c034000 tx=0x7f1d2c02e000 comp rx=0 tx=0).stop 2024-04-28T13:33:06.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.149+0000 7f1d42400640 1 -- 172.21.15.196:0/988588075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c0d0300 msgr2=0x7f1d3c15dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:06.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.149+0000 7f1d42400640 1 --2- 172.21.15.196:0/988588075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c0d0300 0x7f1d3c15dc70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1d34007e40 tx=0x7f1d3400e4b0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.149+0000 7f1d42400640 1 -- 172.21.15.196:0/988588075 shutdown_connections 2024-04-28T13:33:06.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.149+0000 7f1d42400640 1 --2- 172.21.15.196:0/988588075 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1d1c06c980 0x7f1d1c06ee40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.149+0000 7f1d42400640 1 --2- 172.21.15.196:0/988588075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c0d0300 0x7f1d3c15dc70 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.149+0000 7f1d42400640 1 --2- 172.21.15.196:0/988588075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d3c0b3e50 0x7f1d3c15d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.149+0000 7f1d42400640 1 --2- 172.21.15.196:0/988588075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d3c06f340 0x7f1d3c13d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.149+0000 7f1d42400640 1 -- 172.21.15.196:0/988588075 >> 172.21.15.196:0/988588075 conn(0x7f1d3c0cd8a0 msgr2=0x7f1d3c143120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:06.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.149+0000 7f1d42400640 1 -- 172.21.15.196:0/988588075 shutdown_connections 2024-04-28T13:33:06.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.149+0000 7f1d42400640 1 -- 172.21.15.196:0/988588075 wait complete. 2024-04-28T13:33:06.169 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:33:06.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.362+0000 7f08d3000640 1 Processor -- start 2024-04-28T13:33:06.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.363+0000 7f08d3000640 1 -- start start 2024-04-28T13:33:06.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.364+0000 7f08d3000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f08cc0b65f0 0x7f08cc0b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:06.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.364+0000 7f08d3000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08cc163b40 0x7f08cc06f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:06.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.364+0000 7f08d3000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f08cc06f880 0x7f08cc167450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:06.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.364+0000 7f08d3000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f08cc073440 con 0x7f08cc06f880 2024-04-28T13:33:06.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.364+0000 7f08d3000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f08cc0b6f10 con 0x7f08cc0b65f0 2024-04-28T13:33:06.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.364+0000 7f08d3000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f08cc0723a0 con 0x7f08cc163b40 2024-04-28T13:33:06.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.365+0000 7f08d1000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f08cc06f880 0x7f08cc167450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:06.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.365+0000 7f08d1000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f08cc06f880 0x7f08cc167450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54516/0 (socket says 172.21.15.196:54516) 2024-04-28T13:33:06.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.365+0000 7f08d1000640 1 -- 172.21.15.196:0/1806482206 learned_addr learned my addr 172.21.15.196:0/1806482206 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:06.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.365+0000 7f08cbe00640 1 --2- 172.21.15.196:0/1806482206 >> v2:172.21.15.203:3300/0 conn(0x7f08cc0b65f0 0x7f08cc0b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:06.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.365+0000 7f08c3e00640 1 --2- 172.21.15.196:0/1806482206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08cc163b40 0x7f08cc06f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:06.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.365+0000 7f08d1000640 1 -- 172.21.15.196:0/1806482206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08cc163b40 msgr2=0x7f08cc06f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:06.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.365+0000 7f08d1000640 1 --2- 172.21.15.196:0/1806482206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08cc163b40 0x7f08cc06f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.365+0000 7f08d1000640 1 -- 172.21.15.196:0/1806482206 >> v2:172.21.15.203:3300/0 conn(0x7f08cc0b65f0 msgr2=0x7f08cc0b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:06.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.365+0000 7f08d1000640 1 --2- 172.21.15.196:0/1806482206 >> v2:172.21.15.203:3300/0 conn(0x7f08cc0b65f0 0x7f08cc0b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.365+0000 7f08d1000640 1 -- 172.21.15.196:0/1806482206 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc072680 con 0x7f08cc06f880 2024-04-28T13:33:06.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.366+0000 7f08d1000640 1 --2- 172.21.15.196:0/1806482206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08cc06f880 0x7f08cc167450 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0d0950 tx=0x7f08c402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397ac6827115366 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:06.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.367+0000 7f08cb400640 1 -- 172.21.15.196:0/1806482206 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c4036070 con 0x7f08cc06f880 2024-04-28T13:33:06.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.367+0000 7f08cb400640 1 -- 172.21.15.196:0/1806482206 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c4031600 con 0x7f08cc06f880 2024-04-28T13:33:06.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.367+0000 7f08cb400640 1 -- 172.21.15.196:0/1806482206 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c40319d0 con 0x7f08cc06f880 2024-04-28T13:33:06.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.367+0000 7f08d3000640 1 -- 172.21.15.196:0/1806482206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08cc06f880 msgr2=0x7f08cc167450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:06.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.367+0000 7f08d3000640 1 --2- 172.21.15.196:0/1806482206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08cc06f880 0x7f08cc167450 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0d0950 tx=0x7f08c402a610 comp rx=0 tx=0).stop 2024-04-28T13:33:06.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.368+0000 7f08d3000640 1 -- 172.21.15.196:0/1806482206 shutdown_connections 2024-04-28T13:33:06.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.368+0000 7f08d3000640 1 --2- 172.21.15.196:0/1806482206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08cc06f880 0x7f08cc167450 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.368+0000 7f08d3000640 1 --2- 172.21.15.196:0/1806482206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08cc163b40 0x7f08cc06f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.368+0000 7f08d3000640 1 --2- 172.21.15.196:0/1806482206 >> v2:172.21.15.203:3300/0 conn(0x7f08cc0b65f0 0x7f08cc0b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.368+0000 7f08d3000640 1 -- 172.21.15.196:0/1806482206 >> 172.21.15.196:0/1806482206 conn(0x7f08cc0cd8a0 msgr2=0x7f08cc0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:06.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.368+0000 7f08d3000640 1 -- 172.21.15.196:0/1806482206 shutdown_connections 2024-04-28T13:33:06.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.368+0000 7f08d3000640 1 -- 172.21.15.196:0/1806482206 wait complete. 2024-04-28T13:33:06.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.369+0000 7f08d3000640 1 Processor -- start 2024-04-28T13:33:06.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.370+0000 7f08d3000640 1 -- start start 2024-04-28T13:33:06.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.370+0000 7f08d3000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08cc06f880 0x7f08cc15b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:06.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.371+0000 7f08d3000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08cc0b65f0 0x7f08cc165ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:06.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.371+0000 7f08d3000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08cc163b40 0x7f08cc15bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:06.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.371+0000 7f08d3000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc06fd00 con 0x7f08cc06f880 2024-04-28T13:33:06.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.371+0000 7f08d3000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc072c40 con 0x7f08cc163b40 2024-04-28T13:33:06.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.371+0000 7f08d3000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f08cc167990 con 0x7f08cc0b65f0 2024-04-28T13:33:06.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.371+0000 7f08cbe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08cc06f880 0x7f08cc15b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:06.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.371+0000 7f08cbe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08cc06f880 0x7f08cc15b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54522/0 (socket says 172.21.15.196:54522) 2024-04-28T13:33:06.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.371+0000 7f08cbe00640 1 -- 172.21.15.196:0/3706019652 learned_addr learned my addr 172.21.15.196:0/3706019652 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:06.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.371+0000 7f08cbe00640 1 -- 172.21.15.196:0/3706019652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08cc0b65f0 msgr2=0x7f08cc165ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:06.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.371+0000 7f08cbe00640 1 --2- 172.21.15.196:0/3706019652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08cc0b65f0 0x7f08cc165ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.371+0000 7f08cbe00640 1 -- 172.21.15.196:0/3706019652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08cc163b40 msgr2=0x7f08cc15bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:06.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.371+0000 7f08cbe00640 1 --2- 172.21.15.196:0/3706019652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08cc163b40 0x7f08cc15bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.371+0000 7f08cbe00640 1 -- 172.21.15.196:0/3706019652 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc1665a0 con 0x7f08cc06f880 2024-04-28T13:33:06.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.372+0000 7f08cbe00640 1 --2- 172.21.15.196:0/3706019652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08cc06f880 0x7f08cc15b760 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00d800 tx=0x7f08bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:06.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.372+0000 7f08c8c00640 1 -- 172.21.15.196:0/3706019652 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc018020 con 0x7f08cc06f880 2024-04-28T13:33:06.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.372+0000 7f08c8c00640 1 -- 172.21.15.196:0/3706019652 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08bc0096a0 con 0x7f08cc06f880 2024-04-28T13:33:06.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.372+0000 7f08c8c00640 1 -- 172.21.15.196:0/3706019652 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc009a70 con 0x7f08cc06f880 2024-04-28T13:33:06.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.372+0000 7f08d3000640 1 -- 172.21.15.196:0/3706019652 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08cc166880 con 0x7f08cc06f880 2024-04-28T13:33:06.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.372+0000 7f08d3000640 1 -- 172.21.15.196:0/3706019652 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08cc159840 con 0x7f08cc06f880 2024-04-28T13:33:06.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.373+0000 7f08d3000640 1 -- 172.21.15.196:0/3706019652 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08cc06a290 con 0x7f08cc06f880 2024-04-28T13:33:06.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.374+0000 7f08c8c00640 1 -- 172.21.15.196:0/3706019652 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f08bc01b020 con 0x7f08cc06f880 2024-04-28T13:33:06.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.375+0000 7f08c8c00640 1 --2- 172.21.15.196:0/3706019652 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f08a806c9b0 0x7f08a806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:06.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.375+0000 7f08c8c00640 1 -- 172.21.15.196:0/3706019652 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f08bc08bdc0 con 0x7f08cc06f880 2024-04-28T13:33:06.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.376+0000 7f08c3e00640 1 --2- 172.21.15.196:0/3706019652 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f08a806c9b0 0x7f08a806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:06.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.377+0000 7f08c3e00640 1 --2- 172.21.15.196:0/3706019652 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f08a806c9b0 0x7f08a806ee70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f08b4006fd0 tx=0x7f08b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:06.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.380+0000 7f08c8c00640 1 -- 172.21.15.196:0/3706019652 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f08bc022030 con 0x7f08cc06f880 2024-04-28T13:33:06.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.577+0000 7f08d3000640 1 -- 172.21.15.196:0/3706019652 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f08cc166a50 con 0x7f08cc06f880 2024-04-28T13:33:06.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.578+0000 7f08c8c00640 1 -- 172.21.15.196:0/3706019652 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f08bc05a230 con 0x7f08cc06f880 2024-04-28T13:33:06.579 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:33:06.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.581+0000 7f08d3000640 1 -- 172.21.15.196:0/3706019652 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f08a806c9b0 msgr2=0x7f08a806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:06.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.581+0000 7f08d3000640 1 --2- 172.21.15.196:0/3706019652 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f08a806c9b0 0x7f08a806ee70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f08b4006fd0 tx=0x7f08b4008040 comp rx=0 tx=0).stop 2024-04-28T13:33:06.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.581+0000 7f08d3000640 1 -- 172.21.15.196:0/3706019652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08cc06f880 msgr2=0x7f08cc15b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:06.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.581+0000 7f08d3000640 1 --2- 172.21.15.196:0/3706019652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08cc06f880 0x7f08cc15b760 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00d800 tx=0x7f08bc00dcc0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.581+0000 7f08d3000640 1 -- 172.21.15.196:0/3706019652 shutdown_connections 2024-04-28T13:33:06.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.581+0000 7f08d3000640 1 --2- 172.21.15.196:0/3706019652 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f08a806c9b0 0x7f08a806ee70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.581+0000 7f08d3000640 1 --2- 172.21.15.196:0/3706019652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08cc163b40 0x7f08cc15bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.581+0000 7f08d3000640 1 --2- 172.21.15.196:0/3706019652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08cc0b65f0 0x7f08cc165ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.581+0000 7f08d3000640 1 --2- 172.21.15.196:0/3706019652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08cc06f880 0x7f08cc15b760 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.581+0000 7f08d3000640 1 -- 172.21.15.196:0/3706019652 >> 172.21.15.196:0/3706019652 conn(0x7f08cc0cd8a0 msgr2=0x7f08cc06ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:06.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.581+0000 7f08d3000640 1 -- 172.21.15.196:0/3706019652 shutdown_connections 2024-04-28T13:33:06.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.581+0000 7f08d3000640 1 -- 172.21.15.196:0/3706019652 wait complete. 2024-04-28T13:33:06.601 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T13:33:06.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.803+0000 7f4fe9200640 1 Processor -- start 2024-04-28T13:33:06.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.804+0000 7f4fe9200640 1 -- start start 2024-04-28T13:33:06.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.805+0000 7f4fe9200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4fe40b5ee0 0x7f4fe40b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:06.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.805+0000 7f4fe9200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4fe40d1760 0x7f4fe406df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:06.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.805+0000 7f4fe9200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fe415e380 con 0x7f4fe409ec80 2024-04-28T13:33:06.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.805+0000 7f4fe9200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4fe40b4b20 con 0x7f4fe40b5ee0 2024-04-28T13:33:06.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.805+0000 7f4fe9200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4fe40b3f30 con 0x7f4fe40d1760 2024-04-28T13:33:06.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.806+0000 7f4fe3400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4fe40b5ee0 0x7f4fe40b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:06.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.806+0000 7f4fe2a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f4fe409ec80 0x7f4fe40d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:33778/0 (socket says 172.21.15.196:33778) 2024-04-28T13:33:06.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.806+0000 7f4fe2a00640 1 -- 172.21.15.196:0/624211836 learned_addr learned my addr 172.21.15.196:0/624211836 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:06.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.806+0000 7f4fe3e00640 1 --2- 172.21.15.196:0/624211836 >> v2:172.21.15.203:3300/0 conn(0x7f4fe40d1760 0x7f4fe406df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:06.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.806+0000 7f4fe3400640 1 -- 172.21.15.196:0/624211836 >> v2:172.21.15.203:3300/0 conn(0x7f4fe40d1760 msgr2=0x7f4fe406df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:06.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.806+0000 7f4fe3400640 1 --2- 172.21.15.196:0/624211836 >> v2:172.21.15.203:3300/0 conn(0x7f4fe40d1760 0x7f4fe406df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.806+0000 7f4fe3400640 1 -- 172.21.15.196:0/624211836 >> v1:172.21.15.196:6789/0 conn(0x7f4fe409ec80 legacy=0x7f4fe40d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:06.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.806+0000 7f4fe3400640 1 -- 172.21.15.196:0/624211836 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe40b41f0 con 0x7f4fe40b5ee0 2024-04-28T13:33:06.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.807+0000 7f4fe3400640 1 --2- 172.21.15.196:0/624211836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe40b5ee0 0x7f4fe40b4f40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4074210 tx=0x7f4fd402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac7f7c091957c44 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:06.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.807+0000 7f4fe2000640 1 -- 172.21.15.196:0/624211836 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4007740 con 0x7f4fe40b5ee0 2024-04-28T13:33:06.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.807+0000 7f4fe2000640 1 -- 172.21.15.196:0/624211836 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd40078e0 con 0x7f4fe40b5ee0 2024-04-28T13:33:06.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.807+0000 7f4fe2000640 1 -- 172.21.15.196:0/624211836 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4007cb0 con 0x7f4fe40b5ee0 2024-04-28T13:33:06.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.808+0000 7f4fe9200640 1 -- 172.21.15.196:0/624211836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe40b5ee0 msgr2=0x7f4fe40b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:06.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.808+0000 7f4fe9200640 1 --2- 172.21.15.196:0/624211836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe40b5ee0 0x7f4fe40b4f40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4074210 tx=0x7f4fd402a680 comp rx=0 tx=0).stop 2024-04-28T13:33:06.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.808+0000 7f4fe9200640 1 -- 172.21.15.196:0/624211836 shutdown_connections 2024-04-28T13:33:06.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.808+0000 7f4fe9200640 1 --2- 172.21.15.196:0/624211836 >> v2:172.21.15.203:3300/0 conn(0x7f4fe40d1760 0x7f4fe406df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.808+0000 7f4fe9200640 1 --2- 172.21.15.196:0/624211836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe40b5ee0 0x7f4fe40b4f40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.808+0000 7f4fe9200640 1 -- 172.21.15.196:0/624211836 >> 172.21.15.196:0/624211836 conn(0x7f4fe406c6c0 msgr2=0x7f4fe406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:06.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.808+0000 7f4fe9200640 1 -- 172.21.15.196:0/624211836 shutdown_connections 2024-04-28T13:33:06.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.808+0000 7f4fe9200640 1 -- 172.21.15.196:0/624211836 wait complete. 2024-04-28T13:33:06.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.810+0000 7f4fe9200640 1 Processor -- start 2024-04-28T13:33:06.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.810+0000 7f4fe9200640 1 -- start start 2024-04-28T13:33:06.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.811+0000 7f4fe9200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe409ec80 0x7f4fe40c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:06.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.811+0000 7f4fe9200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fe40b5ee0 0x7f4fe409e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:06.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.811+0000 7f4fe9200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fe40d1760 0x7f4fe4073460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:06.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.811+0000 7f4fe9200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe40b3c20 con 0x7f4fe409ec80 2024-04-28T13:33:06.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.811+0000 7f4fe9200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe40b47b0 con 0x7f4fe40b5ee0 2024-04-28T13:33:06.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.811+0000 7f4fe9200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4fe40add70 con 0x7f4fe40d1760 2024-04-28T13:33:06.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.811+0000 7f4fe3400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe409ec80 0x7f4fe40c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:06.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.812+0000 7f4fe3400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe409ec80 0x7f4fe40c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54540/0 (socket says 172.21.15.196:54540) 2024-04-28T13:33:06.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.812+0000 7f4fe3400640 1 -- 172.21.15.196:0/2559622869 learned_addr learned my addr 172.21.15.196:0/2559622869 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:06.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.812+0000 7f4fe2a00640 1 --2- 172.21.15.196:0/2559622869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fe40b5ee0 0x7f4fe409e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:06.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.812+0000 7f4fe3400640 1 -- 172.21.15.196:0/2559622869 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fe40d1760 msgr2=0x7f4fe4073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:06.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.812+0000 7f4fe3400640 1 --2- 172.21.15.196:0/2559622869 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fe40d1760 0x7f4fe4073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.812+0000 7f4fe3400640 1 -- 172.21.15.196:0/2559622869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fe40b5ee0 msgr2=0x7f4fe409e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:06.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.812+0000 7f4fe3400640 1 --2- 172.21.15.196:0/2559622869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fe40b5ee0 0x7f4fe409e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:06.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.812+0000 7f4fe3400640 1 -- 172.21.15.196:0/2559622869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe40b24d0 con 0x7f4fe409ec80 2024-04-28T13:33:06.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.812+0000 7f4fe3400640 1 --2- 172.21.15.196:0/2559622869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe409ec80 0x7f4fe40c9fa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4fd402ac70 tx=0x7f4fd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:06.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.813+0000 7f4fdb400640 1 -- 172.21.15.196:0/2559622869 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4040070 con 0x7f4fe409ec80 2024-04-28T13:33:06.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.813+0000 7f4fdb400640 1 -- 172.21.15.196:0/2559622869 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd4031c40 con 0x7f4fe409ec80 2024-04-28T13:33:06.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.813+0000 7f4fdb400640 1 -- 172.21.15.196:0/2559622869 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd403c420 con 0x7f4fe409ec80 2024-04-28T13:33:06.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.813+0000 7f4fe9200640 1 -- 172.21.15.196:0/2559622869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe409cdf0 con 0x7f4fe409ec80 2024-04-28T13:33:06.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.813+0000 7f4fe9200640 1 -- 172.21.15.196:0/2559622869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe40b2130 con 0x7f4fe409ec80 2024-04-28T13:33:06.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.814+0000 7f4fe9200640 1 -- 172.21.15.196:0/2559622869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe409cc10 con 0x7f4fe409ec80 2024-04-28T13:33:06.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.815+0000 7f4fdb400640 1 -- 172.21.15.196:0/2559622869 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4fd4039040 con 0x7f4fe409ec80 2024-04-28T13:33:06.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.816+0000 7f4fdb400640 1 --2- 172.21.15.196:0/2559622869 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4fc406c980 0x7f4fc406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:06.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.816+0000 7f4fdb400640 1 -- 172.21.15.196:0/2559622869 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f4fd40ad400 con 0x7f4fe409ec80 2024-04-28T13:33:06.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.817+0000 7f4fe2a00640 1 --2- 172.21.15.196:0/2559622869 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4fc406c980 0x7f4fc406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:06.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.818+0000 7f4fe2a00640 1 --2- 172.21.15.196:0/2559622869 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4fc406c980 0x7f4fc406ee40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4fe409cb40 tx=0x7f4fd0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:06.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:06.820+0000 7f4fdb400640 1 -- 172.21.15.196:0/2559622869 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4fe409cc10 con 0x7f4fe409ec80 2024-04-28T13:33:07.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.035+0000 7f4fe9200640 1 -- 172.21.15.196:0/2559622869 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4fe406a290 con 0x7f4fe409ec80 2024-04-28T13:33:07.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.036+0000 7f4fdb400640 1 -- 172.21.15.196:0/2559622869 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v34) ==== 80+0+122 (secure 0 0 0) 0x7f4fd407b010 con 0x7f4fe409ec80 2024-04-28T13:33:07.037 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.203:6836/459640695 2024-04-29T13:32:58.527451+0000 2024-04-28T13:33:07.037 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.203:6835/459640695 2024-04-29T13:32:58.527451+0000 2024-04-28T13:33:07.037 INFO:teuthology.orchestra.run.smithi196.stderr:listed 2 entries 2024-04-28T13:33:07.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.039+0000 7f4fe9200640 1 -- 172.21.15.196:0/2559622869 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4fc406c980 msgr2=0x7f4fc406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:07.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.039+0000 7f4fe9200640 1 --2- 172.21.15.196:0/2559622869 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4fc406c980 0x7f4fc406ee40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4fe409cb40 tx=0x7f4fd0009040 comp rx=0 tx=0).stop 2024-04-28T13:33:07.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.039+0000 7f4fe9200640 1 -- 172.21.15.196:0/2559622869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe409ec80 msgr2=0x7f4fe40c9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:07.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.039+0000 7f4fe9200640 1 --2- 172.21.15.196:0/2559622869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe409ec80 0x7f4fe40c9fa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4fd402ac70 tx=0x7f4fd4004a40 comp rx=0 tx=0).stop 2024-04-28T13:33:07.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.039+0000 7f4fe9200640 1 -- 172.21.15.196:0/2559622869 shutdown_connections 2024-04-28T13:33:07.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.039+0000 7f4fe9200640 1 --2- 172.21.15.196:0/2559622869 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4fc406c980 0x7f4fc406ee40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.039+0000 7f4fe9200640 1 --2- 172.21.15.196:0/2559622869 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fe40d1760 0x7f4fe4073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.039+0000 7f4fe9200640 1 --2- 172.21.15.196:0/2559622869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fe40b5ee0 0x7f4fe409e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.039+0000 7f4fe9200640 1 --2- 172.21.15.196:0/2559622869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe409ec80 0x7f4fe40c9fa0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.039+0000 7f4fe9200640 1 -- 172.21.15.196:0/2559622869 >> 172.21.15.196:0/2559622869 conn(0x7f4fe406c6c0 msgr2=0x7f4fe415f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:07.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.039+0000 7f4fe9200640 1 -- 172.21.15.196:0/2559622869 shutdown_connections 2024-04-28T13:33:07.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.039+0000 7f4fe9200640 1 -- 172.21.15.196:0/2559622869 wait complete. 2024-04-28T13:33:07.059 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-28T13:33:07.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:07.059 DEBUG:teuthology.orchestra.run.smithi196:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-28T13:33:07.150 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:33:07.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:07.150 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-28T13:33:07.202 INFO:tasks.cephfs.mount:Terminating background process 2024-04-28T13:33:07.203 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:33:07.203 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:33:07.261 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:33:07.261 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:33:07.312 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:33:07.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.521+0000 7fa3b3e00640 1 Processor -- start 2024-04-28T13:33:07.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.521+0000 7fa3b3e00640 1 -- start start 2024-04-28T13:33:07.525 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.524+0000 7fa3b3e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa3b4157360 0x7fa3b406dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:07.525 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.524+0000 7fa3b3e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3b415e3e0 con 0x7fa3b40b59b0 2024-04-28T13:33:07.525 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.524+0000 7fa3b3e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3b415d7d0 con 0x7fa3b40d1760 2024-04-28T13:33:07.525 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.524+0000 7fa3b3e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa3b415dab0 con 0x7fa3b4157360 2024-04-28T13:33:07.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.524+0000 7fa3b2a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa3b4157360 0x7fa3b406dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:07.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.524+0000 7fa3b2000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa3b40b59b0 0x7fa3b40d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:33780/0 (socket says 172.21.15.196:33780) 2024-04-28T13:33:07.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.524+0000 7fa3b2000640 1 -- 172.21.15.196:0/1891779190 learned_addr learned my addr 172.21.15.196:0/1891779190 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:07.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.525+0000 7fa3b2a00640 1 -- 172.21.15.196:0/1891779190 >> v1:172.21.15.203:6789/0 conn(0x7fa3b40d1760 legacy=0x7fa3b40cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:07.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.525+0000 7fa3b2a00640 1 -- 172.21.15.196:0/1891779190 >> v1:172.21.15.196:6789/0 conn(0x7fa3b40b59b0 legacy=0x7fa3b40d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:07.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.525+0000 7fa3b2a00640 1 -- 172.21.15.196:0/1891779190 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b415dd90 con 0x7fa3b4157360 2024-04-28T13:33:07.527 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.526+0000 7fa3b2a00640 1 --2- 172.21.15.196:0/1891779190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b4157360 0x7fa3b406dc70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40097a0 tx=0x7fa3a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f827e043a34d7057 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:07.527 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.526+0000 7fa3b1600640 1 -- 172.21.15.196:0/1891779190 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4037070 con 0x7fa3b4157360 2024-04-28T13:33:07.527 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.526+0000 7fa3b1600640 1 -- 172.21.15.196:0/1891779190 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a4032440 con 0x7fa3b4157360 2024-04-28T13:33:07.527 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.526+0000 7fa3b1600640 1 -- 172.21.15.196:0/1891779190 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4032810 con 0x7fa3b4157360 2024-04-28T13:33:07.528 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.527+0000 7fa3b3e00640 1 -- 172.21.15.196:0/1891779190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b4157360 msgr2=0x7fa3b406dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:07.528 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.527+0000 7fa3b3e00640 1 --2- 172.21.15.196:0/1891779190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b4157360 0x7fa3b406dc70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40097a0 tx=0x7fa3a402a1e0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.528 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.527+0000 7fa3b3e00640 1 -- 172.21.15.196:0/1891779190 shutdown_connections 2024-04-28T13:33:07.528 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.527+0000 7fa3b3e00640 1 --2- 172.21.15.196:0/1891779190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b4157360 0x7fa3b406dc70 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.528 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.527+0000 7fa3b3e00640 1 -- 172.21.15.196:0/1891779190 >> 172.21.15.196:0/1891779190 conn(0x7fa3b406c6c0 msgr2=0x7fa3b406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:07.528 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.527+0000 7fa3b3e00640 1 -- 172.21.15.196:0/1891779190 shutdown_connections 2024-04-28T13:33:07.528 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.527+0000 7fa3b3e00640 1 -- 172.21.15.196:0/1891779190 wait complete. 2024-04-28T13:33:07.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.529+0000 7fa3b3e00640 1 Processor -- start 2024-04-28T13:33:07.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.529+0000 7fa3b3e00640 1 -- start start 2024-04-28T13:33:07.532 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.530+0000 7fa3b3e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3b40b59b0 0x7fa3b40b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:07.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.530+0000 7fa3b3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b40d1760 0x7fa3b40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:07.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.530+0000 7fa3b3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3b4157360 0x7fa3b414c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:07.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.530+0000 7fa3b3e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4157740 con 0x7fa3b40d1760 2024-04-28T13:33:07.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.530+0000 7fa3b3e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b415d450 con 0x7fa3b4157360 2024-04-28T13:33:07.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.530+0000 7fa3b3e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa3b41578c0 con 0x7fa3b40b59b0 2024-04-28T13:33:07.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.530+0000 7fa3b2000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b40d1760 0x7fa3b40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:07.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.530+0000 7fa3b2000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b40d1760 0x7fa3b40b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54556/0 (socket says 172.21.15.196:54556) 2024-04-28T13:33:07.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.530+0000 7fa3b2000640 1 -- 172.21.15.196:0/540799686 learned_addr learned my addr 172.21.15.196:0/540799686 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:07.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.531+0000 7fa3b2a00640 1 --2- 172.21.15.196:0/540799686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3b40b59b0 0x7fa3b40b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:07.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.531+0000 7fa3b3400640 1 --2- 172.21.15.196:0/540799686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3b4157360 0x7fa3b414c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:07.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.531+0000 7fa3b2000640 1 -- 172.21.15.196:0/540799686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3b40b59b0 msgr2=0x7fa3b40b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:07.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.531+0000 7fa3b2000640 1 --2- 172.21.15.196:0/540799686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3b40b59b0 0x7fa3b40b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.531+0000 7fa3b2000640 1 -- 172.21.15.196:0/540799686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3b4157360 msgr2=0x7fa3b414c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:07.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.531+0000 7fa3b2000640 1 --2- 172.21.15.196:0/540799686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3b4157360 0x7fa3b414c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.531+0000 7fa3b2000640 1 -- 172.21.15.196:0/540799686 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b40b26b0 con 0x7fa3b40d1760 2024-04-28T13:33:07.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.531+0000 7fa3b3400640 1 --2- 172.21.15.196:0/540799686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3b4157360 0x7fa3b414c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:07.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.531+0000 7fa3b2000640 1 --2- 172.21.15.196:0/540799686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b40d1760 0x7fa3b40b2170 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c400 tx=0x7fa39c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:07.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.531+0000 7fa3aaa00640 1 -- 172.21.15.196:0/540799686 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c017070 con 0x7fa3b40d1760 2024-04-28T13:33:07.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.532+0000 7fa3aaa00640 1 -- 172.21.15.196:0/540799686 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa39c005ce0 con 0x7fa3b40d1760 2024-04-28T13:33:07.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.532+0000 7fa3aaa00640 1 -- 172.21.15.196:0/540799686 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c0072f0 con 0x7fa3b40d1760 2024-04-28T13:33:07.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.532+0000 7fa3b3e00640 1 -- 172.21.15.196:0/540799686 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b40b09f0 con 0x7fa3b40d1760 2024-04-28T13:33:07.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.532+0000 7fa3b3e00640 1 -- 172.21.15.196:0/540799686 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b40b0fb0 con 0x7fa3b40d1760 2024-04-28T13:33:07.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.533+0000 7fa3b3e00640 1 -- 172.21.15.196:0/540799686 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b406a290 con 0x7fa3b40d1760 2024-04-28T13:33:07.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.534+0000 7fa3aaa00640 1 -- 172.21.15.196:0/540799686 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa39c01a070 con 0x7fa3b40d1760 2024-04-28T13:33:07.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.535+0000 7fa3aaa00640 1 --2- 172.21.15.196:0/540799686 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa39006c980 0x7fa39006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:07.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.535+0000 7fa3aaa00640 1 -- 172.21.15.196:0/540799686 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fa39c08aff0 con 0x7fa3b40d1760 2024-04-28T13:33:07.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.535+0000 7fa3b2a00640 1 --2- 172.21.15.196:0/540799686 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa39006c980 0x7fa39006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:07.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.537+0000 7fa3b2a00640 1 --2- 172.21.15.196:0/540799686 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa39006c980 0x7fa39006ee40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4034000 tx=0x7fa3a402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:07.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.540+0000 7fa3aaa00640 1 -- 172.21.15.196:0/540799686 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa39c0593c0 con 0x7fa3b40d1760 2024-04-28T13:33:07.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.737+0000 7fa3b3e00640 1 -- 172.21.15.196:0/540799686 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3b406f720 con 0x7fa3b40d1760 2024-04-28T13:33:07.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.739+0000 7fa3aaa00640 1 -- 172.21.15.196:0/540799686 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16834 (secure 0 0 0) 0x7fa39c082e20 con 0x7fa3b40d1760 2024-04-28T13:33:07.740 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:07.741 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":34,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:58.527505+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:32:29.352348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-28T13:32:30.637498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6836/459640695":"2024-04-29T13:32:58.527451+0000","172.21.15.203:6835/459640695":"2024-04-29T13:32:58.527451+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:33:07.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.742+0000 7fa3b3e00640 1 -- 172.21.15.196:0/540799686 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa39006c980 msgr2=0x7fa39006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:07.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.742+0000 7fa3b3e00640 1 --2- 172.21.15.196:0/540799686 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa39006c980 0x7fa39006ee40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4034000 tx=0x7fa3a402e000 comp rx=0 tx=0).stop 2024-04-28T13:33:07.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.742+0000 7fa3b3e00640 1 -- 172.21.15.196:0/540799686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b40d1760 msgr2=0x7fa3b40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:07.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.742+0000 7fa3b3e00640 1 --2- 172.21.15.196:0/540799686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b40d1760 0x7fa3b40b2170 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c400 tx=0x7fa39c00c8c0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.742+0000 7fa3b3e00640 1 -- 172.21.15.196:0/540799686 shutdown_connections 2024-04-28T13:33:07.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.742+0000 7fa3b3e00640 1 --2- 172.21.15.196:0/540799686 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa39006c980 0x7fa39006ee40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.742+0000 7fa3b3e00640 1 --2- 172.21.15.196:0/540799686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3b4157360 0x7fa3b414c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.742+0000 7fa3b3e00640 1 --2- 172.21.15.196:0/540799686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b40d1760 0x7fa3b40b2170 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.742+0000 7fa3b3e00640 1 --2- 172.21.15.196:0/540799686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3b40b59b0 0x7fa3b40b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.743+0000 7fa3b3e00640 1 -- 172.21.15.196:0/540799686 >> 172.21.15.196:0/540799686 conn(0x7fa3b406c6c0 msgr2=0x7fa3b40b4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:07.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.743+0000 7fa3b3e00640 1 -- 172.21.15.196:0/540799686 shutdown_connections 2024-04-28T13:33:07.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.743+0000 7fa3b3e00640 1 -- 172.21.15.196:0/540799686 wait complete. 2024-04-28T13:33:07.761 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:32:29.352348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:32:30.637498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:33:07.762 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:33:07.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.963+0000 7fcb35a00640 1 Processor -- start 2024-04-28T13:33:07.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.963+0000 7fcb35a00640 1 -- start start 2024-04-28T13:33:07.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.964+0000 7fcb35a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcb300b65f0 0x7fcb300b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:07.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.965+0000 7fcb35a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb3006fa50 0x7fcb300ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:07.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.965+0000 7fcb35a00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb30073440 con 0x7fcb30163b40 2024-04-28T13:33:07.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.965+0000 7fcb35a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcb300b6f10 con 0x7fcb300b65f0 2024-04-28T13:33:07.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.965+0000 7fcb35a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcb300723a0 con 0x7fcb3006fa50 2024-04-28T13:33:07.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.965+0000 7fcb2ea00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcb300b65f0 0x7fcb300b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:07.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.965+0000 7fcb2ea00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcb300b65f0 0x7fcb300b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54570/0 (socket says 172.21.15.196:54570) 2024-04-28T13:33:07.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.965+0000 7fcb2ea00640 1 -- 172.21.15.196:0/3841754027 learned_addr learned my addr 172.21.15.196:0/3841754027 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:07.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.965+0000 7fcb2f400640 1 --2- 172.21.15.196:0/3841754027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb3006fa50 0x7fcb300ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:07.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.965+0000 7fcb2ea00640 1 -- 172.21.15.196:0/3841754027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb3006fa50 msgr2=0x7fcb300ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:07.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.966+0000 7fcb2ea00640 1 --2- 172.21.15.196:0/3841754027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb3006fa50 0x7fcb300ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.966+0000 7fcb2ea00640 1 -- 172.21.15.196:0/3841754027 >> v1:172.21.15.203:6789/0 conn(0x7fcb30163b40 legacy=0x7fcb3006f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:07.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.966+0000 7fcb2ea00640 1 -- 172.21.15.196:0/3841754027 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb30072680 con 0x7fcb300b65f0 2024-04-28T13:33:07.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.966+0000 7fcb2f400640 1 --2- 172.21.15.196:0/3841754027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb3006fa50 0x7fcb300ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:07.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.967+0000 7fcb2ea00640 1 --2- 172.21.15.196:0/3841754027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb300b65f0 0x7fcb300b69d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009870 tx=0x7fcb2002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce0f29ff5306eb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:07.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.967+0000 7fcb2d600640 1 -- 172.21.15.196:0/3841754027 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20037070 con 0x7fcb300b65f0 2024-04-28T13:33:07.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.967+0000 7fcb2d600640 1 -- 172.21.15.196:0/3841754027 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb20032440 con 0x7fcb300b65f0 2024-04-28T13:33:07.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.967+0000 7fcb2d600640 1 -- 172.21.15.196:0/3841754027 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20032810 con 0x7fcb300b65f0 2024-04-28T13:33:07.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.968+0000 7fcb35a00640 1 -- 172.21.15.196:0/3841754027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb300b65f0 msgr2=0x7fcb300b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:07.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.968+0000 7fcb35a00640 1 --2- 172.21.15.196:0/3841754027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb300b65f0 0x7fcb300b69d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009870 tx=0x7fcb2002a180 comp rx=0 tx=0).stop 2024-04-28T13:33:07.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.968+0000 7fcb35a00640 1 -- 172.21.15.196:0/3841754027 shutdown_connections 2024-04-28T13:33:07.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.968+0000 7fcb35a00640 1 --2- 172.21.15.196:0/3841754027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb3006fa50 0x7fcb300ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.968+0000 7fcb35a00640 1 --2- 172.21.15.196:0/3841754027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb300b65f0 0x7fcb300b69d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.968+0000 7fcb35a00640 1 -- 172.21.15.196:0/3841754027 >> 172.21.15.196:0/3841754027 conn(0x7fcb300cd8a0 msgr2=0x7fcb300cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:07.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.968+0000 7fcb35a00640 1 -- 172.21.15.196:0/3841754027 shutdown_connections 2024-04-28T13:33:07.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.968+0000 7fcb35a00640 1 -- 172.21.15.196:0/3841754027 wait complete. 2024-04-28T13:33:07.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.969+0000 7fcb35a00640 1 Processor -- start 2024-04-28T13:33:07.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.970+0000 7fcb35a00640 1 -- start start 2024-04-28T13:33:07.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.970+0000 7fcb35a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb3006fa50 0x7fcb3014cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:07.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb35a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb300b65f0 0x7fcb301591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:07.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb35a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb30163b40 0x7fcb3015dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:07.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb35a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb300cf1e0 con 0x7fcb300b65f0 2024-04-28T13:33:07.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb35a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcb30072c40 con 0x7fcb3006fa50 2024-04-28T13:33:07.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb35a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcb300cf360 con 0x7fcb30163b40 2024-04-28T13:33:07.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb2e000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb300b65f0 0x7fcb301591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:07.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb2e000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb300b65f0 0x7fcb301591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54584/0 (socket says 172.21.15.196:54584) 2024-04-28T13:33:07.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb2e000640 1 -- 172.21.15.196:0/2311044870 learned_addr learned my addr 172.21.15.196:0/2311044870 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:07.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb2ea00640 1 --2- 172.21.15.196:0/2311044870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb3006fa50 0x7fcb3014cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:07.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb2f400640 1 --2- 172.21.15.196:0/2311044870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb30163b40 0x7fcb3015dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:07.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb2e000640 1 -- 172.21.15.196:0/2311044870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb30163b40 msgr2=0x7fcb3015dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:07.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb2e000640 1 --2- 172.21.15.196:0/2311044870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb30163b40 0x7fcb3015dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb2e000640 1 -- 172.21.15.196:0/2311044870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb3006fa50 msgr2=0x7fcb3014cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:07.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb2e000640 1 --2- 172.21.15.196:0/2311044870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb3006fa50 0x7fcb3014cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:07.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb2e000640 1 -- 172.21.15.196:0/2311044870 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3014d4a0 con 0x7fcb300b65f0 2024-04-28T13:33:07.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.971+0000 7fcb2ea00640 1 --2- 172.21.15.196:0/2311044870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb3006fa50 0x7fcb3014cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:07.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.972+0000 7fcb2e000640 1 --2- 172.21.15.196:0/2311044870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb300b65f0 0x7fcb301591e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009fd0 tx=0x7fcb1c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:07.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.972+0000 7fcb26a00640 1 -- 172.21.15.196:0/2311044870 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c015020 con 0x7fcb300b65f0 2024-04-28T13:33:07.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.972+0000 7fcb26a00640 1 -- 172.21.15.196:0/2311044870 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb1c011830 con 0x7fcb300b65f0 2024-04-28T13:33:07.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.972+0000 7fcb26a00640 1 -- 172.21.15.196:0/2311044870 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c011c00 con 0x7fcb300b65f0 2024-04-28T13:33:07.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.972+0000 7fcb35a00640 1 -- 172.21.15.196:0/2311044870 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3015d310 con 0x7fcb300b65f0 2024-04-28T13:33:07.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.972+0000 7fcb35a00640 1 -- 172.21.15.196:0/2311044870 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb30164480 con 0x7fcb300b65f0 2024-04-28T13:33:07.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.973+0000 7fcb35a00640 1 -- 172.21.15.196:0/2311044870 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb30163fb0 con 0x7fcb300b65f0 2024-04-28T13:33:07.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.974+0000 7fcb26a00640 1 -- 172.21.15.196:0/2311044870 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fcb1c018020 con 0x7fcb300b65f0 2024-04-28T13:33:07.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.975+0000 7fcb26a00640 1 --2- 172.21.15.196:0/2311044870 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcb1006c980 0x7fcb1006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:07.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.976+0000 7fcb26a00640 1 -- 172.21.15.196:0/2311044870 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fcb1c08bb50 con 0x7fcb300b65f0 2024-04-28T13:33:07.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.976+0000 7fcb2ea00640 1 --2- 172.21.15.196:0/2311044870 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcb1006c980 0x7fcb1006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:07.978 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.977+0000 7fcb2ea00640 1 --2- 172.21.15.196:0/2311044870 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcb1006c980 0x7fcb1006ee40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcb20034000 tx=0x7fcb2002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:07.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:07.980+0000 7fcb26a00640 1 -- 172.21.15.196:0/2311044870 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcb1c01f020 con 0x7fcb300b65f0 2024-04-28T13:33:08.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.177+0000 7fcb35a00640 1 -- 172.21.15.196:0/2311044870 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcb30073750 con 0x7fcb300b65f0 2024-04-28T13:33:08.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.178+0000 7fcb26a00640 1 -- 172.21.15.196:0/2311044870 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7fcb1c0172c0 con 0x7fcb300b65f0 2024-04-28T13:33:08.179 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:33:08.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.181+0000 7fcb35a00640 1 -- 172.21.15.196:0/2311044870 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcb1006c980 msgr2=0x7fcb1006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:08.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.181+0000 7fcb35a00640 1 --2- 172.21.15.196:0/2311044870 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcb1006c980 0x7fcb1006ee40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcb20034000 tx=0x7fcb2002e000 comp rx=0 tx=0).stop 2024-04-28T13:33:08.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.181+0000 7fcb35a00640 1 -- 172.21.15.196:0/2311044870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb300b65f0 msgr2=0x7fcb301591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:08.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.181+0000 7fcb35a00640 1 --2- 172.21.15.196:0/2311044870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb300b65f0 0x7fcb301591e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009fd0 tx=0x7fcb1c00b870 comp rx=0 tx=0).stop 2024-04-28T13:33:08.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.181+0000 7fcb35a00640 1 -- 172.21.15.196:0/2311044870 shutdown_connections 2024-04-28T13:33:08.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.181+0000 7fcb35a00640 1 --2- 172.21.15.196:0/2311044870 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcb1006c980 0x7fcb1006ee40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.181+0000 7fcb35a00640 1 --2- 172.21.15.196:0/2311044870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb30163b40 0x7fcb3015dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.181+0000 7fcb35a00640 1 --2- 172.21.15.196:0/2311044870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb300b65f0 0x7fcb301591e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.181+0000 7fcb35a00640 1 --2- 172.21.15.196:0/2311044870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb3006fa50 0x7fcb3014cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.181+0000 7fcb35a00640 1 -- 172.21.15.196:0/2311044870 >> 172.21.15.196:0/2311044870 conn(0x7fcb300cd8a0 msgr2=0x7fcb300b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:08.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.181+0000 7fcb35a00640 1 -- 172.21.15.196:0/2311044870 shutdown_connections 2024-04-28T13:33:08.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.181+0000 7fcb35a00640 1 -- 172.21.15.196:0/2311044870 wait complete. 2024-04-28T13:33:08.201 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:33:08.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.401+0000 7f6277a00640 1 Processor -- start 2024-04-28T13:33:08.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.401+0000 7f6277a00640 1 -- start start 2024-04-28T13:33:08.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.402+0000 7f6277a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6270145960 0x7f62700af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:08.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.402+0000 7f6277a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62700afdb0 0x7f62700b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:08.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.403+0000 7f6277a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f627015fa10 0x7f627015fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:08.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.403+0000 7f6277a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f62701607d0 con 0x7f6270145960 2024-04-28T13:33:08.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.403+0000 7f6277a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f62700aea50 con 0x7f627015fa10 2024-04-28T13:33:08.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.403+0000 7f6277a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f62700aed30 con 0x7f62700afdb0 2024-04-28T13:33:08.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.403+0000 7f6275000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6270145960 0x7f62700af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:08.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.403+0000 7f6275000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6270145960 0x7f62700af530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54588/0 (socket says 172.21.15.196:54588) 2024-04-28T13:33:08.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.403+0000 7f6275000640 1 -- 172.21.15.196:0/2857237938 learned_addr learned my addr 172.21.15.196:0/2857237938 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:08.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.403+0000 7f6275000640 1 -- 172.21.15.196:0/2857237938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62700afdb0 msgr2=0x7f62700b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:08.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.403+0000 7f6267e00640 1 --2- 172.21.15.196:0/2857237938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62700afdb0 0x7f62700b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:08.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.403+0000 7f6275a00640 1 --2- 172.21.15.196:0/2857237938 >> v2:172.21.15.203:3300/0 conn(0x7f627015fa10 0x7f627015fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:08.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.403+0000 7f6275000640 1 --2- 172.21.15.196:0/2857237938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62700afdb0 0x7f62700b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.403+0000 7f6275000640 1 -- 172.21.15.196:0/2857237938 >> v2:172.21.15.203:3300/0 conn(0x7f627015fa10 msgr2=0x7f627015fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:08.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.403+0000 7f6275000640 1 --2- 172.21.15.196:0/2857237938 >> v2:172.21.15.203:3300/0 conn(0x7f627015fa10 0x7f627015fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.403+0000 7f6275000640 1 -- 172.21.15.196:0/2857237938 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62700af010 con 0x7f6270145960 2024-04-28T13:33:08.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.403+0000 7f6267e00640 1 --2- 172.21.15.196:0/2857237938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62700afdb0 0x7f62700b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:08.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.404+0000 7f6275000640 1 --2- 172.21.15.196:0/2857237938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6270145960 0x7f62700af530 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f62700d0610 tx=0x7f626002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1dc9df5eea1799 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:08.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.405+0000 7f626fe00640 1 -- 172.21.15.196:0/2857237938 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260037070 con 0x7f6270145960 2024-04-28T13:33:08.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.405+0000 7f626fe00640 1 -- 172.21.15.196:0/2857237938 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6260032440 con 0x7f6270145960 2024-04-28T13:33:08.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.405+0000 7f626fe00640 1 -- 172.21.15.196:0/2857237938 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260032810 con 0x7f6270145960 2024-04-28T13:33:08.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.405+0000 7f6277a00640 1 -- 172.21.15.196:0/2857237938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6270145960 msgr2=0x7f62700af530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:08.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.405+0000 7f6277a00640 1 --2- 172.21.15.196:0/2857237938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6270145960 0x7f62700af530 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f62700d0610 tx=0x7f626002a1e0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.406+0000 7f6277a00640 1 -- 172.21.15.196:0/2857237938 shutdown_connections 2024-04-28T13:33:08.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.406+0000 7f6277a00640 1 --2- 172.21.15.196:0/2857237938 >> v2:172.21.15.203:3300/0 conn(0x7f627015fa10 0x7f627015fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.406+0000 7f6277a00640 1 --2- 172.21.15.196:0/2857237938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62700afdb0 0x7f62700b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.406+0000 7f6277a00640 1 --2- 172.21.15.196:0/2857237938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6270145960 0x7f62700af530 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.406+0000 7f6277a00640 1 -- 172.21.15.196:0/2857237938 >> 172.21.15.196:0/2857237938 conn(0x7f62700cd8a0 msgr2=0x7f62700cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:08.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.406+0000 7f6277a00640 1 -- 172.21.15.196:0/2857237938 shutdown_connections 2024-04-28T13:33:08.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.406+0000 7f6277a00640 1 -- 172.21.15.196:0/2857237938 wait complete. 2024-04-28T13:33:08.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.407+0000 7f6277a00640 1 Processor -- start 2024-04-28T13:33:08.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.407+0000 7f6277a00640 1 -- start start 2024-04-28T13:33:08.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.408+0000 7f6277a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62700afdb0 0x7f62700cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:08.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.408+0000 7f6277a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6270145960 0x7f627015e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:08.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.408+0000 7f6277a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f627015fa10 0x7f627015e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:08.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.408+0000 7f6277a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6270160560 con 0x7f62700afdb0 2024-04-28T13:33:08.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.408+0000 7f6277a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f62701603e0 con 0x7f6270145960 2024-04-28T13:33:08.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.408+0000 7f6277a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6270073370 con 0x7f627015fa10 2024-04-28T13:33:08.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.409+0000 7f6275000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62700afdb0 0x7f62700cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:08.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.409+0000 7f6275000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62700afdb0 0x7f62700cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54596/0 (socket says 172.21.15.196:54596) 2024-04-28T13:33:08.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.409+0000 7f6275000640 1 -- 172.21.15.196:0/1700739128 learned_addr learned my addr 172.21.15.196:0/1700739128 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:08.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.409+0000 7f6275a00640 1 --2- 172.21.15.196:0/1700739128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f627015fa10 0x7f627015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:08.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.409+0000 7f6267e00640 1 --2- 172.21.15.196:0/1700739128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6270145960 0x7f627015e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:08.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.409+0000 7f6275000640 1 -- 172.21.15.196:0/1700739128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f627015fa10 msgr2=0x7f627015e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:08.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.409+0000 7f6275000640 1 --2- 172.21.15.196:0/1700739128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f627015fa10 0x7f627015e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.409+0000 7f6275000640 1 -- 172.21.15.196:0/1700739128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6270145960 msgr2=0x7f627015e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:08.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.409+0000 7f6275000640 1 --2- 172.21.15.196:0/1700739128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6270145960 0x7f627015e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.409+0000 7f6275000640 1 -- 172.21.15.196:0/1700739128 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62700c7690 con 0x7f62700afdb0 2024-04-28T13:33:08.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.409+0000 7f6275a00640 1 --2- 172.21.15.196:0/1700739128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f627015fa10 0x7f627015e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:08.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.410+0000 7f6275000640 1 --2- 172.21.15.196:0/1700739128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62700afdb0 0x7f62700cad40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f626002a220 tx=0x7f62600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:08.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.410+0000 7f626d600640 1 -- 172.21.15.196:0/1700739128 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260041070 con 0x7f62700afdb0 2024-04-28T13:33:08.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.410+0000 7f626d600640 1 -- 172.21.15.196:0/1700739128 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f626003c870 con 0x7f62700afdb0 2024-04-28T13:33:08.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.410+0000 7f6277a00640 1 -- 172.21.15.196:0/1700739128 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62700c7970 con 0x7f62700afdb0 2024-04-28T13:33:08.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.410+0000 7f626d600640 1 -- 172.21.15.196:0/1700739128 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260037040 con 0x7f62700afdb0 2024-04-28T13:33:08.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.410+0000 7f6277a00640 1 -- 172.21.15.196:0/1700739128 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6270135490 con 0x7f62700afdb0 2024-04-28T13:33:08.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.411+0000 7f6277a00640 1 -- 172.21.15.196:0/1700739128 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f627006a290 con 0x7f62700afdb0 2024-04-28T13:33:08.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.412+0000 7f626d600640 1 -- 172.21.15.196:0/1700739128 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6260044070 con 0x7f62700afdb0 2024-04-28T13:33:08.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.413+0000 7f626d600640 1 --2- 172.21.15.196:0/1700739128 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f624c06c980 0x7f624c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:08.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.413+0000 7f626d600640 1 -- 172.21.15.196:0/1700739128 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f62600ac730 con 0x7f62700afdb0 2024-04-28T13:33:08.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.414+0000 7f6267e00640 1 --2- 172.21.15.196:0/1700739128 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f624c06c980 0x7f624c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:08.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.415+0000 7f6267e00640 1 --2- 172.21.15.196:0/1700739128 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f624c06c980 0x7f624c06ee40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6258006fd0 tx=0x7f6258008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:08.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.417+0000 7f626d600640 1 -- 172.21.15.196:0/1700739128 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f626007aba0 con 0x7f62700afdb0 2024-04-28T13:33:08.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.618+0000 7f6277a00640 1 -- 172.21.15.196:0/1700739128 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f62701601a0 con 0x7f62700afdb0 2024-04-28T13:33:08.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.619+0000 7f626d600640 1 -- 172.21.15.196:0/1700739128 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f626007a4a0 con 0x7f62700afdb0 2024-04-28T13:33:08.620 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:33:08.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.621+0000 7f6277a00640 1 -- 172.21.15.196:0/1700739128 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f624c06c980 msgr2=0x7f624c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:08.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.621+0000 7f6277a00640 1 --2- 172.21.15.196:0/1700739128 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f624c06c980 0x7f624c06ee40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6258006fd0 tx=0x7f6258008040 comp rx=0 tx=0).stop 2024-04-28T13:33:08.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.621+0000 7f6277a00640 1 -- 172.21.15.196:0/1700739128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62700afdb0 msgr2=0x7f62700cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:08.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.621+0000 7f6277a00640 1 --2- 172.21.15.196:0/1700739128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62700afdb0 0x7f62700cad40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f626002a220 tx=0x7f62600046c0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.621+0000 7f6277a00640 1 -- 172.21.15.196:0/1700739128 shutdown_connections 2024-04-28T13:33:08.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.621+0000 7f6277a00640 1 --2- 172.21.15.196:0/1700739128 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f624c06c980 0x7f624c06ee40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.621+0000 7f6277a00640 1 --2- 172.21.15.196:0/1700739128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f627015fa10 0x7f627015e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.621+0000 7f6277a00640 1 --2- 172.21.15.196:0/1700739128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6270145960 0x7f627015e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.621+0000 7f6277a00640 1 --2- 172.21.15.196:0/1700739128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62700afdb0 0x7f62700cad40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.621+0000 7f6277a00640 1 -- 172.21.15.196:0/1700739128 >> 172.21.15.196:0/1700739128 conn(0x7f62700cd8a0 msgr2=0x7f627006e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:08.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.621+0000 7f6277a00640 1 -- 172.21.15.196:0/1700739128 shutdown_connections 2024-04-28T13:33:08.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.622+0000 7f6277a00640 1 -- 172.21.15.196:0/1700739128 wait complete. 2024-04-28T13:33:08.640 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:33:08.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.836+0000 7f3096c00640 1 Processor -- start 2024-04-28T13:33:08.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.837+0000 7f3096c00640 1 -- start start 2024-04-28T13:33:08.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.838+0000 7f3096c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f30900b3c20 0x7f309006db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:08.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.838+0000 7f3096c00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f30900b33e0 0x7f3090163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:08.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.838+0000 7f3096c00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f309009c700 con 0x7f30900d0010 2024-04-28T13:33:08.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.838+0000 7f3096c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f30900b4480 con 0x7f30900b3c20 2024-04-28T13:33:08.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.838+0000 7f3096c00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f309006e050 con 0x7f30900b33e0 2024-04-28T13:33:08.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.839+0000 7f308f400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f30900b3c20 0x7f309006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:08.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.839+0000 7f308f400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f30900b3c20 0x7f309006db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54600/0 (socket says 172.21.15.196:54600) 2024-04-28T13:33:08.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.839+0000 7f308f400640 1 -- 172.21.15.196:0/932422663 learned_addr learned my addr 172.21.15.196:0/932422663 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:08.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.839+0000 7f3094c00640 1 --2- 172.21.15.196:0/932422663 >> v2:172.21.15.203:3300/0 conn(0x7f30900b33e0 0x7f3090163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:08.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.839+0000 7f308f400640 1 -- 172.21.15.196:0/932422663 >> v2:172.21.15.203:3300/0 conn(0x7f30900b33e0 msgr2=0x7f3090163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:08.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.839+0000 7f308f400640 1 --2- 172.21.15.196:0/932422663 >> v2:172.21.15.203:3300/0 conn(0x7f30900b33e0 0x7f3090163280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.839+0000 7f308f400640 1 -- 172.21.15.196:0/932422663 >> v1:172.21.15.203:6789/0 conn(0x7f30900d0010 legacy=0x7f30900b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:08.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.839+0000 7f308f400640 1 -- 172.21.15.196:0/932422663 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30900d1c50 con 0x7f30900b3c20 2024-04-28T13:33:08.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.840+0000 7f308f400640 1 --2- 172.21.15.196:0/932422663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30900b3c20 0x7f309006db10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3090073bb0 tx=0x7f308002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa02cdfc143a3a6c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:08.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.840+0000 7f308ea00640 1 -- 172.21.15.196:0/932422663 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3080036070 con 0x7f30900b3c20 2024-04-28T13:33:08.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.840+0000 7f308ea00640 1 -- 172.21.15.196:0/932422663 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3080031600 con 0x7f30900b3c20 2024-04-28T13:33:08.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.841+0000 7f308ea00640 1 -- 172.21.15.196:0/932422663 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30800319d0 con 0x7f30900b3c20 2024-04-28T13:33:08.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.841+0000 7f3096c00640 1 -- 172.21.15.196:0/932422663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30900b3c20 msgr2=0x7f309006db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:08.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.841+0000 7f3096c00640 1 --2- 172.21.15.196:0/932422663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30900b3c20 0x7f309006db10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3090073bb0 tx=0x7f308002a6f0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.841+0000 7f3096c00640 1 -- 172.21.15.196:0/932422663 shutdown_connections 2024-04-28T13:33:08.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.841+0000 7f3096c00640 1 --2- 172.21.15.196:0/932422663 >> v2:172.21.15.203:3300/0 conn(0x7f30900b33e0 0x7f3090163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.841+0000 7f3096c00640 1 --2- 172.21.15.196:0/932422663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30900b3c20 0x7f309006db10 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.841+0000 7f3096c00640 1 -- 172.21.15.196:0/932422663 >> 172.21.15.196:0/932422663 conn(0x7f309009ba40 msgr2=0x7f309009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:08.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.841+0000 7f3096c00640 1 -- 172.21.15.196:0/932422663 shutdown_connections 2024-04-28T13:33:08.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.841+0000 7f3096c00640 1 -- 172.21.15.196:0/932422663 wait complete. 2024-04-28T13:33:08.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.843+0000 7f3096c00640 1 Processor -- start 2024-04-28T13:33:08.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.843+0000 7f3096c00640 1 -- start start 2024-04-28T13:33:08.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.844+0000 7f3096c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30900b33e0 0x7f309009d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:08.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.844+0000 7f3096c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30900b3c20 0x7f30901f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:08.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.844+0000 7f3096c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30900d0010 0x7f3090161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:08.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.844+0000 7f3096c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30900b3990 con 0x7f30900b3c20 2024-04-28T13:33:08.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.844+0000 7f3096c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f30900b3810 con 0x7f30900d0010 2024-04-28T13:33:08.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.844+0000 7f3096c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f30901637c0 con 0x7f30900b33e0 2024-04-28T13:33:08.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.844+0000 7f308f400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30900b3c20 0x7f30901f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:08.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.844+0000 7f308f400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30900b3c20 0x7f30901f5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54602/0 (socket says 172.21.15.196:54602) 2024-04-28T13:33:08.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.844+0000 7f308f400640 1 -- 172.21.15.196:0/2023918662 learned_addr learned my addr 172.21.15.196:0/2023918662 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:08.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.845+0000 7f308f400640 1 -- 172.21.15.196:0/2023918662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30900b33e0 msgr2=0x7f309009d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:08.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.845+0000 7f3094c00640 1 --2- 172.21.15.196:0/2023918662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30900d0010 0x7f3090161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:08.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.845+0000 7f308fe00640 1 --2- 172.21.15.196:0/2023918662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30900b33e0 0x7f309009d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:08.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.845+0000 7f308f400640 1 --2- 172.21.15.196:0/2023918662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30900b33e0 0x7f309009d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.845+0000 7f308f400640 1 -- 172.21.15.196:0/2023918662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30900d0010 msgr2=0x7f3090161ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:08.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.845+0000 7f308f400640 1 --2- 172.21.15.196:0/2023918662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30900d0010 0x7f3090161ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:08.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.845+0000 7f308f400640 1 -- 172.21.15.196:0/2023918662 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309009dfd0 con 0x7f30900b3c20 2024-04-28T13:33:08.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.845+0000 7f3094c00640 1 --2- 172.21.15.196:0/2023918662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30900d0010 0x7f3090161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:08.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.845+0000 7f308f400640 1 --2- 172.21.15.196:0/2023918662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30900b3c20 0x7f30901f5340 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3080034040 tx=0x7f3080004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:08.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.845+0000 7f3087e00640 1 -- 172.21.15.196:0/2023918662 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3080036050 con 0x7f30900b3c20 2024-04-28T13:33:08.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.845+0000 7f3087e00640 1 -- 172.21.15.196:0/2023918662 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3080042070 con 0x7f30900b3c20 2024-04-28T13:33:08.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.846+0000 7f3087e00640 1 -- 172.21.15.196:0/2023918662 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3080009510 con 0x7f30900b3c20 2024-04-28T13:33:08.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.846+0000 7f3096c00640 1 -- 172.21.15.196:0/2023918662 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309009e2b0 con 0x7f30900b3c20 2024-04-28T13:33:08.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.846+0000 7f3096c00640 1 -- 172.21.15.196:0/2023918662 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f309009e870 con 0x7f30900b3c20 2024-04-28T13:33:08.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.847+0000 7f3096c00640 1 -- 172.21.15.196:0/2023918662 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f309006a290 con 0x7f30900b3c20 2024-04-28T13:33:08.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.848+0000 7f3087e00640 1 -- 172.21.15.196:0/2023918662 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f3080007c30 con 0x7f30900b3c20 2024-04-28T13:33:08.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.849+0000 7f3087e00640 1 --2- 172.21.15.196:0/2023918662 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f306c06c980 0x7f306c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:08.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.849+0000 7f3087e00640 1 -- 172.21.15.196:0/2023918662 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f30800acbf0 con 0x7f30900b3c20 2024-04-28T13:33:08.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.849+0000 7f308fe00640 1 --2- 172.21.15.196:0/2023918662 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f306c06c980 0x7f306c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:08.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.851+0000 7f308fe00640 1 --2- 172.21.15.196:0/2023918662 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f306c06c980 0x7f306c06ee40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f307800a730 tx=0x7f3078008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:08.854 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:08.853+0000 7f3087e00640 1 -- 172.21.15.196:0/2023918662 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3080045360 con 0x7f30900b3c20 2024-04-28T13:33:09.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.053+0000 7f3096c00640 1 -- 172.21.15.196:0/2023918662 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3090068e00 con 0x7f30900b3c20 2024-04-28T13:33:09.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.054+0000 7f3087e00640 1 -- 172.21.15.196:0/2023918662 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f308007af50 con 0x7f30900b3c20 2024-04-28T13:33:09.055 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:33:09.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.056+0000 7f3096c00640 1 -- 172.21.15.196:0/2023918662 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f306c06c980 msgr2=0x7f306c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:09.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.056+0000 7f3096c00640 1 --2- 172.21.15.196:0/2023918662 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f306c06c980 0x7f306c06ee40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f307800a730 tx=0x7f3078008040 comp rx=0 tx=0).stop 2024-04-28T13:33:09.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.056+0000 7f3096c00640 1 -- 172.21.15.196:0/2023918662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30900b3c20 msgr2=0x7f30901f5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:09.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.056+0000 7f3096c00640 1 --2- 172.21.15.196:0/2023918662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30900b3c20 0x7f30901f5340 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3080034040 tx=0x7f3080004a40 comp rx=0 tx=0).stop 2024-04-28T13:33:09.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.057+0000 7f3096c00640 1 -- 172.21.15.196:0/2023918662 shutdown_connections 2024-04-28T13:33:09.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.057+0000 7f3096c00640 1 --2- 172.21.15.196:0/2023918662 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f306c06c980 0x7f306c06ee40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.057+0000 7f3096c00640 1 --2- 172.21.15.196:0/2023918662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30900d0010 0x7f3090161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.057+0000 7f3096c00640 1 --2- 172.21.15.196:0/2023918662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30900b3c20 0x7f30901f5340 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.057+0000 7f3096c00640 1 --2- 172.21.15.196:0/2023918662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30900b33e0 0x7f309009d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.057+0000 7f3096c00640 1 -- 172.21.15.196:0/2023918662 >> 172.21.15.196:0/2023918662 conn(0x7f309009ba40 msgr2=0x7f309006ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:09.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.057+0000 7f3096c00640 1 -- 172.21.15.196:0/2023918662 shutdown_connections 2024-04-28T13:33:09.059 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.057+0000 7f3096c00640 1 -- 172.21.15.196:0/2023918662 wait complete. 2024-04-28T13:33:09.076 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T13:33:09.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.272+0000 7fa0e5e00640 1 Processor -- start 2024-04-28T13:33:09.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.273+0000 7fa0e5e00640 1 -- start start 2024-04-28T13:33:09.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.274+0000 7fa0e5e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0e0163780 0x7fa0e006f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:09.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.274+0000 7fa0e5e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa0e0073620 0x7fa0e00d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:09.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.274+0000 7fa0e5e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0e0073ab0 con 0x7fa0e01633a0 2024-04-28T13:33:09.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.274+0000 7fa0e5e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e006dc70 con 0x7fa0e0073620 2024-04-28T13:33:09.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.274+0000 7fa0e5e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e006e230 con 0x7fa0e0163780 2024-04-28T13:33:09.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.274+0000 7fa0e5400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa0e0073620 0x7fa0e00d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:09.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.275+0000 7fa0e5400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa0e0073620 0x7fa0e00d1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54616/0 (socket says 172.21.15.196:54616) 2024-04-28T13:33:09.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.275+0000 7fa0e5400640 1 -- 172.21.15.196:0/611699764 learned_addr learned my addr 172.21.15.196:0/611699764 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:09.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.275+0000 7fa0dfe00640 1 --2- 172.21.15.196:0/611699764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0e0163780 0x7fa0e006f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:09.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.275+0000 7fa0e5400640 1 -- 172.21.15.196:0/611699764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0e0163780 msgr2=0x7fa0e006f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:09.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.275+0000 7fa0e5400640 1 --2- 172.21.15.196:0/611699764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0e0163780 0x7fa0e006f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.275+0000 7fa0e5400640 1 -- 172.21.15.196:0/611699764 >> v1:172.21.15.203:6789/0 conn(0x7fa0e01633a0 legacy=0x7fa0e006e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:09.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.275+0000 7fa0e5400640 1 -- 172.21.15.196:0/611699764 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e0073d50 con 0x7fa0e0073620 2024-04-28T13:33:09.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.276+0000 7fa0e5400640 1 --2- 172.21.15.196:0/611699764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e0073620 0x7fa0e00d1050 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa0e006ddf0 tx=0x7fa0d802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81443d424914cb37 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:09.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.276+0000 7fa0df400640 1 -- 172.21.15.196:0/611699764 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8036070 con 0x7fa0e0073620 2024-04-28T13:33:09.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.276+0000 7fa0df400640 1 -- 172.21.15.196:0/611699764 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d8031600 con 0x7fa0e0073620 2024-04-28T13:33:09.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.276+0000 7fa0df400640 1 -- 172.21.15.196:0/611699764 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d802ae60 con 0x7fa0e0073620 2024-04-28T13:33:09.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.277+0000 7fa0e5e00640 1 -- 172.21.15.196:0/611699764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e0073620 msgr2=0x7fa0e00d1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:09.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.277+0000 7fa0e5e00640 1 --2- 172.21.15.196:0/611699764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e0073620 0x7fa0e00d1050 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa0e006ddf0 tx=0x7fa0d802a680 comp rx=0 tx=0).stop 2024-04-28T13:33:09.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.277+0000 7fa0e5e00640 1 -- 172.21.15.196:0/611699764 shutdown_connections 2024-04-28T13:33:09.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.277+0000 7fa0e5e00640 1 --2- 172.21.15.196:0/611699764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e0073620 0x7fa0e00d1050 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.277+0000 7fa0e5e00640 1 --2- 172.21.15.196:0/611699764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0e0163780 0x7fa0e006f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.277+0000 7fa0e5e00640 1 -- 172.21.15.196:0/611699764 >> 172.21.15.196:0/611699764 conn(0x7fa0e006c6c0 msgr2=0x7fa0e006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:09.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.277+0000 7fa0e5e00640 1 -- 172.21.15.196:0/611699764 shutdown_connections 2024-04-28T13:33:09.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.277+0000 7fa0e5e00640 1 -- 172.21.15.196:0/611699764 wait complete. 2024-04-28T13:33:09.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.279+0000 7fa0e5e00640 1 Processor -- start 2024-04-28T13:33:09.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.279+0000 7fa0e5e00640 1 -- start start 2024-04-28T13:33:09.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.280+0000 7fa0e5e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0e0073620 0x7fa0e0163bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:09.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.280+0000 7fa0e5e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0e01633a0 0x7fa0e0161d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:09.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.280+0000 7fa0e5e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e0163780 0x7fa0e0154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:09.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.280+0000 7fa0e5e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e00d1590 con 0x7fa0e0163780 2024-04-28T13:33:09.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.280+0000 7fa0e5e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e006dec0 con 0x7fa0e0073620 2024-04-28T13:33:09.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.280+0000 7fa0e5e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e00d1710 con 0x7fa0e01633a0 2024-04-28T13:33:09.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.280+0000 7fa0e5400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e0163780 0x7fa0e0154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:09.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.280+0000 7fa0e5400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e0163780 0x7fa0e0154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54628/0 (socket says 172.21.15.196:54628) 2024-04-28T13:33:09.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.280+0000 7fa0e5400640 1 -- 172.21.15.196:0/3763755314 learned_addr learned my addr 172.21.15.196:0/3763755314 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:09.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.280+0000 7fa0e5400640 1 -- 172.21.15.196:0/3763755314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0e01633a0 msgr2=0x7fa0e0161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:09.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.280+0000 7fa0e5400640 1 --2- 172.21.15.196:0/3763755314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0e01633a0 0x7fa0e0161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.280+0000 7fa0e4a00640 1 --2- 172.21.15.196:0/3763755314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0e0073620 0x7fa0e0163bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.280+0000 7fa0e5400640 1 -- 172.21.15.196:0/3763755314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0e0073620 msgr2=0x7fa0e0163bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.280+0000 7fa0e5400640 1 --2- 172.21.15.196:0/3763755314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0e0073620 0x7fa0e0163bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.280+0000 7fa0e5400640 1 -- 172.21.15.196:0/3763755314 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e0164200 con 0x7fa0e0163780 2024-04-28T13:33:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.281+0000 7fa0e5400640 1 --2- 172.21.15.196:0/3763755314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e0163780 0x7fa0e0154d30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa0d802ac70 tx=0x7fa0d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.281+0000 7fa0dcc00640 1 -- 172.21.15.196:0/3763755314 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8041020 con 0x7fa0e0163780 2024-04-28T13:33:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.281+0000 7fa0dcc00640 1 -- 172.21.15.196:0/3763755314 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d802ae60 con 0x7fa0e0163780 2024-04-28T13:33:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.281+0000 7fa0dcc00640 1 -- 172.21.15.196:0/3763755314 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8036040 con 0x7fa0e0163780 2024-04-28T13:33:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.281+0000 7fa0e5e00640 1 -- 172.21.15.196:0/3763755314 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e01644e0 con 0x7fa0e0163780 2024-04-28T13:33:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.281+0000 7fa0e5e00640 1 -- 172.21.15.196:0/3763755314 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e0164aa0 con 0x7fa0e0163780 2024-04-28T13:33:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.282+0000 7fa0e5e00640 1 -- 172.21.15.196:0/3763755314 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e006a290 con 0x7fa0e0163780 2024-04-28T13:33:09.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.283+0000 7fa0dcc00640 1 -- 172.21.15.196:0/3763755314 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa0d8044020 con 0x7fa0e0163780 2024-04-28T13:33:09.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.284+0000 7fa0dcc00640 1 --2- 172.21.15.196:0/3763755314 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa0c406c980 0x7fa0c406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:09.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.284+0000 7fa0dcc00640 1 -- 172.21.15.196:0/3763755314 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fa0d80ac9e0 con 0x7fa0e0163780 2024-04-28T13:33:09.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.285+0000 7fa0e4a00640 1 --2- 172.21.15.196:0/3763755314 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa0c406c980 0x7fa0c406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:09.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.286+0000 7fa0e4a00640 1 --2- 172.21.15.196:0/3763755314 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa0c406c980 0x7fa0c406ee40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4003d90 tx=0x7fa0d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:09.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.288+0000 7fa0dcc00640 1 -- 172.21.15.196:0/3763755314 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa0d804b030 con 0x7fa0e0163780 2024-04-28T13:33:09.500 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.499+0000 7fa0e5e00640 1 -- 172.21.15.196:0/3763755314 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa0e00b4450 con 0x7fa0e0163780 2024-04-28T13:33:09.501 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.500+0000 7fa0dcc00640 1 -- 172.21.15.196:0/3763755314 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v34) ==== 80+0+122 (secure 0 0 0) 0x7fa0d807ab20 con 0x7fa0e0163780 2024-04-28T13:33:09.501 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.203:6836/459640695 2024-04-29T13:32:58.527451+0000 2024-04-28T13:33:09.501 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.203:6835/459640695 2024-04-29T13:32:58.527451+0000 2024-04-28T13:33:09.501 INFO:teuthology.orchestra.run.smithi196.stderr:listed 2 entries 2024-04-28T13:33:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.502+0000 7fa0e5e00640 1 -- 172.21.15.196:0/3763755314 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa0c406c980 msgr2=0x7fa0c406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.502+0000 7fa0e5e00640 1 --2- 172.21.15.196:0/3763755314 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa0c406c980 0x7fa0c406ee40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4003d90 tx=0x7fa0d4009040 comp rx=0 tx=0).stop 2024-04-28T13:33:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.502+0000 7fa0e5e00640 1 -- 172.21.15.196:0/3763755314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e0163780 msgr2=0x7fa0e0154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.503+0000 7fa0e5e00640 1 --2- 172.21.15.196:0/3763755314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e0163780 0x7fa0e0154d30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa0d802ac70 tx=0x7fa0d8004a40 comp rx=0 tx=0).stop 2024-04-28T13:33:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.503+0000 7fa0e5e00640 1 -- 172.21.15.196:0/3763755314 shutdown_connections 2024-04-28T13:33:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.503+0000 7fa0e5e00640 1 --2- 172.21.15.196:0/3763755314 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa0c406c980 0x7fa0c406ee40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.503+0000 7fa0e5e00640 1 --2- 172.21.15.196:0/3763755314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e0163780 0x7fa0e0154d30 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.503+0000 7fa0e5e00640 1 --2- 172.21.15.196:0/3763755314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0e01633a0 0x7fa0e0161d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.503+0000 7fa0e5e00640 1 --2- 172.21.15.196:0/3763755314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0e0073620 0x7fa0e0163bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.503+0000 7fa0e5e00640 1 -- 172.21.15.196:0/3763755314 >> 172.21.15.196:0/3763755314 conn(0x7fa0e006c6c0 msgr2=0x7fa0e00cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.503+0000 7fa0e5e00640 1 -- 172.21.15.196:0/3763755314 shutdown_connections 2024-04-28T13:33:09.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.503+0000 7fa0e5e00640 1 -- 172.21.15.196:0/3763755314 wait complete. 2024-04-28T13:33:09.523 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-28T13:33:09.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:09.523 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-28T13:33:09.619 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:33:09.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:09.620 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-28T13:33:09.670 INFO:tasks.cephfs.mount:Terminating background process 2024-04-28T13:33:09.670 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:33:09.670 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:33:09.725 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:33:09.725 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:33:09.777 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:33:09.941 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:33:09.949 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:33:09.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.979+0000 7fa76b400640 1 Processor -- start 2024-04-28T13:33:09.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.979+0000 7fa76b400640 1 -- start start 2024-04-28T13:33:09.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.980+0000 7fa76b400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa7640d0d00 0x7fa7640d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:09.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.980+0000 7fa76b400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa764160f40 con 0x7fa7641469f0 2024-04-28T13:33:09.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.980+0000 7fa76b400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa764161220 con 0x7fa7640b5dd0 2024-04-28T13:33:09.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.980+0000 7fa76b400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa7640b5960 con 0x7fa7640d0d00 2024-04-28T13:33:09.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.980+0000 7fa768a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa7641469f0 0x7fa7640ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:33794/0 (socket says 172.21.15.196:33794) 2024-04-28T13:33:09.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.980+0000 7fa768a00640 1 -- 172.21.15.196:0/3889599433 learned_addr learned my addr 172.21.15.196:0/3889599433 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:09.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.981+0000 7fa763e00640 1 --2- 172.21.15.196:0/3889599433 >> v2:172.21.15.203:3300/0 conn(0x7fa7640d0d00 0x7fa7640d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:09.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.981+0000 7fa763400640 1 -- 172.21.15.196:0/3889599433 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3585446926 0 0) 0x7fa764160f40 con 0x7fa7641469f0 2024-04-28T13:33:09.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.981+0000 7fa763400640 1 -- 172.21.15.196:0/3889599433 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa74c003520 con 0x7fa7641469f0 2024-04-28T13:33:09.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.981+0000 7fa763e00640 1 -- 172.21.15.196:0/3889599433 >> v1:172.21.15.203:6789/0 conn(0x7fa7640b5dd0 legacy=0x7fa7640b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:09.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.981+0000 7fa763e00640 1 -- 172.21.15.196:0/3889599433 >> v1:172.21.15.196:6789/0 conn(0x7fa7641469f0 legacy=0x7fa7640ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:09.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.981+0000 7fa763e00640 1 -- 172.21.15.196:0/3889599433 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa764161500 con 0x7fa7640d0d00 2024-04-28T13:33:09.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.982+0000 7fa763e00640 1 --2- 172.21.15.196:0/3889599433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7640d0d00 0x7fa7640d1190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa7500097a0 tx=0x7fa75002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbfd9e13bf370dde server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:09.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.983+0000 7fa763400640 1 -- 172.21.15.196:0/3889599433 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750037070 con 0x7fa7640d0d00 2024-04-28T13:33:09.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.983+0000 7fa763400640 1 -- 172.21.15.196:0/3889599433 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa750032440 con 0x7fa7640d0d00 2024-04-28T13:33:09.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.983+0000 7fa763400640 1 -- 172.21.15.196:0/3889599433 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7500327f0 con 0x7fa7640d0d00 2024-04-28T13:33:09.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.983+0000 7fa76b400640 1 -- 172.21.15.196:0/3889599433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7640d0d00 msgr2=0x7fa7640d1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:09.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.983+0000 7fa76b400640 1 --2- 172.21.15.196:0/3889599433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7640d0d00 0x7fa7640d1190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa7500097a0 tx=0x7fa75002a230 comp rx=0 tx=0).stop 2024-04-28T13:33:09.985 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.983+0000 7fa76b400640 1 -- 172.21.15.196:0/3889599433 shutdown_connections 2024-04-28T13:33:09.985 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.983+0000 7fa76b400640 1 --2- 172.21.15.196:0/3889599433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7640d0d00 0x7fa7640d1190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.985 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.983+0000 7fa76b400640 1 -- 172.21.15.196:0/3889599433 >> 172.21.15.196:0/3889599433 conn(0x7fa7640cd830 msgr2=0x7fa7640cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:09.985 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.984+0000 7fa76b400640 1 -- 172.21.15.196:0/3889599433 shutdown_connections 2024-04-28T13:33:09.985 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.984+0000 7fa76b400640 1 -- 172.21.15.196:0/3889599433 wait complete. 2024-04-28T13:33:09.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.985+0000 7fa76b400640 1 Processor -- start 2024-04-28T13:33:09.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.985+0000 7fa76b400640 1 -- start start 2024-04-28T13:33:09.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.986+0000 7fa76b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7640b5dd0 0x7fa7640b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:09.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.986+0000 7fa76b400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7640d0d00 0x7fa7640b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:09.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.986+0000 7fa76b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7641469f0 0x7fa764159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:09.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.986+0000 7fa76b400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7640cd5f0 con 0x7fa7641469f0 2024-04-28T13:33:09.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.986+0000 7fa76b400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa764161ac0 con 0x7fa7640b5dd0 2024-04-28T13:33:09.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.986+0000 7fa76b400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa764072020 con 0x7fa7640d0d00 2024-04-28T13:33:09.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.987+0000 7fa769400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7641469f0 0x7fa764159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:09.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.987+0000 7fa769400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7641469f0 0x7fa764159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54644/0 (socket says 172.21.15.196:54644) 2024-04-28T13:33:09.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.987+0000 7fa769400640 1 -- 172.21.15.196:0/961006933 learned_addr learned my addr 172.21.15.196:0/961006933 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:09.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.987+0000 7fa768a00640 1 --2- 172.21.15.196:0/961006933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7640b5dd0 0x7fa7640b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:09.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.987+0000 7fa763e00640 1 --2- 172.21.15.196:0/961006933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7640d0d00 0x7fa7640b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:09.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.987+0000 7fa769400640 1 -- 172.21.15.196:0/961006933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7640d0d00 msgr2=0x7fa7640b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:09.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.987+0000 7fa769400640 1 --2- 172.21.15.196:0/961006933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7640d0d00 0x7fa7640b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.987+0000 7fa769400640 1 -- 172.21.15.196:0/961006933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7640b5dd0 msgr2=0x7fa7640b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:09.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.987+0000 7fa769400640 1 --2- 172.21.15.196:0/961006933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7640b5dd0 0x7fa7640b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:09.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.987+0000 7fa769400640 1 -- 172.21.15.196:0/961006933 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7640b21e0 con 0x7fa7641469f0 2024-04-28T13:33:09.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.988+0000 7fa769400640 1 --2- 172.21.15.196:0/961006933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7641469f0 0x7fa764159520 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00c730 tx=0x7fa75c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:09.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.988+0000 7fa760c00640 1 -- 172.21.15.196:0/961006933 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c003b70 con 0x7fa7641469f0 2024-04-28T13:33:09.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.988+0000 7fa760c00640 1 -- 172.21.15.196:0/961006933 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa75c003d30 con 0x7fa7641469f0 2024-04-28T13:33:09.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.988+0000 7fa760c00640 1 -- 172.21.15.196:0/961006933 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c005310 con 0x7fa7641469f0 2024-04-28T13:33:09.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.988+0000 7fa76b400640 1 -- 172.21.15.196:0/961006933 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7640b24c0 con 0x7fa7641469f0 2024-04-28T13:33:09.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.988+0000 7fa76b400640 1 -- 172.21.15.196:0/961006933 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7640b2a80 con 0x7fa7641469f0 2024-04-28T13:33:09.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.989+0000 7fa76b400640 1 -- 172.21.15.196:0/961006933 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa76406a290 con 0x7fa7641469f0 2024-04-28T13:33:09.992 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.991+0000 7fa760c00640 1 -- 172.21.15.196:0/961006933 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa75c0056c0 con 0x7fa7641469f0 2024-04-28T13:33:09.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.992+0000 7fa760c00640 1 --2- 172.21.15.196:0/961006933 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa74406c980 0x7fa74406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:09.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.992+0000 7fa760c00640 1 -- 172.21.15.196:0/961006933 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fa75c08bc40 con 0x7fa7641469f0 2024-04-28T13:33:09.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.993+0000 7fa768a00640 1 --2- 172.21.15.196:0/961006933 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa74406c980 0x7fa74406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:09.994 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.993+0000 7fa768a00640 1 --2- 172.21.15.196:0/961006933 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa74406c980 0x7fa74406ee40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7580011e0 tx=0x7fa758007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:09.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:09.997+0000 7fa760c00640 1 -- 172.21.15.196:0/961006933 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa75c0598c0 con 0x7fa7641469f0 2024-04-28T13:33:10.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.200+0000 7fa76b400640 1 -- 172.21.15.196:0/961006933 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa764143bd0 con 0x7fa7641469f0 2024-04-28T13:33:10.204 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.203+0000 7fa760c00640 1 -- 172.21.15.196:0/961006933 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16834 (secure 0 0 0) 0x7fa75c05dbc0 con 0x7fa7641469f0 2024-04-28T13:33:10.204 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:10.204 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":34,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:58.527505+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:32:29.352348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-28T13:32:30.637498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6836/459640695":"2024-04-29T13:32:58.527451+0000","172.21.15.203:6835/459640695":"2024-04-29T13:32:58.527451+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:33:10.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.205+0000 7fa76b400640 1 -- 172.21.15.196:0/961006933 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa74406c980 msgr2=0x7fa74406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:10.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.205+0000 7fa76b400640 1 --2- 172.21.15.196:0/961006933 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa74406c980 0x7fa74406ee40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7580011e0 tx=0x7fa758007d30 comp rx=0 tx=0).stop 2024-04-28T13:33:10.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.205+0000 7fa76b400640 1 -- 172.21.15.196:0/961006933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7641469f0 msgr2=0x7fa764159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:10.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.205+0000 7fa76b400640 1 --2- 172.21.15.196:0/961006933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7641469f0 0x7fa764159520 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00c730 tx=0x7fa75c00cbf0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.206+0000 7fa76b400640 1 -- 172.21.15.196:0/961006933 shutdown_connections 2024-04-28T13:33:10.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.206+0000 7fa76b400640 1 --2- 172.21.15.196:0/961006933 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa74406c980 0x7fa74406ee40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.206+0000 7fa76b400640 1 --2- 172.21.15.196:0/961006933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7641469f0 0x7fa764159520 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.206+0000 7fa76b400640 1 --2- 172.21.15.196:0/961006933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7640d0d00 0x7fa7640b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.206+0000 7fa76b400640 1 --2- 172.21.15.196:0/961006933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7640b5dd0 0x7fa7640b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.206+0000 7fa76b400640 1 -- 172.21.15.196:0/961006933 >> 172.21.15.196:0/961006933 conn(0x7fa7640cd830 msgr2=0x7fa7640b6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:10.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.206+0000 7fa76b400640 1 -- 172.21.15.196:0/961006933 shutdown_connections 2024-04-28T13:33:10.207 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.206+0000 7fa76b400640 1 -- 172.21.15.196:0/961006933 wait complete. 2024-04-28T13:33:10.227 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:32:29.352348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:32:30.637498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:33:10.228 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:33:10.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.433+0000 7fc98ec00640 1 Processor -- start 2024-04-28T13:33:10.435 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.434+0000 7fc98ec00640 1 -- start start 2024-04-28T13:33:10.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.435+0000 7fc98ec00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc98815e7e0 0x7fc9880ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.435+0000 7fc98ec00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc988161cb0 0x7fc9880d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.435+0000 7fc98ec00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9880d1d90 0x7fc988161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.435+0000 7fc98ec00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc9880d2280 con 0x7fc988161cb0 2024-04-28T13:33:10.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.435+0000 7fc98ec00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc9880cdd40 con 0x7fc98815e7e0 2024-04-28T13:33:10.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.435+0000 7fc98ec00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc9880cdfe0 con 0x7fc9880d1d90 2024-04-28T13:33:10.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.435+0000 7fc987400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc988161cb0 0x7fc9880d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:10.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.435+0000 7fc987400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc988161cb0 0x7fc9880d1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54656/0 (socket says 172.21.15.196:54656) 2024-04-28T13:33:10.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.435+0000 7fc987400640 1 -- 172.21.15.196:0/2301067752 learned_addr learned my addr 172.21.15.196:0/2301067752 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:10.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.436+0000 7fc987e00640 1 --2- 172.21.15.196:0/2301067752 >> v2:172.21.15.203:3300/0 conn(0x7fc98815e7e0 0x7fc9880ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:10.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.436+0000 7fc98cc00640 1 --2- 172.21.15.196:0/2301067752 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9880d1d90 0x7fc988161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:10.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.436+0000 7fc987400640 1 -- 172.21.15.196:0/2301067752 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9880d1d90 msgr2=0x7fc988161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:10.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.436+0000 7fc987400640 1 --2- 172.21.15.196:0/2301067752 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9880d1d90 0x7fc988161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.436+0000 7fc987400640 1 -- 172.21.15.196:0/2301067752 >> v2:172.21.15.203:3300/0 conn(0x7fc98815e7e0 msgr2=0x7fc9880ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:10.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.436+0000 7fc987400640 1 --2- 172.21.15.196:0/2301067752 >> v2:172.21.15.203:3300/0 conn(0x7fc98815e7e0 0x7fc9880ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.436+0000 7fc987400640 1 -- 172.21.15.196:0/2301067752 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9880ce2c0 con 0x7fc988161cb0 2024-04-28T13:33:10.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.437+0000 7fc987400640 1 --2- 172.21.15.196:0/2301067752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc988161cb0 0x7fc9880d1850 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc9880734e0 tx=0x7fc97002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7b818a2cfe6804 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:10.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.437+0000 7fc986a00640 1 -- 172.21.15.196:0/2301067752 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970036070 con 0x7fc988161cb0 2024-04-28T13:33:10.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.437+0000 7fc986a00640 1 -- 172.21.15.196:0/2301067752 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc970031600 con 0x7fc988161cb0 2024-04-28T13:33:10.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.437+0000 7fc986a00640 1 -- 172.21.15.196:0/2301067752 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9700319d0 con 0x7fc988161cb0 2024-04-28T13:33:10.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.438+0000 7fc98ec00640 1 -- 172.21.15.196:0/2301067752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc988161cb0 msgr2=0x7fc9880d1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:10.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.438+0000 7fc98ec00640 1 --2- 172.21.15.196:0/2301067752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc988161cb0 0x7fc9880d1850 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc9880734e0 tx=0x7fc97002a6a0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.438+0000 7fc98ec00640 1 -- 172.21.15.196:0/2301067752 shutdown_connections 2024-04-28T13:33:10.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.438+0000 7fc98ec00640 1 --2- 172.21.15.196:0/2301067752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc988161cb0 0x7fc9880d1850 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.438+0000 7fc98ec00640 1 --2- 172.21.15.196:0/2301067752 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9880d1d90 0x7fc988161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.438+0000 7fc98ec00640 1 --2- 172.21.15.196:0/2301067752 >> v2:172.21.15.203:3300/0 conn(0x7fc98815e7e0 0x7fc9880ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.438+0000 7fc98ec00640 1 -- 172.21.15.196:0/2301067752 >> 172.21.15.196:0/2301067752 conn(0x7fc98809ba40 msgr2=0x7fc98809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:10.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.438+0000 7fc98ec00640 1 -- 172.21.15.196:0/2301067752 shutdown_connections 2024-04-28T13:33:10.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.438+0000 7fc98ec00640 1 -- 172.21.15.196:0/2301067752 wait complete. 2024-04-28T13:33:10.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.440+0000 7fc98ec00640 1 Processor -- start 2024-04-28T13:33:10.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.440+0000 7fc98ec00640 1 -- start start 2024-04-28T13:33:10.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.441+0000 7fc98ec00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9880d1d90 0x7fc9881601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.441+0000 7fc98ec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc98815e7e0 0x7fc988150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.444 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.441+0000 7fc98ec00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc988161cb0 0x7fc98815ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.441+0000 7fc98ec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9880cf2e0 con 0x7fc98815e7e0 2024-04-28T13:33:10.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.441+0000 7fc98ec00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc9880ce880 con 0x7fc988161cb0 2024-04-28T13:33:10.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.441+0000 7fc98ec00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc9881615c0 con 0x7fc9880d1d90 2024-04-28T13:33:10.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.441+0000 7fc987400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc98815e7e0 0x7fc988150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:10.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.441+0000 7fc987400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc98815e7e0 0x7fc988150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54672/0 (socket says 172.21.15.196:54672) 2024-04-28T13:33:10.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.441+0000 7fc987400640 1 -- 172.21.15.196:0/1784349042 learned_addr learned my addr 172.21.15.196:0/1784349042 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:10.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.442+0000 7fc987400640 1 -- 172.21.15.196:0/1784349042 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9880d1d90 msgr2=0x7fc9881601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:10.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.442+0000 7fc987400640 1 --2- 172.21.15.196:0/1784349042 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9880d1d90 0x7fc9881601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.442+0000 7fc987400640 1 -- 172.21.15.196:0/1784349042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc988161cb0 msgr2=0x7fc98815ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:10.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.442+0000 7fc98cc00640 1 --2- 172.21.15.196:0/1784349042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc988161cb0 0x7fc98815ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:10.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.442+0000 7fc987400640 1 --2- 172.21.15.196:0/1784349042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc988161cb0 0x7fc98815ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.442+0000 7fc987400640 1 -- 172.21.15.196:0/1784349042 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98815d350 con 0x7fc98815e7e0 2024-04-28T13:33:10.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.442+0000 7fc987400640 1 --2- 172.21.15.196:0/1784349042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc98815e7e0 0x7fc988150d90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc970000f80 tx=0x7fc970004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:10.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.442+0000 7fc97fe00640 1 -- 172.21.15.196:0/1784349042 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970036050 con 0x7fc98815e7e0 2024-04-28T13:33:10.445 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.442+0000 7fc97fe00640 1 -- 172.21.15.196:0/1784349042 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc970042070 con 0x7fc98815e7e0 2024-04-28T13:33:10.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.442+0000 7fc98ec00640 1 -- 172.21.15.196:0/1784349042 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc988160790 con 0x7fc98815e7e0 2024-04-28T13:33:10.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.442+0000 7fc97fe00640 1 -- 172.21.15.196:0/1784349042 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970009510 con 0x7fc98815e7e0 2024-04-28T13:33:10.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.442+0000 7fc98ec00640 1 -- 172.21.15.196:0/1784349042 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc98815ecf0 con 0x7fc98815e7e0 2024-04-28T13:33:10.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.444+0000 7fc98ec00640 1 -- 172.21.15.196:0/1784349042 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc98806a290 con 0x7fc98815e7e0 2024-04-28T13:33:10.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.445+0000 7fc97fe00640 1 -- 172.21.15.196:0/1784349042 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc970007c70 con 0x7fc98815e7e0 2024-04-28T13:33:10.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.446+0000 7fc97fe00640 1 --2- 172.21.15.196:0/1784349042 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc96406c980 0x7fc96406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.446+0000 7fc97fe00640 1 -- 172.21.15.196:0/1784349042 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fc9700acc00 con 0x7fc98815e7e0 2024-04-28T13:33:10.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.446+0000 7fc987e00640 1 --2- 172.21.15.196:0/1784349042 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc96406c980 0x7fc96406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:10.448 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.448+0000 7fc987e00640 1 --2- 172.21.15.196:0/1784349042 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc96406c980 0x7fc96406ee40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc9780098f0 tx=0x7fc978008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:10.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.450+0000 7fc97fe00640 1 -- 172.21.15.196:0/1784349042 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc970045360 con 0x7fc98815e7e0 2024-04-28T13:33:10.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.650+0000 7fc98ec00640 1 -- 172.21.15.196:0/1784349042 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc9881613a0 con 0x7fc98815e7e0 2024-04-28T13:33:10.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.651+0000 7fc97fe00640 1 -- 172.21.15.196:0/1784349042 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7fc97007ae60 con 0x7fc98815e7e0 2024-04-28T13:33:10.652 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:33:10.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.653+0000 7fc98ec00640 1 -- 172.21.15.196:0/1784349042 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc96406c980 msgr2=0x7fc96406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:10.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.653+0000 7fc98ec00640 1 --2- 172.21.15.196:0/1784349042 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc96406c980 0x7fc96406ee40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc9780098f0 tx=0x7fc978008040 comp rx=0 tx=0).stop 2024-04-28T13:33:10.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.653+0000 7fc98ec00640 1 -- 172.21.15.196:0/1784349042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc98815e7e0 msgr2=0x7fc988150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:10.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.653+0000 7fc98ec00640 1 --2- 172.21.15.196:0/1784349042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc98815e7e0 0x7fc988150d90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc970000f80 tx=0x7fc970004a40 comp rx=0 tx=0).stop 2024-04-28T13:33:10.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.653+0000 7fc98ec00640 1 -- 172.21.15.196:0/1784349042 shutdown_connections 2024-04-28T13:33:10.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.653+0000 7fc98ec00640 1 --2- 172.21.15.196:0/1784349042 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc96406c980 0x7fc96406ee40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.653+0000 7fc98ec00640 1 --2- 172.21.15.196:0/1784349042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc988161cb0 0x7fc98815ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.653+0000 7fc98ec00640 1 --2- 172.21.15.196:0/1784349042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc98815e7e0 0x7fc988150d90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.653+0000 7fc98ec00640 1 --2- 172.21.15.196:0/1784349042 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9880d1d90 0x7fc9881601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.653+0000 7fc98ec00640 1 -- 172.21.15.196:0/1784349042 >> 172.21.15.196:0/1784349042 conn(0x7fc98809ba40 msgr2=0x7fc98815b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:10.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.654+0000 7fc98ec00640 1 -- 172.21.15.196:0/1784349042 shutdown_connections 2024-04-28T13:33:10.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.654+0000 7fc98ec00640 1 -- 172.21.15.196:0/1784349042 wait complete. 2024-04-28T13:33:10.673 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:33:10.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.871+0000 7fc6a2a00640 1 Processor -- start 2024-04-28T13:33:10.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.871+0000 7fc6a2a00640 1 -- start start 2024-04-28T13:33:10.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.873+0000 7fc6a2a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc69c0b4eb0 0x7fc69c0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.873+0000 7fc6a2a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc69c0ceff0 0x7fc69c0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.873+0000 7fc6a2a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc69c15dca0 con 0x7fc69c143840 2024-04-28T13:33:10.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.873+0000 7fc6a2a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc69c143c20 con 0x7fc69c0ceff0 2024-04-28T13:33:10.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.873+0000 7fc6a2a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c15df80 con 0x7fc69c0b4eb0 2024-04-28T13:33:10.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.873+0000 7fc69be00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc69c143840 0x7fc69c0b3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:33798/0 (socket says 172.21.15.196:33798) 2024-04-28T13:33:10.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.873+0000 7fc69be00640 1 -- 172.21.15.196:0/2852202197 learned_addr learned my addr 172.21.15.196:0/2852202197 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:10.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc6a0a00640 1 --2- 172.21.15.196:0/2852202197 >> v2:172.21.15.203:3300/0 conn(0x7fc69c0ceff0 0x7fc69c0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:10.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.873+0000 7fc69b400640 1 --2- 172.21.15.196:0/2852202197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc69c0b4eb0 0x7fc69c0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:10.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 -- 172.21.15.196:0/2852202197 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3622989105 0 0) 0x7fc69c15dca0 con 0x7fc69c143840 2024-04-28T13:33:10.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 -- 172.21.15.196:0/2852202197 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc680003520 con 0x7fc69c143840 2024-04-28T13:33:10.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 -- 172.21.15.196:0/2852202197 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1480803740 0 0) 0x7fc680003520 con 0x7fc69c143840 2024-04-28T13:33:10.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 -- 172.21.15.196:0/2852202197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc69c0b4eb0 msgr2=0x7fc69c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:10.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 --2- 172.21.15.196:0/2852202197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc69c0b4eb0 0x7fc69c0b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 -- 172.21.15.196:0/2852202197 >> v2:172.21.15.203:3300/0 conn(0x7fc69c0ceff0 msgr2=0x7fc69c0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:10.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 --2- 172.21.15.196:0/2852202197 >> v2:172.21.15.203:3300/0 conn(0x7fc69c0ceff0 0x7fc69c0cf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 -- 172.21.15.196:0/2852202197 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c15e260 con 0x7fc69c143840 2024-04-28T13:33:10.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 -- 172.21.15.196:0/2852202197 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fc68c001670 con 0x7fc69c143840 2024-04-28T13:33:10.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc6a0a00640 1 --2- 172.21.15.196:0/2852202197 >> v2:172.21.15.203:3300/0 conn(0x7fc69c0ceff0 0x7fc69c0cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:33:10.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 -- 172.21.15.196:0/2852202197 >> v1:172.21.15.196:6789/0 conn(0x7fc69c143840 legacy=0x7fc69c0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:10.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 --2- 172.21.15.196:0/2852202197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6800039b0 0x7fc680003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 --2- 172.21.15.196:0/2852202197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc69c0b4eb0 0x7fc680004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 --2- 172.21.15.196:0/2852202197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc680004d40 0x7fc6800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 -- 172.21.15.196:0/2852202197 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c143c20 con 0x7fc6800039b0 2024-04-28T13:33:10.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 -- 172.21.15.196:0/2852202197 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c15df80 con 0x7fc680004d40 2024-04-28T13:33:10.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69aa00640 1 -- 172.21.15.196:0/2852202197 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc68000b9c0 con 0x7fc69c0b4eb0 2024-04-28T13:33:10.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.874+0000 7fc69be00640 1 --2- 172.21.15.196:0/2852202197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6800039b0 0x7fc680003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:10.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.875+0000 7fc69be00640 1 -- 172.21.15.196:0/2852202197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc69c0b4eb0 msgr2=0x7fc680004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:10.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.875+0000 7fc6a0a00640 1 --2- 172.21.15.196:0/2852202197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc680004d40 0x7fc6800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:10.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.875+0000 7fc69b400640 1 --2- 172.21.15.196:0/2852202197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc69c0b4eb0 0x7fc680004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:10.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.875+0000 7fc69be00640 1 --2- 172.21.15.196:0/2852202197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc69c0b4eb0 0x7fc680004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.875+0000 7fc69be00640 1 -- 172.21.15.196:0/2852202197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc680004d40 msgr2=0x7fc6800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:10.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.875+0000 7fc69be00640 1 --2- 172.21.15.196:0/2852202197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc680004d40 0x7fc6800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.875+0000 7fc69be00640 1 -- 172.21.15.196:0/2852202197 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc680003520 con 0x7fc6800039b0 2024-04-28T13:33:10.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.875+0000 7fc6a0a00640 1 --2- 172.21.15.196:0/2852202197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc680004d40 0x7fc6800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:10.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.876+0000 7fc69be00640 1 --2- 172.21.15.196:0/2852202197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6800039b0 0x7fc680003dd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c210 tx=0x7fc680005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabf761bd4cbf960 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:10.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.876+0000 7fc69aa00640 1 -- 172.21.15.196:0/2852202197 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c007d70 con 0x7fc6800039b0 2024-04-28T13:33:10.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.876+0000 7fc69aa00640 1 -- 172.21.15.196:0/2852202197 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc68c005af0 con 0x7fc6800039b0 2024-04-28T13:33:10.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.876+0000 7fc69aa00640 1 -- 172.21.15.196:0/2852202197 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c007070 con 0x7fc6800039b0 2024-04-28T13:33:10.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.877+0000 7fc6a2a00640 1 -- 172.21.15.196:0/2852202197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6800039b0 msgr2=0x7fc680003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:10.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.877+0000 7fc6a2a00640 1 --2- 172.21.15.196:0/2852202197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6800039b0 0x7fc680003dd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c210 tx=0x7fc680005830 comp rx=0 tx=0).stop 2024-04-28T13:33:10.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.877+0000 7fc6a0a00640 1 -- 172.21.15.196:0/2852202197 reap_dead start 2024-04-28T13:33:10.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.877+0000 7fc6a2a00640 1 -- 172.21.15.196:0/2852202197 shutdown_connections 2024-04-28T13:33:10.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.877+0000 7fc6a2a00640 1 -- 172.21.15.196:0/2852202197 >> 172.21.15.196:0/2852202197 conn(0x7fc69c0cd8a0 msgr2=0x7fc69c0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:10.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.877+0000 7fc6a2a00640 1 -- 172.21.15.196:0/2852202197 shutdown_connections 2024-04-28T13:33:10.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.877+0000 7fc6a2a00640 1 -- 172.21.15.196:0/2852202197 wait complete. 2024-04-28T13:33:10.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.879+0000 7fc6a2a00640 1 Processor -- start 2024-04-28T13:33:10.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.879+0000 7fc6a2a00640 1 -- start start 2024-04-28T13:33:10.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.880+0000 7fc6a2a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6800039b0 0x7fc69c160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.880+0000 7fc6a2a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc69c15f4a0 0x7fc69c15a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.880+0000 7fc6a2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc69c15c490 0x7fc69c15c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.880+0000 7fc6a2a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c15d8c0 con 0x7fc6800039b0 2024-04-28T13:33:10.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.880+0000 7fc6a2a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c0b46f0 con 0x7fc69c15c490 2024-04-28T13:33:10.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.880+0000 7fc6a2a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c15da40 con 0x7fc69c15f4a0 2024-04-28T13:33:10.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.880+0000 7fc69be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6800039b0 0x7fc69c160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:10.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.880+0000 7fc69be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6800039b0 0x7fc69c160060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54686/0 (socket says 172.21.15.196:54686) 2024-04-28T13:33:10.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.880+0000 7fc69be00640 1 -- 172.21.15.196:0/1541695895 learned_addr learned my addr 172.21.15.196:0/1541695895 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:10.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.881+0000 7fc6a0a00640 1 --2- 172.21.15.196:0/1541695895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc69c15c490 0x7fc69c15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:10.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.881+0000 7fc69be00640 1 -- 172.21.15.196:0/1541695895 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc69c15f4a0 msgr2=0x7fc69c15a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:10.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.881+0000 7fc69be00640 1 --2- 172.21.15.196:0/1541695895 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc69c15f4a0 0x7fc69c15a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.881+0000 7fc69be00640 1 -- 172.21.15.196:0/1541695895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc69c15c490 msgr2=0x7fc69c15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:10.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.881+0000 7fc69be00640 1 --2- 172.21.15.196:0/1541695895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc69c15c490 0x7fc69c15c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:10.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.881+0000 7fc69be00640 1 -- 172.21.15.196:0/1541695895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c13fe00 con 0x7fc6800039b0 2024-04-28T13:33:10.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.881+0000 7fc69be00640 1 --2- 172.21.15.196:0/1541695895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6800039b0 0x7fc69c160060 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c330 tx=0x7fc68c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:10.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.881+0000 7fc693e00640 1 -- 172.21.15.196:0/1541695895 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c042070 con 0x7fc6800039b0 2024-04-28T13:33:10.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.881+0000 7fc693e00640 1 -- 172.21.15.196:0/1541695895 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc68c007480 con 0x7fc6800039b0 2024-04-28T13:33:10.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.881+0000 7fc6a2a00640 1 -- 172.21.15.196:0/1541695895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69c1400e0 con 0x7fc6800039b0 2024-04-28T13:33:10.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.881+0000 7fc693e00640 1 -- 172.21.15.196:0/1541695895 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c007d50 con 0x7fc6800039b0 2024-04-28T13:33:10.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.881+0000 7fc6a2a00640 1 -- 172.21.15.196:0/1541695895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69c1406a0 con 0x7fc6800039b0 2024-04-28T13:33:10.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.883+0000 7fc6a2a00640 1 -- 172.21.15.196:0/1541695895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc69c06a290 con 0x7fc6800039b0 2024-04-28T13:33:10.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.883+0000 7fc693e00640 1 -- 172.21.15.196:0/1541695895 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc68c03b050 con 0x7fc6800039b0 2024-04-28T13:33:10.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.884+0000 7fc693e00640 1 --2- 172.21.15.196:0/1541695895 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc67c06c980 0x7fc67c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:10.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.884+0000 7fc693e00640 1 -- 172.21.15.196:0/1541695895 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fc68c0ac570 con 0x7fc6800039b0 2024-04-28T13:33:10.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.885+0000 7fc69b400640 1 --2- 172.21.15.196:0/1541695895 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc67c06c980 0x7fc67c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:10.887 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.886+0000 7fc69b400640 1 --2- 172.21.15.196:0/1541695895 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc67c06c980 0x7fc67c06ee40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc69c136f60 tx=0x7fc688008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:10.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:10.889+0000 7fc693e00640 1 -- 172.21.15.196:0/1541695895 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc68c005ac0 con 0x7fc6800039b0 2024-04-28T13:33:11.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.087+0000 7fc6a2a00640 1 -- 172.21.15.196:0/1541695895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc69c0ce420 con 0x7fc6800039b0 2024-04-28T13:33:11.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.088+0000 7fc693e00640 1 -- 172.21.15.196:0/1541695895 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7fc68c07a810 con 0x7fc6800039b0 2024-04-28T13:33:11.090 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:33:11.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.091+0000 7fc6a2a00640 1 -- 172.21.15.196:0/1541695895 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc67c06c980 msgr2=0x7fc67c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:11.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.091+0000 7fc6a2a00640 1 --2- 172.21.15.196:0/1541695895 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc67c06c980 0x7fc67c06ee40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc69c136f60 tx=0x7fc688008040 comp rx=0 tx=0).stop 2024-04-28T13:33:11.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.091+0000 7fc6a2a00640 1 -- 172.21.15.196:0/1541695895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6800039b0 msgr2=0x7fc69c160060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:11.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.091+0000 7fc6a2a00640 1 --2- 172.21.15.196:0/1541695895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6800039b0 0x7fc69c160060 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c330 tx=0x7fc68c000fe0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.091+0000 7fc6a2a00640 1 -- 172.21.15.196:0/1541695895 shutdown_connections 2024-04-28T13:33:11.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.091+0000 7fc6a2a00640 1 --2- 172.21.15.196:0/1541695895 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc67c06c980 0x7fc67c06ee40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.091+0000 7fc6a2a00640 1 --2- 172.21.15.196:0/1541695895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc69c15c490 0x7fc69c15c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.091+0000 7fc6a2a00640 1 --2- 172.21.15.196:0/1541695895 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc69c15f4a0 0x7fc69c15a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.091+0000 7fc6a2a00640 1 --2- 172.21.15.196:0/1541695895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6800039b0 0x7fc69c160060 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.091+0000 7fc6a2a00640 1 -- 172.21.15.196:0/1541695895 >> 172.21.15.196:0/1541695895 conn(0x7fc69c0cd8a0 msgr2=0x7fc69c0aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:11.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.091+0000 7fc6a2a00640 1 -- 172.21.15.196:0/1541695895 shutdown_connections 2024-04-28T13:33:11.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.091+0000 7fc6a2a00640 1 -- 172.21.15.196:0/1541695895 wait complete. 2024-04-28T13:33:11.110 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:33:11.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.307+0000 7f402b200640 1 Processor -- start 2024-04-28T13:33:11.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.308+0000 7f402b200640 1 -- start start 2024-04-28T13:33:11.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.309+0000 7f402b200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f402c072d00 0x7f402c0723d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:11.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.309+0000 7f402b200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f402c160570 con 0x7f402c06f830 2024-04-28T13:33:11.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.309+0000 7f402b200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f402c06df30 con 0x7f402c15d0a0 2024-04-28T13:33:11.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.309+0000 7f402b200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f402c0b3c90 con 0x7f402c072d00 2024-04-28T13:33:11.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.310+0000 7f4029e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f402c072d00 0x7f402c0723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:11.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.310+0000 7f4029e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f402c072d00 0x7f402c0723d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54700/0 (socket says 172.21.15.196:54700) 2024-04-28T13:33:11.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.310+0000 7f4029e00640 1 -- 172.21.15.196:0/3313249248 learned_addr learned my addr 172.21.15.196:0/3313249248 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:11.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.310+0000 7f4023e00640 1 --1- 172.21.15.196:0/3313249248 >> v1:172.21.15.196:6789/0 conn(0x7f402c06f830 0x7f402c0730e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:33808/0 (socket says 172.21.15.196:33808) 2024-04-28T13:33:11.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.310+0000 7f4029e00640 1 -- 172.21.15.196:0/3313249248 >> v1:172.21.15.203:6789/0 conn(0x7f402c15d0a0 legacy=0x7f402c0b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:11.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.310+0000 7f4029e00640 1 -- 172.21.15.196:0/3313249248 >> v1:172.21.15.196:6789/0 conn(0x7f402c06f830 legacy=0x7f402c0730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:11.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.310+0000 7f4029400640 1 -- 172.21.15.196:0/3313249248 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 942343972 0 0) 0x7f402c160570 con 0x7f402c06f830 2024-04-28T13:33:11.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.310+0000 7f4029e00640 1 -- 172.21.15.196:0/3313249248 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c06e210 con 0x7f402c072d00 2024-04-28T13:33:11.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.311+0000 7f4029e00640 1 --2- 172.21.15.196:0/3313249248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f402c072d00 0x7f402c0723d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f401c009870 tx=0x7f401c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2121f52bf5837b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:11.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.311+0000 7f4029400640 1 -- 172.21.15.196:0/3313249248 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c037070 con 0x7f402c072d00 2024-04-28T13:33:11.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.312+0000 7f4029400640 1 -- 172.21.15.196:0/3313249248 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f401c032440 con 0x7f402c072d00 2024-04-28T13:33:11.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.312+0000 7f4029400640 1 -- 172.21.15.196:0/3313249248 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c032810 con 0x7f402c072d00 2024-04-28T13:33:11.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.312+0000 7f402b200640 1 -- 172.21.15.196:0/3313249248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f402c072d00 msgr2=0x7f402c0723d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:11.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.312+0000 7f402b200640 1 --2- 172.21.15.196:0/3313249248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f402c072d00 0x7f402c0723d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f401c009870 tx=0x7f401c02a2a0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.312+0000 7f402b200640 1 -- 172.21.15.196:0/3313249248 shutdown_connections 2024-04-28T13:33:11.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.312+0000 7f402b200640 1 --2- 172.21.15.196:0/3313249248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f402c072d00 0x7f402c0723d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.312+0000 7f402b200640 1 -- 172.21.15.196:0/3313249248 >> 172.21.15.196:0/3313249248 conn(0x7f402c0b2990 msgr2=0x7f402c0b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:11.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.312+0000 7f402b200640 1 -- 172.21.15.196:0/3313249248 shutdown_connections 2024-04-28T13:33:11.313 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.312+0000 7f402b200640 1 -- 172.21.15.196:0/3313249248 wait complete. 2024-04-28T13:33:11.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.314+0000 7f402b200640 1 Processor -- start 2024-04-28T13:33:11.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.314+0000 7f402b200640 1 -- start start 2024-04-28T13:33:11.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.315+0000 7f402b200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f402c06f830 0x7f402c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:11.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.315+0000 7f402b200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f402c072d00 0x7f402c14b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:11.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.315+0000 7f402b200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c09e760 0x7f402c14b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:11.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.315+0000 7f402b200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f402c0b4830 con 0x7f402c072d00 2024-04-28T13:33:11.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.315+0000 7f402b200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f402c06fc70 con 0x7f402c06f830 2024-04-28T13:33:11.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.315+0000 7f402b200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f402c0b49b0 con 0x7f402c09e760 2024-04-28T13:33:11.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.316+0000 7f4023e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f402c072d00 0x7f402c14b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:11.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.316+0000 7f4023e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f402c072d00 0x7f402c14b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54710/0 (socket says 172.21.15.196:54710) 2024-04-28T13:33:11.317 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.316+0000 7f4023e00640 1 -- 172.21.15.196:0/2397891898 learned_addr learned my addr 172.21.15.196:0/2397891898 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:11.318 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.316+0000 7f4029e00640 1 --2- 172.21.15.196:0/2397891898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f402c06f830 0x7f402c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:11.318 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.316+0000 7f402a800640 1 --2- 172.21.15.196:0/2397891898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c09e760 0x7f402c14b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:11.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.316+0000 7f4023e00640 1 -- 172.21.15.196:0/2397891898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c09e760 msgr2=0x7f402c14b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:11.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.316+0000 7f4023e00640 1 --2- 172.21.15.196:0/2397891898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c09e760 0x7f402c14b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.316+0000 7f4023e00640 1 -- 172.21.15.196:0/2397891898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f402c06f830 msgr2=0x7f402c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:11.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.316+0000 7f4023e00640 1 --2- 172.21.15.196:0/2397891898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f402c06f830 0x7f402c15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.316+0000 7f4023e00640 1 -- 172.21.15.196:0/2397891898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c09c160 con 0x7f402c072d00 2024-04-28T13:33:11.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.316+0000 7f4029e00640 1 --2- 172.21.15.196:0/2397891898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f402c06f830 0x7f402c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:11.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.317+0000 7f4023e00640 1 --2- 172.21.15.196:0/2397891898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f402c072d00 0x7f402c14b400 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f401400c500 tx=0x7f401400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:11.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.317+0000 7f4022000640 1 -- 172.21.15.196:0/2397891898 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014018070 con 0x7f402c072d00 2024-04-28T13:33:11.319 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.317+0000 7f4022000640 1 -- 172.21.15.196:0/2397891898 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4014007580 con 0x7f402c072d00 2024-04-28T13:33:11.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.317+0000 7f4022000640 1 -- 172.21.15.196:0/2397891898 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014004a20 con 0x7f402c072d00 2024-04-28T13:33:11.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.317+0000 7f402b200640 1 -- 172.21.15.196:0/2397891898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402c09c440 con 0x7f402c072d00 2024-04-28T13:33:11.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.317+0000 7f402b200640 1 -- 172.21.15.196:0/2397891898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402c09ca00 con 0x7f402c072d00 2024-04-28T13:33:11.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.318+0000 7f402b200640 1 -- 172.21.15.196:0/2397891898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f402c050730 con 0x7f402c072d00 2024-04-28T13:33:11.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.319+0000 7f4022000640 1 -- 172.21.15.196:0/2397891898 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4014004bc0 con 0x7f402c072d00 2024-04-28T13:33:11.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.320+0000 7f4022000640 1 --2- 172.21.15.196:0/2397891898 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f400806c980 0x7f400806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:11.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.320+0000 7f4022000640 1 -- 172.21.15.196:0/2397891898 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f401408afb0 con 0x7f402c072d00 2024-04-28T13:33:11.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.321+0000 7f4029e00640 1 --2- 172.21.15.196:0/2397891898 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f400806c980 0x7f400806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:11.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.322+0000 7f4029e00640 1 --2- 172.21.15.196:0/2397891898 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f400806c980 0x7f400806ee40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f401c034000 tx=0x7f401c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:11.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.324+0000 7f4022000640 1 -- 172.21.15.196:0/2397891898 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4014059320 con 0x7f402c072d00 2024-04-28T13:33:11.524 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.523+0000 7f402b200640 1 -- 172.21.15.196:0/2397891898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f402c06ef30 con 0x7f402c072d00 2024-04-28T13:33:11.524 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.524+0000 7f4022000640 1 -- 172.21.15.196:0/2397891898 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f4014058c20 con 0x7f402c072d00 2024-04-28T13:33:11.525 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:33:11.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.527+0000 7f402b200640 1 -- 172.21.15.196:0/2397891898 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f400806c980 msgr2=0x7f400806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:11.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.527+0000 7f402b200640 1 --2- 172.21.15.196:0/2397891898 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f400806c980 0x7f400806ee40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f401c034000 tx=0x7f401c02e000 comp rx=0 tx=0).stop 2024-04-28T13:33:11.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.527+0000 7f402b200640 1 -- 172.21.15.196:0/2397891898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f402c072d00 msgr2=0x7f402c14b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:11.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.527+0000 7f402b200640 1 --2- 172.21.15.196:0/2397891898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f402c072d00 0x7f402c14b400 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f401400c500 tx=0x7f401400c9c0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.527+0000 7f402b200640 1 -- 172.21.15.196:0/2397891898 shutdown_connections 2024-04-28T13:33:11.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.527+0000 7f402b200640 1 --2- 172.21.15.196:0/2397891898 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f400806c980 0x7f400806ee40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.527+0000 7f402b200640 1 --2- 172.21.15.196:0/2397891898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c09e760 0x7f402c14b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.527+0000 7f402b200640 1 --2- 172.21.15.196:0/2397891898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f402c072d00 0x7f402c14b400 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.527+0000 7f402b200640 1 --2- 172.21.15.196:0/2397891898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f402c06f830 0x7f402c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.527+0000 7f402b200640 1 -- 172.21.15.196:0/2397891898 >> 172.21.15.196:0/2397891898 conn(0x7f402c0b2990 msgr2=0x7f402c15d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:11.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.527+0000 7f402b200640 1 -- 172.21.15.196:0/2397891898 shutdown_connections 2024-04-28T13:33:11.531 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.527+0000 7f402b200640 1 -- 172.21.15.196:0/2397891898 wait complete. 2024-04-28T13:33:11.547 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T13:33:11.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.750+0000 7f9747000640 1 Processor -- start 2024-04-28T13:33:11.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.750+0000 7f9747000640 1 -- start start 2024-04-28T13:33:11.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.752+0000 7f9747000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f97401469f0 0x7f97400ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:11.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.752+0000 7f9747000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97400d1410 0x7f97400b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:11.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.752+0000 7f9747000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9740160f40 con 0x7f97400cd420 2024-04-28T13:33:11.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.752+0000 7f9747000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f97400b5960 con 0x7f97401469f0 2024-04-28T13:33:11.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.752+0000 7f9747000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9740161ac0 con 0x7f97400d1410 2024-04-28T13:33:11.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.753+0000 7f973fe00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f97401469f0 0x7f97400ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:11.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.753+0000 7f973fe00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f97401469f0 0x7f97400ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:53324/0 (socket says 172.21.15.196:53324) 2024-04-28T13:33:11.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.753+0000 7f973fe00640 1 -- 172.21.15.196:0/323661779 learned_addr learned my addr 172.21.15.196:0/323661779 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:11.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.753+0000 7f9745000640 1 --2- 172.21.15.196:0/323661779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97400d1410 0x7f97400b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:11.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.753+0000 7f973fe00640 1 -- 172.21.15.196:0/323661779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97400d1410 msgr2=0x7f97400b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:11.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.753+0000 7f973fe00640 1 --2- 172.21.15.196:0/323661779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97400d1410 0x7f97400b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.754+0000 7f973fe00640 1 -- 172.21.15.196:0/323661779 >> v1:172.21.15.203:6789/0 conn(0x7f97400cd420 legacy=0x7f97400d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:11.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.754+0000 7f973fe00640 1 -- 172.21.15.196:0/323661779 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9740161220 con 0x7f97401469f0 2024-04-28T13:33:11.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.755+0000 7f973fe00640 1 --2- 172.21.15.196:0/323661779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97401469f0 0x7f97400ccee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9730009870 tx=0x7f973002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf43526b5ca5f91 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:11.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.756+0000 7f973f400640 1 -- 172.21.15.196:0/323661779 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730037070 con 0x7f97401469f0 2024-04-28T13:33:11.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.756+0000 7f973f400640 1 -- 172.21.15.196:0/323661779 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9730032440 con 0x7f97401469f0 2024-04-28T13:33:11.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.756+0000 7f973f400640 1 -- 172.21.15.196:0/323661779 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730032810 con 0x7f97401469f0 2024-04-28T13:33:11.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.756+0000 7f9747000640 1 -- 172.21.15.196:0/323661779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97401469f0 msgr2=0x7f97400ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:11.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.756+0000 7f9747000640 1 --2- 172.21.15.196:0/323661779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97401469f0 0x7f97400ccee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9730009870 tx=0x7f973002a250 comp rx=0 tx=0).stop 2024-04-28T13:33:11.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.756+0000 7f9747000640 1 -- 172.21.15.196:0/323661779 shutdown_connections 2024-04-28T13:33:11.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.756+0000 7f9747000640 1 --2- 172.21.15.196:0/323661779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97400d1410 0x7f97400b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.756+0000 7f9747000640 1 --2- 172.21.15.196:0/323661779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97401469f0 0x7f97400ccee0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.756+0000 7f9747000640 1 -- 172.21.15.196:0/323661779 >> 172.21.15.196:0/323661779 conn(0x7f97400cd830 msgr2=0x7f97400cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:11.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.756+0000 7f9747000640 1 -- 172.21.15.196:0/323661779 shutdown_connections 2024-04-28T13:33:11.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.756+0000 7f9747000640 1 -- 172.21.15.196:0/323661779 wait complete. 2024-04-28T13:33:11.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.758+0000 7f9747000640 1 Processor -- start 2024-04-28T13:33:11.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.758+0000 7f9747000640 1 -- start start 2024-04-28T13:33:11.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.759+0000 7f9747000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97400cd420 0x7f974013c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:11.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.759+0000 7f9747000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97400d1410 0x7f97400b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:11.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.759+0000 7f9747000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97401469f0 0x7f97400b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:11.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.759+0000 7f9747000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9740072020 con 0x7f97400cd420 2024-04-28T13:33:11.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.759+0000 7f9747000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f97401617e0 con 0x7f97400d1410 2024-04-28T13:33:11.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.759+0000 7f9747000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9740160b80 con 0x7f97401469f0 2024-04-28T13:33:11.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.759+0000 7f973fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97400cd420 0x7f974013c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:11.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.760+0000 7f973fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97400cd420 0x7f974013c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54720/0 (socket says 172.21.15.196:54720) 2024-04-28T13:33:11.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.760+0000 7f973fe00640 1 -- 172.21.15.196:0/54730839 learned_addr learned my addr 172.21.15.196:0/54730839 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:11.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.760+0000 7f973fe00640 1 -- 172.21.15.196:0/54730839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97401469f0 msgr2=0x7f97400b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:11.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.760+0000 7f973fe00640 1 --2- 172.21.15.196:0/54730839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97401469f0 0x7f97400b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.760+0000 7f973fe00640 1 -- 172.21.15.196:0/54730839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97400d1410 msgr2=0x7f97400b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:11.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.760+0000 7f9737e00640 1 --2- 172.21.15.196:0/54730839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97400d1410 0x7f97400b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:11.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.760+0000 7f973fe00640 1 --2- 172.21.15.196:0/54730839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97400d1410 0x7f97400b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.760+0000 7f973fe00640 1 -- 172.21.15.196:0/54730839 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9740153010 con 0x7f97400cd420 2024-04-28T13:33:11.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.760+0000 7f973fe00640 1 --2- 172.21.15.196:0/54730839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97400cd420 0x7f974013c370 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9730000c00 tx=0x7f97300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:11.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.761+0000 7f973cc00640 1 -- 172.21.15.196:0/54730839 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730037040 con 0x7f97400cd420 2024-04-28T13:33:11.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.761+0000 7f973cc00640 1 -- 172.21.15.196:0/54730839 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9730042070 con 0x7f97400cd420 2024-04-28T13:33:11.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.761+0000 7f9747000640 1 -- 172.21.15.196:0/54730839 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9740151f70 con 0x7f97400cd420 2024-04-28T13:33:11.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.761+0000 7f973cc00640 1 -- 172.21.15.196:0/54730839 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973003b870 con 0x7f97400cd420 2024-04-28T13:33:11.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.761+0000 7f9747000640 1 -- 172.21.15.196:0/54730839 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9740152530 con 0x7f97400cd420 2024-04-28T13:33:11.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.762+0000 7f9747000640 1 -- 172.21.15.196:0/54730839 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f974006a290 con 0x7f97400cd420 2024-04-28T13:33:11.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.763+0000 7f973cc00640 1 -- 172.21.15.196:0/54730839 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f973003ba10 con 0x7f97400cd420 2024-04-28T13:33:11.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.764+0000 7f973cc00640 1 --2- 172.21.15.196:0/54730839 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f971c06c980 0x7f971c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:11.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.764+0000 7f973cc00640 1 -- 172.21.15.196:0/54730839 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f97300ac450 con 0x7f97400cd420 2024-04-28T13:33:11.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.764+0000 7f9737e00640 1 --2- 172.21.15.196:0/54730839 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f971c06c980 0x7f971c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:11.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.765+0000 7f9737e00640 1 --2- 172.21.15.196:0/54730839 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f971c06c980 0x7f971c06ee40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f974015c620 tx=0x7f9728009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:11.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.768+0000 7f973cc00640 1 -- 172.21.15.196:0/54730839 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f973007a830 con 0x7f97400cd420 2024-04-28T13:33:11.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.981+0000 7f9747000640 1 -- 172.21.15.196:0/54730839 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f974013d080 con 0x7f97400cd420 2024-04-28T13:33:11.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.982+0000 7f973cc00640 1 -- 172.21.15.196:0/54730839 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v34) ==== 80+0+122 (secure 0 0 0) 0x7f973007a130 con 0x7f97400cd420 2024-04-28T13:33:11.983 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.203:6836/459640695 2024-04-29T13:32:58.527451+0000 2024-04-28T13:33:11.983 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.203:6835/459640695 2024-04-29T13:32:58.527451+0000 2024-04-28T13:33:11.983 INFO:teuthology.orchestra.run.smithi196.stderr:listed 2 entries 2024-04-28T13:33:11.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.985+0000 7f9747000640 1 -- 172.21.15.196:0/54730839 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f971c06c980 msgr2=0x7f971c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:11.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.985+0000 7f9747000640 1 --2- 172.21.15.196:0/54730839 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f971c06c980 0x7f971c06ee40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f974015c620 tx=0x7f9728009040 comp rx=0 tx=0).stop 2024-04-28T13:33:11.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.985+0000 7f9747000640 1 -- 172.21.15.196:0/54730839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97400cd420 msgr2=0x7f974013c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:11.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.985+0000 7f9747000640 1 --2- 172.21.15.196:0/54730839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97400cd420 0x7f974013c370 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9730000c00 tx=0x7f97300046c0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.985+0000 7f9747000640 1 -- 172.21.15.196:0/54730839 shutdown_connections 2024-04-28T13:33:11.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.985+0000 7f9747000640 1 --2- 172.21.15.196:0/54730839 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f971c06c980 0x7f971c06ee40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.985+0000 7f9747000640 1 --2- 172.21.15.196:0/54730839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97401469f0 0x7f97400b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.985+0000 7f9747000640 1 --2- 172.21.15.196:0/54730839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97400d1410 0x7f97400b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.985+0000 7f9747000640 1 --2- 172.21.15.196:0/54730839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97400cd420 0x7f974013c370 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:11.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.985+0000 7f9747000640 1 -- 172.21.15.196:0/54730839 >> 172.21.15.196:0/54730839 conn(0x7f97400cd830 msgr2=0x7f97400cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:11.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.985+0000 7f9747000640 1 -- 172.21.15.196:0/54730839 shutdown_connections 2024-04-28T13:33:11.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:11.985+0000 7f9747000640 1 -- 172.21.15.196:0/54730839 wait complete. 2024-04-28T13:33:12.004 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-28T13:33:12.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:12.005 DEBUG:teuthology.orchestra.run.smithi196:> sudo umount /home/ubuntu/cephtest/mnt.2 2024-04-28T13:33:12.103 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:33:12.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:12.103 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-28T13:33:12.151 INFO:tasks.cephfs.mount:Terminating background process 2024-04-28T13:33:12.151 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:33:12.151 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:33:12.209 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:33:12.209 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:33:12.261 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:33:12.469 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.468+0000 7ff66b200640 1 Processor -- start 2024-04-28T13:33:12.469 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.468+0000 7ff66b200640 1 -- start start 2024-04-28T13:33:12.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.470+0000 7ff66b200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff66c15bec0 0x7ff66c06f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:12.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.470+0000 7ff66b200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff66c0704d0 0x7ff66c0cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:12.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.470+0000 7ff66b200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff66c0cea20 con 0x7ff66c06f710 2024-04-28T13:33:12.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.470+0000 7ff66b200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff66c0cecc0 con 0x7ff66c15bec0 2024-04-28T13:33:12.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.470+0000 7ff66b200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff66c0cefa0 con 0x7ff66c0704d0 2024-04-28T13:33:12.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.470+0000 7ff669400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ff66c06f710 0x7ff66c06fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56974/0 (socket says 172.21.15.196:56974) 2024-04-28T13:33:12.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.470+0000 7ff669400640 1 -- 172.21.15.196:0/279213023 learned_addr learned my addr 172.21.15.196:0/279213023 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:12.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.471+0000 7ff66a800640 1 --2- 172.21.15.196:0/279213023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff66c0704d0 0x7ff66c0cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:12.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.471+0000 7ff669e00640 1 --2- 172.21.15.196:0/279213023 >> v2:172.21.15.203:3300/0 conn(0x7ff66c15bec0 0x7ff66c06f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:12.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.471+0000 7ff668a00640 1 -- 172.21.15.196:0/279213023 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 243573340 0 0) 0x7ff66c0cea20 con 0x7ff66c06f710 2024-04-28T13:33:12.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.471+0000 7ff668a00640 1 -- 172.21.15.196:0/279213023 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff650003520 con 0x7ff66c06f710 2024-04-28T13:33:12.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.471+0000 7ff668a00640 1 -- 172.21.15.196:0/279213023 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4058539937 0 0) 0x7ff650003520 con 0x7ff66c06f710 2024-04-28T13:33:12.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.471+0000 7ff668a00640 1 -- 172.21.15.196:0/279213023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff66c0704d0 msgr2=0x7ff66c0cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:12.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.471+0000 7ff668a00640 1 --2- 172.21.15.196:0/279213023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff66c0704d0 0x7ff66c0cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.471+0000 7ff668a00640 1 -- 172.21.15.196:0/279213023 >> v2:172.21.15.203:3300/0 conn(0x7ff66c15bec0 msgr2=0x7ff66c06f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:12.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.471+0000 7ff668a00640 1 --2- 172.21.15.196:0/279213023 >> v2:172.21.15.203:3300/0 conn(0x7ff66c15bec0 0x7ff66c06f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.471+0000 7ff668a00640 1 -- 172.21.15.196:0/279213023 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c0cf280 con 0x7ff66c06f710 2024-04-28T13:33:12.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.471+0000 7ff668a00640 1 -- 172.21.15.196:0/279213023 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7ff658001670 con 0x7ff66c06f710 2024-04-28T13:33:12.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.471+0000 7ff668a00640 1 -- 172.21.15.196:0/279213023 >> v1:172.21.15.196:6789/0 conn(0x7ff66c06f710 legacy=0x7ff66c06fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:12.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.471+0000 7ff668a00640 1 --2- 172.21.15.196:0/279213023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6500039b0 0x7ff650003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:12.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.472+0000 7ff668a00640 1 --2- 172.21.15.196:0/279213023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff66c0704d0 0x7ff650004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:12.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.472+0000 7ff668a00640 1 --2- 172.21.15.196:0/279213023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff650004d40 0x7ff6500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:12.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.472+0000 7ff668a00640 1 -- 172.21.15.196:0/279213023 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c0cecc0 con 0x7ff6500039b0 2024-04-28T13:33:12.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.472+0000 7ff668a00640 1 -- 172.21.15.196:0/279213023 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c0cefa0 con 0x7ff650004d40 2024-04-28T13:33:12.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.472+0000 7ff668a00640 1 -- 172.21.15.196:0/279213023 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff65000b9c0 con 0x7ff66c0704d0 2024-04-28T13:33:12.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.472+0000 7ff669e00640 1 --2- 172.21.15.196:0/279213023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6500039b0 0x7ff650003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:12.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.472+0000 7ff66a800640 1 --2- 172.21.15.196:0/279213023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff650004d40 0x7ff6500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:12.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.472+0000 7ff669e00640 1 -- 172.21.15.196:0/279213023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff66c0704d0 msgr2=0x7ff650004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:12.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.472+0000 7ff669e00640 1 --2- 172.21.15.196:0/279213023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff66c0704d0 0x7ff650004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.472+0000 7ff669e00640 1 -- 172.21.15.196:0/279213023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff650004d40 msgr2=0x7ff6500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:12.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.472+0000 7ff669e00640 1 --2- 172.21.15.196:0/279213023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff650004d40 0x7ff6500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.472+0000 7ff669e00640 1 -- 172.21.15.196:0/279213023 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff650003520 con 0x7ff6500039b0 2024-04-28T13:33:12.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.474+0000 7ff669e00640 1 --2- 172.21.15.196:0/279213023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6500039b0 0x7ff650003dd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009af0 tx=0x7ff650005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89228cbc52033a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:12.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.474+0000 7ff668a00640 1 -- 172.21.15.196:0/279213023 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c036070 con 0x7ff6500039b0 2024-04-28T13:33:12.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.474+0000 7ff668a00640 1 -- 172.21.15.196:0/279213023 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff65c031400 con 0x7ff6500039b0 2024-04-28T13:33:12.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.474+0000 7ff668a00640 1 -- 172.21.15.196:0/279213023 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c009690 con 0x7ff6500039b0 2024-04-28T13:33:12.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.475+0000 7ff66b200640 1 -- 172.21.15.196:0/279213023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6500039b0 msgr2=0x7ff650003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:12.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.475+0000 7ff66b200640 1 --2- 172.21.15.196:0/279213023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6500039b0 0x7ff650003dd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009af0 tx=0x7ff650005830 comp rx=0 tx=0).stop 2024-04-28T13:33:12.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.475+0000 7ff66a800640 1 -- 172.21.15.196:0/279213023 reap_dead start 2024-04-28T13:33:12.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.475+0000 7ff66b200640 1 -- 172.21.15.196:0/279213023 shutdown_connections 2024-04-28T13:33:12.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.475+0000 7ff66b200640 1 -- 172.21.15.196:0/279213023 >> 172.21.15.196:0/279213023 conn(0x7ff66c0d1440 msgr2=0x7ff66c0d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:12.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.475+0000 7ff66b200640 1 -- 172.21.15.196:0/279213023 shutdown_connections 2024-04-28T13:33:12.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.475+0000 7ff66b200640 1 -- 172.21.15.196:0/279213023 wait complete. 2024-04-28T13:33:12.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.477+0000 7ff66b200640 1 Processor -- start 2024-04-28T13:33:12.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.477+0000 7ff66b200640 1 -- start start 2024-04-28T13:33:12.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.478+0000 7ff66b200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6500039b0 0x7ff66c1f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:12.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.478+0000 7ff66b200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff66c147770 0x7ff66c147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:12.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.478+0000 7ff66b200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff66c149800 0x7ff66c149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:12.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.478+0000 7ff66b200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c06d830 con 0x7ff66c147770 2024-04-28T13:33:12.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.478+0000 7ff66b200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c06d6b0 con 0x7ff6500039b0 2024-04-28T13:33:12.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.478+0000 7ff66b200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff66c0cfa80 con 0x7ff66c149800 2024-04-28T13:33:12.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.478+0000 7ff669400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff66c147770 0x7ff66c147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:12.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.478+0000 7ff669400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff66c147770 0x7ff66c147bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34746/0 (socket says 172.21.15.196:34746) 2024-04-28T13:33:12.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.478+0000 7ff669400640 1 -- 172.21.15.196:0/535527194 learned_addr learned my addr 172.21.15.196:0/535527194 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:12.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.479+0000 7ff669e00640 1 --2- 172.21.15.196:0/535527194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6500039b0 0x7ff66c1f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:12.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.479+0000 7ff66a800640 1 --2- 172.21.15.196:0/535527194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff66c149800 0x7ff66c149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:12.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.479+0000 7ff669400640 1 -- 172.21.15.196:0/535527194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff66c149800 msgr2=0x7ff66c149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:12.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.479+0000 7ff669400640 1 --2- 172.21.15.196:0/535527194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff66c149800 0x7ff66c149c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.479+0000 7ff669400640 1 -- 172.21.15.196:0/535527194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6500039b0 msgr2=0x7ff66c1f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:12.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.479+0000 7ff669400640 1 --2- 172.21.15.196:0/535527194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6500039b0 0x7ff66c1f4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.479+0000 7ff669400640 1 -- 172.21.15.196:0/535527194 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c1482e0 con 0x7ff66c147770 2024-04-28T13:33:12.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.479+0000 7ff66a800640 1 --2- 172.21.15.196:0/535527194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff66c149800 0x7ff66c149c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:12.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.479+0000 7ff669400640 1 --2- 172.21.15.196:0/535527194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff66c147770 0x7ff66c147bf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff65800c800 tx=0x7ff65800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:12.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.479+0000 7ff662000640 1 -- 172.21.15.196:0/535527194 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658018070 con 0x7ff66c147770 2024-04-28T13:33:12.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.479+0000 7ff662000640 1 -- 172.21.15.196:0/535527194 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff658007500 con 0x7ff66c147770 2024-04-28T13:33:12.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.479+0000 7ff66b200640 1 -- 172.21.15.196:0/535527194 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66c1485c0 con 0x7ff66c147770 2024-04-28T13:33:12.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.480+0000 7ff662000640 1 -- 172.21.15.196:0/535527194 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658004a20 con 0x7ff66c147770 2024-04-28T13:33:12.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.480+0000 7ff66b200640 1 -- 172.21.15.196:0/535527194 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66c148b80 con 0x7ff66c147770 2024-04-28T13:33:12.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.481+0000 7ff66b200640 1 -- 172.21.15.196:0/535527194 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff66c06a290 con 0x7ff66c147770 2024-04-28T13:33:12.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.482+0000 7ff662000640 1 -- 172.21.15.196:0/535527194 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff65801b070 con 0x7ff66c147770 2024-04-28T13:33:12.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.483+0000 7ff662000640 1 --2- 172.21.15.196:0/535527194 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff64c06c980 0x7ff64c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:12.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.483+0000 7ff662000640 1 -- 172.21.15.196:0/535527194 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7ff65808b1e0 con 0x7ff66c147770 2024-04-28T13:33:12.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.484+0000 7ff669e00640 1 --2- 172.21.15.196:0/535527194 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff64c06c980 0x7ff64c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:12.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.485+0000 7ff669e00640 1 --2- 172.21.15.196:0/535527194 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff64c06c980 0x7ff64c06ee40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff65c033000 tx=0x7ff65c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:12.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.487+0000 7ff662000640 1 -- 172.21.15.196:0/535527194 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff6580594c0 con 0x7ff66c147770 2024-04-28T13:33:12.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.688+0000 7ff66b200640 1 -- 172.21.15.196:0/535527194 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff66c155cf0 con 0x7ff66c147770 2024-04-28T13:33:12.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.691+0000 7ff662000640 1 -- 172.21.15.196:0/535527194 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16834 (secure 0 0 0) 0x7ff658058dc0 con 0x7ff66c147770 2024-04-28T13:33:12.692 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:12.693 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":34,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:58.527505+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:32:29.352348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-28T13:32:30.637498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6836/459640695":"2024-04-29T13:32:58.527451+0000","172.21.15.203:6835/459640695":"2024-04-29T13:32:58.527451+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:33:12.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.694+0000 7ff66b200640 1 -- 172.21.15.196:0/535527194 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff64c06c980 msgr2=0x7ff64c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:12.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.694+0000 7ff66b200640 1 --2- 172.21.15.196:0/535527194 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff64c06c980 0x7ff64c06ee40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff65c033000 tx=0x7ff65c008030 comp rx=0 tx=0).stop 2024-04-28T13:33:12.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.694+0000 7ff66b200640 1 -- 172.21.15.196:0/535527194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff66c147770 msgr2=0x7ff66c147bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:12.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.694+0000 7ff66b200640 1 --2- 172.21.15.196:0/535527194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff66c147770 0x7ff66c147bf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff65800c800 tx=0x7ff65800ccc0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.694+0000 7ff66b200640 1 -- 172.21.15.196:0/535527194 shutdown_connections 2024-04-28T13:33:12.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.694+0000 7ff66b200640 1 --2- 172.21.15.196:0/535527194 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff64c06c980 0x7ff64c06ee40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.694+0000 7ff66b200640 1 --2- 172.21.15.196:0/535527194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff66c149800 0x7ff66c149c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.694+0000 7ff66b200640 1 --2- 172.21.15.196:0/535527194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff66c147770 0x7ff66c147bf0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.694+0000 7ff66b200640 1 --2- 172.21.15.196:0/535527194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6500039b0 0x7ff66c1f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.694+0000 7ff66b200640 1 -- 172.21.15.196:0/535527194 >> 172.21.15.196:0/535527194 conn(0x7ff66c0d1440 msgr2=0x7ff66c155fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:12.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.694+0000 7ff66b200640 1 -- 172.21.15.196:0/535527194 shutdown_connections 2024-04-28T13:33:12.696 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.694+0000 7ff66b200640 1 -- 172.21.15.196:0/535527194 wait complete. 2024-04-28T13:33:12.715 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:32:29.352348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:32:30.637498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:33:12.716 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:33:12.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.926+0000 7f7330000640 1 Processor -- start 2024-04-28T13:33:12.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.926+0000 7f7330000640 1 -- start start 2024-04-28T13:33:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.927+0000 7f7330000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7328143840 0x7f73280b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.927+0000 7f7330000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f73280b55c0 0x7f73280ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.927+0000 7f7330000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f732815dca0 con 0x7f73280b4520 2024-04-28T13:33:12.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.927+0000 7f7330000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7328143c20 con 0x7f73280b55c0 2024-04-28T13:33:12.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.927+0000 7f7330000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f732815df80 con 0x7f7328143840 2024-04-28T13:33:12.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.928+0000 7f732e000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f73280b55c0 0x7f73280ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:12.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.928+0000 7f732e000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f73280b55c0 0x7f73280ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34762/0 (socket says 172.21.15.196:34762) 2024-04-28T13:33:12.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.928+0000 7f732e000640 1 -- 172.21.15.196:0/3019459194 learned_addr learned my addr 172.21.15.196:0/3019459194 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:12.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.928+0000 7f732d600640 1 --2- 172.21.15.196:0/3019459194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7328143840 0x7f73280b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:12.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.928+0000 7f732e000640 1 -- 172.21.15.196:0/3019459194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7328143840 msgr2=0x7f73280b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:12.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.928+0000 7f732e000640 1 --2- 172.21.15.196:0/3019459194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7328143840 0x7f73280b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.928+0000 7f732e000640 1 -- 172.21.15.196:0/3019459194 >> v1:172.21.15.203:6789/0 conn(0x7f73280b4520 legacy=0x7f73280b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:12.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.928+0000 7f732e000640 1 -- 172.21.15.196:0/3019459194 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732815e260 con 0x7f73280b55c0 2024-04-28T13:33:12.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.929+0000 7f732e000640 1 --2- 172.21.15.196:0/3019459194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73280b55c0 0x7f73280ceff0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7320009e50 tx=0x7f732002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d335c26f938be7 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:12.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.930+0000 7f7327e00640 1 -- 172.21.15.196:0/3019459194 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320036070 con 0x7f73280b55c0 2024-04-28T13:33:12.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.930+0000 7f7327e00640 1 -- 172.21.15.196:0/3019459194 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7320031600 con 0x7f73280b55c0 2024-04-28T13:33:12.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.930+0000 7f7327e00640 1 -- 172.21.15.196:0/3019459194 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73200319d0 con 0x7f73280b55c0 2024-04-28T13:33:12.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.930+0000 7f7330000640 1 -- 172.21.15.196:0/3019459194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73280b55c0 msgr2=0x7f73280ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:12.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.930+0000 7f7330000640 1 --2- 172.21.15.196:0/3019459194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73280b55c0 0x7f73280ceff0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7320009e50 tx=0x7f732002a6c0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.931+0000 7f7330000640 1 -- 172.21.15.196:0/3019459194 shutdown_connections 2024-04-28T13:33:12.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.931+0000 7f7330000640 1 --2- 172.21.15.196:0/3019459194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73280b55c0 0x7f73280ceff0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.931+0000 7f7330000640 1 --2- 172.21.15.196:0/3019459194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7328143840 0x7f73280b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.931+0000 7f7330000640 1 -- 172.21.15.196:0/3019459194 >> 172.21.15.196:0/3019459194 conn(0x7f73280cd8a0 msgr2=0x7f73280cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:12.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.931+0000 7f7330000640 1 -- 172.21.15.196:0/3019459194 shutdown_connections 2024-04-28T13:33:12.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.931+0000 7f7330000640 1 -- 172.21.15.196:0/3019459194 wait complete. 2024-04-28T13:33:12.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.933+0000 7f7330000640 1 Processor -- start 2024-04-28T13:33:12.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.933+0000 7f7330000640 1 -- start start 2024-04-28T13:33:12.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.934+0000 7f7330000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73280b4520 0x7f73280aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:12.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.934+0000 7f7330000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73280b55c0 0x7f73280cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:12.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.934+0000 7f7330000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7328143840 0x7f73280ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:12.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.934+0000 7f7330000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f732815da40 con 0x7f73280b55c0 2024-04-28T13:33:12.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.934+0000 7f7330000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f732815d8c0 con 0x7f7328143840 2024-04-28T13:33:12.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.934+0000 7f7330000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f73280cf530 con 0x7f73280b4520 2024-04-28T13:33:12.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.934+0000 7f732cc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73280b55c0 0x7f73280cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:12.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.935+0000 7f732cc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73280b55c0 0x7f73280cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34776/0 (socket says 172.21.15.196:34776) 2024-04-28T13:33:12.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.935+0000 7f732cc00640 1 -- 172.21.15.196:0/2297510019 learned_addr learned my addr 172.21.15.196:0/2297510019 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:12.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.935+0000 7f732d600640 1 --2- 172.21.15.196:0/2297510019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73280b4520 0x7f73280aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:12.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.935+0000 7f732e000640 1 --2- 172.21.15.196:0/2297510019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7328143840 0x7f73280ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:12.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.935+0000 7f732cc00640 1 -- 172.21.15.196:0/2297510019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73280b4520 msgr2=0x7f73280aacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:12.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.935+0000 7f732cc00640 1 --2- 172.21.15.196:0/2297510019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73280b4520 0x7f73280aacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.935+0000 7f732cc00640 1 -- 172.21.15.196:0/2297510019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7328143840 msgr2=0x7f73280ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:12.937 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.935+0000 7f732cc00640 1 --2- 172.21.15.196:0/2297510019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7328143840 0x7f73280ad420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:12.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.935+0000 7f732cc00640 1 -- 172.21.15.196:0/2297510019 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73280cb430 con 0x7f73280b55c0 2024-04-28T13:33:12.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.935+0000 7f732d600640 1 --2- 172.21.15.196:0/2297510019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73280b4520 0x7f73280aacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:12.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.935+0000 7f732cc00640 1 --2- 172.21.15.196:0/2297510019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73280b55c0 0x7f73280cabe0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f731400c430 tx=0x7f731400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:12.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.936+0000 7f7325600640 1 -- 172.21.15.196:0/2297510019 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314011790 con 0x7f73280b55c0 2024-04-28T13:33:12.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.936+0000 7f7325600640 1 -- 172.21.15.196:0/2297510019 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7314017070 con 0x7f73280b55c0 2024-04-28T13:33:12.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.936+0000 7f7325600640 1 -- 172.21.15.196:0/2297510019 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314011bb0 con 0x7f73280b55c0 2024-04-28T13:33:12.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.936+0000 7f7330000640 1 -- 172.21.15.196:0/2297510019 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73280cb710 con 0x7f73280b55c0 2024-04-28T13:33:12.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.936+0000 7f7330000640 1 -- 172.21.15.196:0/2297510019 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73280af7d0 con 0x7f73280b55c0 2024-04-28T13:33:12.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.937+0000 7f7330000640 1 -- 172.21.15.196:0/2297510019 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f732806a290 con 0x7f73280b55c0 2024-04-28T13:33:12.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.938+0000 7f7325600640 1 -- 172.21.15.196:0/2297510019 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f731401c460 con 0x7f73280b55c0 2024-04-28T13:33:12.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.939+0000 7f7325600640 1 --2- 172.21.15.196:0/2297510019 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f730806c980 0x7f730806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:12.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.939+0000 7f7325600640 1 -- 172.21.15.196:0/2297510019 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f731408bc70 con 0x7f73280b55c0 2024-04-28T13:33:12.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.939+0000 7f732d600640 1 --2- 172.21.15.196:0/2297510019 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f730806c980 0x7f730806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:12.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.941+0000 7f732d600640 1 --2- 172.21.15.196:0/2297510019 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f730806c980 0x7f730806ee40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f731c00b820 tx=0x7f731c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:12.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:12.945+0000 7f7325600640 1 -- 172.21.15.196:0/2297510019 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7314059ec0 con 0x7f73280b55c0 2024-04-28T13:33:13.143 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.142+0000 7f7330000640 1 -- 172.21.15.196:0/2297510019 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f73280ce420 con 0x7f73280b55c0 2024-04-28T13:33:13.144 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.143+0000 7f7325600640 1 -- 172.21.15.196:0/2297510019 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f73140597c0 con 0x7f73280b55c0 2024-04-28T13:33:13.144 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:33:13.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.146+0000 7f7330000640 1 -- 172.21.15.196:0/2297510019 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f730806c980 msgr2=0x7f730806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:13.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.146+0000 7f7330000640 1 --2- 172.21.15.196:0/2297510019 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f730806c980 0x7f730806ee40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f731c00b820 tx=0x7f731c008040 comp rx=0 tx=0).stop 2024-04-28T13:33:13.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.146+0000 7f7330000640 1 -- 172.21.15.196:0/2297510019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73280b55c0 msgr2=0x7f73280cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:13.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.146+0000 7f7330000640 1 --2- 172.21.15.196:0/2297510019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73280b55c0 0x7f73280cabe0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f731400c430 tx=0x7f731400c8f0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.146+0000 7f7330000640 1 -- 172.21.15.196:0/2297510019 shutdown_connections 2024-04-28T13:33:13.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.146+0000 7f7330000640 1 --2- 172.21.15.196:0/2297510019 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f730806c980 0x7f730806ee40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.146+0000 7f7330000640 1 --2- 172.21.15.196:0/2297510019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7328143840 0x7f73280ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.146+0000 7f7330000640 1 --2- 172.21.15.196:0/2297510019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73280b55c0 0x7f73280cabe0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.146+0000 7f7330000640 1 --2- 172.21.15.196:0/2297510019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73280b4520 0x7f73280aacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.146+0000 7f7330000640 1 -- 172.21.15.196:0/2297510019 >> 172.21.15.196:0/2297510019 conn(0x7f73280cd8a0 msgr2=0x7f732815b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:13.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.146+0000 7f7330000640 1 -- 172.21.15.196:0/2297510019 shutdown_connections 2024-04-28T13:33:13.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.146+0000 7f7330000640 1 -- 172.21.15.196:0/2297510019 wait complete. 2024-04-28T13:33:13.166 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:33:13.362 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.361+0000 7fc59f000640 1 Processor -- start 2024-04-28T13:33:13.362 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.361+0000 7fc59f000640 1 -- start start 2024-04-28T13:33:13.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.363+0000 7fc59f000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc598143840 0x7fc5980b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.363+0000 7fc59f000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc5980b4520 0x7fc5980b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.363+0000 7fc59f000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc5980b53f0 0x7fc598160e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.363+0000 7fc59f000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc598143c20 con 0x7fc5980b4520 2024-04-28T13:33:13.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.364+0000 7fc59f000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc59815dca0 con 0x7fc5980b53f0 2024-04-28T13:33:13.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.364+0000 7fc59f000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc59815df80 con 0x7fc598143840 2024-04-28T13:33:13.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.364+0000 7fc597400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc5980b4520 0x7fc5980b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:13.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.364+0000 7fc597400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc5980b4520 0x7fc5980b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34790/0 (socket says 172.21.15.196:34790) 2024-04-28T13:33:13.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.364+0000 7fc597400640 1 -- 172.21.15.196:0/1277883620 learned_addr learned my addr 172.21.15.196:0/1277883620 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:13.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.364+0000 7fc59d000640 1 --2- 172.21.15.196:0/1277883620 >> v2:172.21.15.203:3300/0 conn(0x7fc5980b53f0 0x7fc598160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:13.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.364+0000 7fc597400640 1 -- 172.21.15.196:0/1277883620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc598143840 msgr2=0x7fc5980b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:13.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.364+0000 7fc597e00640 1 --2- 172.21.15.196:0/1277883620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc598143840 0x7fc5980b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:13.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.364+0000 7fc597400640 1 --2- 172.21.15.196:0/1277883620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc598143840 0x7fc5980b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.364+0000 7fc597400640 1 -- 172.21.15.196:0/1277883620 >> v2:172.21.15.203:3300/0 conn(0x7fc5980b53f0 msgr2=0x7fc598160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:13.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.364+0000 7fc597400640 1 --2- 172.21.15.196:0/1277883620 >> v2:172.21.15.203:3300/0 conn(0x7fc5980b53f0 0x7fc598160e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.364+0000 7fc597400640 1 -- 172.21.15.196:0/1277883620 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59815e260 con 0x7fc5980b4520 2024-04-28T13:33:13.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.364+0000 7fc59d000640 1 --2- 172.21.15.196:0/1277883620 >> v2:172.21.15.203:3300/0 conn(0x7fc5980b53f0 0x7fc598160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:13.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.365+0000 7fc597400640 1 --2- 172.21.15.196:0/1277883620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5980b4520 0x7fc5980b4eb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc598070480 tx=0x7fc58402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82be80d547008e39 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:13.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.366+0000 7fc596a00640 1 -- 172.21.15.196:0/1277883620 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584037070 con 0x7fc5980b4520 2024-04-28T13:33:13.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.366+0000 7fc596a00640 1 -- 172.21.15.196:0/1277883620 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc584032440 con 0x7fc5980b4520 2024-04-28T13:33:13.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.366+0000 7fc596a00640 1 -- 172.21.15.196:0/1277883620 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584032810 con 0x7fc5980b4520 2024-04-28T13:33:13.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.366+0000 7fc59f000640 1 -- 172.21.15.196:0/1277883620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5980b4520 msgr2=0x7fc5980b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:13.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.366+0000 7fc59f000640 1 --2- 172.21.15.196:0/1277883620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5980b4520 0x7fc5980b4eb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc598070480 tx=0x7fc58402a1e0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.366+0000 7fc59f000640 1 -- 172.21.15.196:0/1277883620 shutdown_connections 2024-04-28T13:33:13.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.366+0000 7fc59f000640 1 --2- 172.21.15.196:0/1277883620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5980b4520 0x7fc5980b4eb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.366+0000 7fc59f000640 1 --2- 172.21.15.196:0/1277883620 >> v2:172.21.15.203:3300/0 conn(0x7fc5980b53f0 0x7fc598160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.366+0000 7fc59f000640 1 --2- 172.21.15.196:0/1277883620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc598143840 0x7fc5980b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.366+0000 7fc59f000640 1 -- 172.21.15.196:0/1277883620 >> 172.21.15.196:0/1277883620 conn(0x7fc5980cd8a0 msgr2=0x7fc5980cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:13.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.366+0000 7fc59f000640 1 -- 172.21.15.196:0/1277883620 shutdown_connections 2024-04-28T13:33:13.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.366+0000 7fc59f000640 1 -- 172.21.15.196:0/1277883620 wait complete. 2024-04-28T13:33:13.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.368+0000 7fc59f000640 1 Processor -- start 2024-04-28T13:33:13.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.368+0000 7fc59f000640 1 -- start start 2024-04-28T13:33:13.371 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.369+0000 7fc59f000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5980b4520 0x7fc598141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.369+0000 7fc59f000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5980b53f0 0x7fc5980c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.369+0000 7fc59f000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc598143840 0x7fc5980ab000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.369+0000 7fc59f000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc59815da40 con 0x7fc5980b53f0 2024-04-28T13:33:13.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.369+0000 7fc59f000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc59815d8c0 con 0x7fc5980b4520 2024-04-28T13:33:13.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.369+0000 7fc59f000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc598161380 con 0x7fc598143840 2024-04-28T13:33:13.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.369+0000 7fc597400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5980b53f0 0x7fc5980c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:13.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.369+0000 7fc597400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5980b53f0 0x7fc5980c8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34796/0 (socket says 172.21.15.196:34796) 2024-04-28T13:33:13.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.369+0000 7fc597400640 1 -- 172.21.15.196:0/1924419721 learned_addr learned my addr 172.21.15.196:0/1924419721 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:13.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.370+0000 7fc59d000640 1 --2- 172.21.15.196:0/1924419721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc598143840 0x7fc5980ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:13.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.370+0000 7fc597e00640 1 --2- 172.21.15.196:0/1924419721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5980b4520 0x7fc598141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:13.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.370+0000 7fc597400640 1 -- 172.21.15.196:0/1924419721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc598143840 msgr2=0x7fc5980ab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:13.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.370+0000 7fc597400640 1 --2- 172.21.15.196:0/1924419721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc598143840 0x7fc5980ab000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.370+0000 7fc597400640 1 -- 172.21.15.196:0/1924419721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5980b4520 msgr2=0x7fc598141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:13.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.370+0000 7fc597400640 1 --2- 172.21.15.196:0/1924419721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5980b4520 0x7fc598141940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.370+0000 7fc597400640 1 -- 172.21.15.196:0/1924419721 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc598141e80 con 0x7fc5980b53f0 2024-04-28T13:33:13.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.370+0000 7fc597400640 1 --2- 172.21.15.196:0/1924419721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5980b53f0 0x7fc5980c8e20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc58402a220 tx=0x7fc5840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:13.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.371+0000 7fc58fe00640 1 -- 172.21.15.196:0/1924419721 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584037040 con 0x7fc5980b53f0 2024-04-28T13:33:13.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.371+0000 7fc58fe00640 1 -- 172.21.15.196:0/1924419721 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc584042070 con 0x7fc5980b53f0 2024-04-28T13:33:13.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.371+0000 7fc58fe00640 1 -- 172.21.15.196:0/1924419721 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584030bb0 con 0x7fc5980b53f0 2024-04-28T13:33:13.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.371+0000 7fc59f000640 1 -- 172.21.15.196:0/1924419721 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5980c98e0 con 0x7fc5980b53f0 2024-04-28T13:33:13.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.371+0000 7fc59f000640 1 -- 172.21.15.196:0/1924419721 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5980c9ea0 con 0x7fc5980b53f0 2024-04-28T13:33:13.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.372+0000 7fc59f000640 1 -- 172.21.15.196:0/1924419721 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc59806a290 con 0x7fc5980b53f0 2024-04-28T13:33:13.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.373+0000 7fc58fe00640 1 -- 172.21.15.196:0/1924419721 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc584047460 con 0x7fc5980b53f0 2024-04-28T13:33:13.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.374+0000 7fc58fe00640 1 --2- 172.21.15.196:0/1924419721 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc57806c980 0x7fc57806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.375+0000 7fc58fe00640 1 -- 172.21.15.196:0/1924419721 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fc5840ac950 con 0x7fc5980b53f0 2024-04-28T13:33:13.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.375+0000 7fc597e00640 1 --2- 172.21.15.196:0/1924419721 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc57806c980 0x7fc57806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:13.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.376+0000 7fc597e00640 1 --2- 172.21.15.196:0/1924419721 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc57806c980 0x7fc57806ee40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc58800b7c0 tx=0x7fc588008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:13.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.379+0000 7fc58fe00640 1 -- 172.21.15.196:0/1924419721 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc584045360 con 0x7fc5980b53f0 2024-04-28T13:33:13.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.577+0000 7fc59f000640 1 -- 172.21.15.196:0/1924419721 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc5980ce420 con 0x7fc5980b53f0 2024-04-28T13:33:13.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.579+0000 7fc58fe00640 1 -- 172.21.15.196:0/1924419721 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7fc58407ac30 con 0x7fc5980b53f0 2024-04-28T13:33:13.580 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:33:13.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.581+0000 7fc59f000640 1 -- 172.21.15.196:0/1924419721 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc57806c980 msgr2=0x7fc57806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:13.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.581+0000 7fc59f000640 1 --2- 172.21.15.196:0/1924419721 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc57806c980 0x7fc57806ee40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc58800b7c0 tx=0x7fc588008040 comp rx=0 tx=0).stop 2024-04-28T13:33:13.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.581+0000 7fc59f000640 1 -- 172.21.15.196:0/1924419721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5980b53f0 msgr2=0x7fc5980c8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:13.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.581+0000 7fc59f000640 1 --2- 172.21.15.196:0/1924419721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5980b53f0 0x7fc5980c8e20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc58402a220 tx=0x7fc5840046c0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.581+0000 7fc59f000640 1 -- 172.21.15.196:0/1924419721 shutdown_connections 2024-04-28T13:33:13.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.581+0000 7fc59f000640 1 --2- 172.21.15.196:0/1924419721 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc57806c980 0x7fc57806ee40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.581+0000 7fc59f000640 1 --2- 172.21.15.196:0/1924419721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc598143840 0x7fc5980ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.581+0000 7fc59f000640 1 --2- 172.21.15.196:0/1924419721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5980b53f0 0x7fc5980c8e20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.581+0000 7fc59f000640 1 --2- 172.21.15.196:0/1924419721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5980b4520 0x7fc598141940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.581+0000 7fc59f000640 1 -- 172.21.15.196:0/1924419721 >> 172.21.15.196:0/1924419721 conn(0x7fc5980cd8a0 msgr2=0x7fc59806f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:13.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.581+0000 7fc59f000640 1 -- 172.21.15.196:0/1924419721 shutdown_connections 2024-04-28T13:33:13.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.581+0000 7fc59f000640 1 -- 172.21.15.196:0/1924419721 wait complete. 2024-04-28T13:33:13.600 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:33:13.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.807+0000 7fc217200640 1 Processor -- start 2024-04-28T13:33:13.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.807+0000 7fc217200640 1 -- start start 2024-04-28T13:33:13.809 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.809+0000 7fc217200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc21015f490 0x7fc2100d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.809+0000 7fc217200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2100d0ff0 0x7fc2100d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.809+0000 7fc217200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2100d4240 con 0x7fc2100d2120 2024-04-28T13:33:13.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.809+0000 7fc217200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc2100d2570 con 0x7fc21015f490 2024-04-28T13:33:13.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.809+0000 7fc217200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc2100d31a0 con 0x7fc2100d0ff0 2024-04-28T13:33:13.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.809+0000 7fc215200640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc2100d2120 0x7fc2100ba3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56980/0 (socket says 172.21.15.196:56980) 2024-04-28T13:33:13.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.809+0000 7fc215200640 1 -- 172.21.15.196:0/813678052 learned_addr learned my addr 172.21.15.196:0/813678052 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:13.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20fe00640 1 --2- 172.21.15.196:0/813678052 >> v2:172.21.15.203:3300/0 conn(0x7fc21015f490 0x7fc2100d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:13.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20f400640 1 --2- 172.21.15.196:0/813678052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2100d0ff0 0x7fc2100d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:13.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 -- 172.21.15.196:0/813678052 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2508103194 0 0) 0x7fc2100d4240 con 0x7fc2100d2120 2024-04-28T13:33:13.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 -- 172.21.15.196:0/813678052 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1f4003520 con 0x7fc2100d2120 2024-04-28T13:33:13.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 -- 172.21.15.196:0/813678052 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3474228560 0 0) 0x7fc1f4003520 con 0x7fc2100d2120 2024-04-28T13:33:13.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 -- 172.21.15.196:0/813678052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2100d0ff0 msgr2=0x7fc2100d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:13.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 --2- 172.21.15.196:0/813678052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2100d0ff0 0x7fc2100d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 -- 172.21.15.196:0/813678052 >> v2:172.21.15.203:3300/0 conn(0x7fc21015f490 msgr2=0x7fc2100d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:13.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 --2- 172.21.15.196:0/813678052 >> v2:172.21.15.203:3300/0 conn(0x7fc21015f490 0x7fc2100d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 -- 172.21.15.196:0/813678052 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2100d3480 con 0x7fc2100d2120 2024-04-28T13:33:13.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 -- 172.21.15.196:0/813678052 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fc208001670 con 0x7fc2100d2120 2024-04-28T13:33:13.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20fe00640 1 --2- 172.21.15.196:0/813678052 >> v2:172.21.15.203:3300/0 conn(0x7fc21015f490 0x7fc2100d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:33:13.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 -- 172.21.15.196:0/813678052 >> v1:172.21.15.196:6789/0 conn(0x7fc2100d2120 legacy=0x7fc2100ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:13.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 --2- 172.21.15.196:0/813678052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2100d0ff0 0x7fc1f40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 --2- 172.21.15.196:0/813678052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f4003f30 0x7fc1f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 --2- 172.21.15.196:0/813678052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f4004d40 0x7fc1f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 -- 172.21.15.196:0/813678052 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2100d2570 con 0x7fc1f4003f30 2024-04-28T13:33:13.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 -- 172.21.15.196:0/813678052 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc2100d31a0 con 0x7fc1f4004d40 2024-04-28T13:33:13.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.810+0000 7fc20ea00640 1 -- 172.21.15.196:0/813678052 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f400ba10 con 0x7fc2100d0ff0 2024-04-28T13:33:13.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.811+0000 7fc20f400640 1 --2- 172.21.15.196:0/813678052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f4003f30 0x7fc1f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:13.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.811+0000 7fc20f400640 1 -- 172.21.15.196:0/813678052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2100d0ff0 msgr2=0x7fc1f40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:13.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.811+0000 7fc20f400640 1 --2- 172.21.15.196:0/813678052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2100d0ff0 0x7fc1f40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.811+0000 7fc20f400640 1 -- 172.21.15.196:0/813678052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f4004d40 msgr2=0x7fc1f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:13.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.811+0000 7fc215200640 1 --2- 172.21.15.196:0/813678052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f4004d40 0x7fc1f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:13.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.811+0000 7fc20f400640 1 --2- 172.21.15.196:0/813678052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f4004d40 0x7fc1f40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.811+0000 7fc20f400640 1 -- 172.21.15.196:0/813678052 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f4003520 con 0x7fc1f4003f30 2024-04-28T13:33:13.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.812+0000 7fc20f400640 1 --2- 172.21.15.196:0/813678052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f4003f30 0x7fc1f4004800 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009a90 tx=0x7fc1f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a00a8a1da7c0127 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:13.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.812+0000 7fc20ea00640 1 -- 172.21.15.196:0/813678052 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc036070 con 0x7fc1f4003f30 2024-04-28T13:33:13.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.812+0000 7fc20ea00640 1 -- 172.21.15.196:0/813678052 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1fc031400 con 0x7fc1f4003f30 2024-04-28T13:33:13.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.812+0000 7fc20ea00640 1 -- 172.21.15.196:0/813678052 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc009690 con 0x7fc1f4003f30 2024-04-28T13:33:13.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.813+0000 7fc217200640 1 -- 172.21.15.196:0/813678052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f4003f30 msgr2=0x7fc1f4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:13.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.813+0000 7fc217200640 1 --2- 172.21.15.196:0/813678052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f4003f30 0x7fc1f4004800 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009a90 tx=0x7fc1f4005830 comp rx=0 tx=0).stop 2024-04-28T13:33:13.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.813+0000 7fc215200640 1 -- 172.21.15.196:0/813678052 reap_dead start 2024-04-28T13:33:13.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.813+0000 7fc217200640 1 -- 172.21.15.196:0/813678052 shutdown_connections 2024-04-28T13:33:13.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.813+0000 7fc217200640 1 -- 172.21.15.196:0/813678052 >> 172.21.15.196:0/813678052 conn(0x7fc21009e530 msgr2=0x7fc2100d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:13.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.813+0000 7fc217200640 1 -- 172.21.15.196:0/813678052 shutdown_connections 2024-04-28T13:33:13.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.813+0000 7fc217200640 1 -- 172.21.15.196:0/813678052 wait complete. 2024-04-28T13:33:13.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.815+0000 7fc217200640 1 Processor -- start 2024-04-28T13:33:13.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.815+0000 7fc217200640 1 -- start start 2024-04-28T13:33:13.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.816+0000 7fc217200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f4003f30 0x7fc21015c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.816+0000 7fc217200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2101f5e10 0x7fc210160e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.816+0000 7fc217200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2101613a0 0x7fc2100cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.816+0000 7fc217200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2100d3d20 con 0x7fc1f4003f30 2024-04-28T13:33:13.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.816+0000 7fc217200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc2100d3a40 con 0x7fc2101613a0 2024-04-28T13:33:13.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.816+0000 7fc217200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc21015f870 con 0x7fc2101f5e10 2024-04-28T13:33:13.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.816+0000 7fc20fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f4003f30 0x7fc21015c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:13.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.816+0000 7fc20fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f4003f30 0x7fc21015c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34812/0 (socket says 172.21.15.196:34812) 2024-04-28T13:33:13.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.816+0000 7fc20fe00640 1 -- 172.21.15.196:0/2951046558 learned_addr learned my addr 172.21.15.196:0/2951046558 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:13.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.816+0000 7fc215200640 1 --2- 172.21.15.196:0/2951046558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2101613a0 0x7fc2100cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:13.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.817+0000 7fc20f400640 1 --2- 172.21.15.196:0/2951046558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2101f5e10 0x7fc210160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:13.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.817+0000 7fc20fe00640 1 -- 172.21.15.196:0/2951046558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2101f5e10 msgr2=0x7fc210160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:13.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.817+0000 7fc20fe00640 1 --2- 172.21.15.196:0/2951046558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2101f5e10 0x7fc210160e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.817+0000 7fc20fe00640 1 -- 172.21.15.196:0/2951046558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2101613a0 msgr2=0x7fc2100cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:13.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.817+0000 7fc20fe00640 1 --2- 172.21.15.196:0/2951046558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2101613a0 0x7fc2100cad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:13.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.817+0000 7fc20fe00640 1 -- 172.21.15.196:0/2951046558 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2100cb970 con 0x7fc1f4003f30 2024-04-28T13:33:13.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.817+0000 7fc20fe00640 1 --2- 172.21.15.196:0/2951046558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f4003f30 0x7fc21015c670 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc200007fa0 tx=0x7fc20000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:13.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.817+0000 7fc207e00640 1 -- 172.21.15.196:0/2951046558 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20001a070 con 0x7fc1f4003f30 2024-04-28T13:33:13.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.817+0000 7fc207e00640 1 -- 172.21.15.196:0/2951046558 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc200003b90 con 0x7fc1f4003f30 2024-04-28T13:33:13.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.817+0000 7fc217200640 1 -- 172.21.15.196:0/2951046558 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2101f5330 con 0x7fc1f4003f30 2024-04-28T13:33:13.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.817+0000 7fc207e00640 1 -- 172.21.15.196:0/2951046558 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc200004030 con 0x7fc1f4003f30 2024-04-28T13:33:13.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.817+0000 7fc217200640 1 -- 172.21.15.196:0/2951046558 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc210162400 con 0x7fc1f4003f30 2024-04-28T13:33:13.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.819+0000 7fc217200640 1 -- 172.21.15.196:0/2951046558 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc21006a290 con 0x7fc1f4003f30 2024-04-28T13:33:13.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.820+0000 7fc207e00640 1 -- 172.21.15.196:0/2951046558 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc20001d070 con 0x7fc1f4003f30 2024-04-28T13:33:13.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.821+0000 7fc207e00640 1 --2- 172.21.15.196:0/2951046558 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc1f006c9b0 0x7fc1f006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:13.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.821+0000 7fc207e00640 1 -- 172.21.15.196:0/2951046558 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fc20008c160 con 0x7fc1f4003f30 2024-04-28T13:33:13.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.822+0000 7fc20f400640 1 --2- 172.21.15.196:0/2951046558 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc1f006c9b0 0x7fc1f006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:13.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.823+0000 7fc20f400640 1 --2- 172.21.15.196:0/2951046558 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc1f006c9b0 0x7fc1f006ee70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc033000 tx=0x7fc1fc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:13.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:13.825+0000 7fc207e00640 1 -- 172.21.15.196:0/2951046558 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc200010070 con 0x7fc1f4003f30 2024-04-28T13:33:14.025 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.024+0000 7fc217200640 1 -- 172.21.15.196:0/2951046558 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc210161780 con 0x7fc1f4003f30 2024-04-28T13:33:14.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.025+0000 7fc207e00640 1 -- 172.21.15.196:0/2951046558 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7fc20005a330 con 0x7fc1f4003f30 2024-04-28T13:33:14.027 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:33:14.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.027+0000 7fc217200640 1 -- 172.21.15.196:0/2951046558 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc1f006c9b0 msgr2=0x7fc1f006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:14.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.027+0000 7fc217200640 1 --2- 172.21.15.196:0/2951046558 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc1f006c9b0 0x7fc1f006ee70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc033000 tx=0x7fc1fc008030 comp rx=0 tx=0).stop 2024-04-28T13:33:14.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.027+0000 7fc217200640 1 -- 172.21.15.196:0/2951046558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f4003f30 msgr2=0x7fc21015c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:14.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.027+0000 7fc217200640 1 --2- 172.21.15.196:0/2951046558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f4003f30 0x7fc21015c670 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc200007fa0 tx=0x7fc20000d560 comp rx=0 tx=0).stop 2024-04-28T13:33:14.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.028+0000 7fc217200640 1 -- 172.21.15.196:0/2951046558 shutdown_connections 2024-04-28T13:33:14.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.028+0000 7fc217200640 1 --2- 172.21.15.196:0/2951046558 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc1f006c9b0 0x7fc1f006ee70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.028+0000 7fc217200640 1 --2- 172.21.15.196:0/2951046558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2101613a0 0x7fc2100cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.028+0000 7fc217200640 1 --2- 172.21.15.196:0/2951046558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2101f5e10 0x7fc210160e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.028+0000 7fc217200640 1 --2- 172.21.15.196:0/2951046558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f4003f30 0x7fc21015c670 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.028+0000 7fc217200640 1 -- 172.21.15.196:0/2951046558 >> 172.21.15.196:0/2951046558 conn(0x7fc21009e530 msgr2=0x7fc21006cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:14.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.028+0000 7fc217200640 1 -- 172.21.15.196:0/2951046558 shutdown_connections 2024-04-28T13:33:14.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.028+0000 7fc217200640 1 -- 172.21.15.196:0/2951046558 wait complete. 2024-04-28T13:33:14.048 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T13:33:14.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.245+0000 7f76bd800640 1 Processor -- start 2024-04-28T13:33:14.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.246+0000 7f76bd800640 1 -- start start 2024-04-28T13:33:14.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.247+0000 7f76bd800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f76b8163bf0 0x7f76b8163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:14.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.247+0000 7f76bd800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b8162da0 con 0x7f76b8163570 2024-04-28T13:33:14.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.247+0000 7f76bd800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b80b6190 con 0x7f76b81629c0 2024-04-28T13:33:14.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.247+0000 7f76bd800640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f76b80d3e20 con 0x7f76b8163bf0 2024-04-28T13:33:14.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.247+0000 7f76bce00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f76b8163570 0x7f76b80b2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56986/0 (socket says 172.21.15.196:56986) 2024-04-28T13:33:14.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.247+0000 7f76bce00640 1 -- 172.21.15.196:0/2636638324 learned_addr learned my addr 172.21.15.196:0/2636638324 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:14.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.248+0000 7f76b7400640 1 --2- 172.21.15.196:0/2636638324 >> v2:172.21.15.203:3300/0 conn(0x7f76b8163bf0 0x7f76b8163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:14.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.248+0000 7f76b6a00640 1 -- 172.21.15.196:0/2636638324 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3185087702 0 0) 0x7f76b8162da0 con 0x7f76b8163570 2024-04-28T13:33:14.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.248+0000 7f76b6a00640 1 -- 172.21.15.196:0/2636638324 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7698003520 con 0x7f76b8163570 2024-04-28T13:33:14.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.248+0000 7f76b6a00640 1 -- 172.21.15.196:0/2636638324 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 809564912 0 0) 0x7f7698003520 con 0x7f76b8163570 2024-04-28T13:33:14.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.248+0000 7f76b6a00640 1 -- 172.21.15.196:0/2636638324 >> v2:172.21.15.203:3300/0 conn(0x7f76b8163bf0 msgr2=0x7f76b8163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:14.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.248+0000 7f76b6a00640 1 --2- 172.21.15.196:0/2636638324 >> v2:172.21.15.203:3300/0 conn(0x7f76b8163bf0 0x7f76b8163030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.248+0000 7f76b6a00640 1 -- 172.21.15.196:0/2636638324 >> v1:172.21.15.203:6789/0 conn(0x7f76b81629c0 legacy=0x7f76b80d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:14.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.248+0000 7f76b6a00640 1 -- 172.21.15.196:0/2636638324 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b80b6470 con 0x7f76b8163570 2024-04-28T13:33:14.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.248+0000 7f76b6a00640 1 -- 172.21.15.196:0/2636638324 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f76b0001670 con 0x7f76b8163570 2024-04-28T13:33:14.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.248+0000 7f76b6a00640 1 -- 172.21.15.196:0/2636638324 >> v1:172.21.15.196:6789/0 conn(0x7f76b8163570 legacy=0x7f76b80b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:14.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.248+0000 7f76b6a00640 1 --2- 172.21.15.196:0/2636638324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7698003a00 0x7f7698003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.248+0000 7f76b6a00640 1 --2- 172.21.15.196:0/2636638324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7698004800 0x7f7698004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.249+0000 7f76b6a00640 1 --2- 172.21.15.196:0/2636638324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76980051c0 0x7f769800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.249+0000 7f76b6a00640 1 -- 172.21.15.196:0/2636638324 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f76b80d3e20 con 0x7f7698003a00 2024-04-28T13:33:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.249+0000 7f76b6a00640 1 -- 172.21.15.196:0/2636638324 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f769800c0c0 con 0x7f7698004800 2024-04-28T13:33:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.249+0000 7f76b6a00640 1 -- 172.21.15.196:0/2636638324 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f769800c2c0 con 0x7f76980051c0 2024-04-28T13:33:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.249+0000 7f76b7e00640 1 --2- 172.21.15.196:0/2636638324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7698003a00 0x7f7698003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.249+0000 7f76b7e00640 1 -- 172.21.15.196:0/2636638324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76980051c0 msgr2=0x7f769800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.249+0000 7f76b7e00640 1 --2- 172.21.15.196:0/2636638324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76980051c0 0x7f769800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.249+0000 7f76b7e00640 1 -- 172.21.15.196:0/2636638324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7698004800 msgr2=0x7f7698004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.249+0000 7f76b7e00640 1 --2- 172.21.15.196:0/2636638324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7698004800 0x7f7698004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.249+0000 7f76b7e00640 1 -- 172.21.15.196:0/2636638324 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7698003520 con 0x7f7698003a00 2024-04-28T13:33:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.249+0000 7f76bce00640 1 -- 172.21.15.196:0/2636638324 reap_dead start 2024-04-28T13:33:14.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.250+0000 7f76b7e00640 1 --2- 172.21.15.196:0/2636638324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7698003a00 0x7f7698003e20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f76a8001750 tx=0x7f7698005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f45eecdf36c2bee server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:14.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.250+0000 7f76b6a00640 1 -- 172.21.15.196:0/2636638324 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8001370 con 0x7f7698003a00 2024-04-28T13:33:14.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.250+0000 7f76b6a00640 1 -- 172.21.15.196:0/2636638324 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a80052d0 con 0x7f7698003a00 2024-04-28T13:33:14.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.250+0000 7f76b6a00640 1 -- 172.21.15.196:0/2636638324 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a802abc0 con 0x7f7698003a00 2024-04-28T13:33:14.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.251+0000 7f76bd800640 1 -- 172.21.15.196:0/2636638324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7698003a00 msgr2=0x7f7698003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:14.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.251+0000 7f76bd800640 1 --2- 172.21.15.196:0/2636638324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7698003a00 0x7f7698003e20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f76a8001750 tx=0x7f7698005830 comp rx=0 tx=0).stop 2024-04-28T13:33:14.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.251+0000 7f76bd800640 1 -- 172.21.15.196:0/2636638324 shutdown_connections 2024-04-28T13:33:14.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.251+0000 7f76bd800640 1 --2- 172.21.15.196:0/2636638324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7698003a00 0x7f7698003e20 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.251+0000 7f76bd800640 1 -- 172.21.15.196:0/2636638324 >> 172.21.15.196:0/2636638324 conn(0x7f76b809e570 msgr2=0x7f76b80d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:14.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.251+0000 7f76bd800640 1 -- 172.21.15.196:0/2636638324 shutdown_connections 2024-04-28T13:33:14.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.251+0000 7f76bd800640 1 -- 172.21.15.196:0/2636638324 wait complete. 2024-04-28T13:33:14.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.253+0000 7f76bd800640 1 Processor -- start 2024-04-28T13:33:14.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.253+0000 7f76bd800640 1 -- start start 2024-04-28T13:33:14.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.254+0000 7f76bd800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7698003a00 0x7f76b81f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:14.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.254+0000 7f76bd800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76b80cc570 0x7f76b81f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:14.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.254+0000 7f76bd800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76b80cc9f0 0x7f76b81f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:14.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.254+0000 7f76bd800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f76b80b1e60 con 0x7f76b80cc570 2024-04-28T13:33:14.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.254+0000 7f76bd800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f76b80b6a30 con 0x7f76b80cc9f0 2024-04-28T13:33:14.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.254+0000 7f76bd800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f76b8163030 con 0x7f7698003a00 2024-04-28T13:33:14.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.254+0000 7f76b7400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76b80cc570 0x7f76b81f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:14.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.254+0000 7f76b7400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76b80cc570 0x7f76b81f7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34836/0 (socket says 172.21.15.196:34836) 2024-04-28T13:33:14.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.254+0000 7f76b7400640 1 -- 172.21.15.196:0/1298926055 learned_addr learned my addr 172.21.15.196:0/1298926055 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:14.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.254+0000 7f76b7e00640 1 --2- 172.21.15.196:0/1298926055 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7698003a00 0x7f76b81f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:14.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.254+0000 7f76bce00640 1 --2- 172.21.15.196:0/1298926055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76b80cc9f0 0x7f76b81f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:14.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.255+0000 7f76b7400640 1 -- 172.21.15.196:0/1298926055 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7698003a00 msgr2=0x7f76b81f53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:14.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.255+0000 7f76b7400640 1 --2- 172.21.15.196:0/1298926055 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7698003a00 0x7f76b81f53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.255+0000 7f76b7400640 1 -- 172.21.15.196:0/1298926055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76b80cc9f0 msgr2=0x7f76b81f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:14.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.255+0000 7f76b7400640 1 --2- 172.21.15.196:0/1298926055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76b80cc9f0 0x7f76b81f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.255+0000 7f76b7400640 1 -- 172.21.15.196:0/1298926055 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b80d0410 con 0x7f76b80cc570 2024-04-28T13:33:14.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.255+0000 7f76bce00640 1 --2- 172.21.15.196:0/1298926055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76b80cc9f0 0x7f76b81f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:14.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.255+0000 7f76b7400640 1 --2- 172.21.15.196:0/1298926055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76b80cc570 0x7f76b81f7d00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009d60 tx=0x7f76a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:14.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.255+0000 7f76afe00640 1 -- 172.21.15.196:0/1298926055 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0018020 con 0x7f76b80cc570 2024-04-28T13:33:14.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.255+0000 7f76afe00640 1 -- 172.21.15.196:0/1298926055 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a0014480 con 0x7f76b80cc570 2024-04-28T13:33:14.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.255+0000 7f76afe00640 1 -- 172.21.15.196:0/1298926055 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0014870 con 0x7f76b80cc570 2024-04-28T13:33:14.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.255+0000 7f76bd800640 1 -- 172.21.15.196:0/1298926055 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b80d06f0 con 0x7f76b80cc570 2024-04-28T13:33:14.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.255+0000 7f76bd800640 1 -- 172.21.15.196:0/1298926055 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b80d0cb0 con 0x7f76b80cc570 2024-04-28T13:33:14.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.257+0000 7f76bd800640 1 -- 172.21.15.196:0/1298926055 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b806a290 con 0x7f76b80cc570 2024-04-28T13:33:14.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.257+0000 7f76afe00640 1 -- 172.21.15.196:0/1298926055 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f76a0005d10 con 0x7f76b80cc570 2024-04-28T13:33:14.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.258+0000 7f76afe00640 1 --2- 172.21.15.196:0/1298926055 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f769406c980 0x7f769406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:14.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.258+0000 7f76afe00640 1 -- 172.21.15.196:0/1298926055 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f76a008cad0 con 0x7f76b80cc570 2024-04-28T13:33:14.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.259+0000 7f76b7e00640 1 --2- 172.21.15.196:0/1298926055 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f769406c980 0x7f769406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:14.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.260+0000 7f76b7e00640 1 --2- 172.21.15.196:0/1298926055 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f769406c980 0x7f769406ee40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f76a8032000 tx=0x7f76a802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:14.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.263+0000 7f76afe00640 1 -- 172.21.15.196:0/1298926055 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f76a0022030 con 0x7f76b80cc570 2024-04-28T13:33:14.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.478+0000 7f76bd800640 1 -- 172.21.15.196:0/1298926055 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f76b809ea50 con 0x7f76b80cc570 2024-04-28T13:33:14.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.479+0000 7f76afe00640 1 -- 172.21.15.196:0/1298926055 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v34) ==== 80+0+122 (secure 0 0 0) 0x7f76a0010040 con 0x7f76b80cc570 2024-04-28T13:33:14.480 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.203:6836/459640695 2024-04-29T13:32:58.527451+0000 2024-04-28T13:33:14.480 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.203:6835/459640695 2024-04-29T13:32:58.527451+0000 2024-04-28T13:33:14.481 INFO:teuthology.orchestra.run.smithi196.stderr:listed 2 entries 2024-04-28T13:33:14.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.482+0000 7f76bd800640 1 -- 172.21.15.196:0/1298926055 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f769406c980 msgr2=0x7f769406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:14.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.482+0000 7f76bd800640 1 --2- 172.21.15.196:0/1298926055 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f769406c980 0x7f769406ee40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f76a8032000 tx=0x7f76a802e000 comp rx=0 tx=0).stop 2024-04-28T13:33:14.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.482+0000 7f76bd800640 1 -- 172.21.15.196:0/1298926055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76b80cc570 msgr2=0x7f76b81f7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:14.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.482+0000 7f76bd800640 1 --2- 172.21.15.196:0/1298926055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76b80cc570 0x7f76b81f7d00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009d60 tx=0x7f76a000c490 comp rx=0 tx=0).stop 2024-04-28T13:33:14.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.482+0000 7f76bd800640 1 -- 172.21.15.196:0/1298926055 shutdown_connections 2024-04-28T13:33:14.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.482+0000 7f76bd800640 1 --2- 172.21.15.196:0/1298926055 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f769406c980 0x7f769406ee40 secure :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f76a8032000 tx=0x7f76a802e000 comp rx=0 tx=0).stop 2024-04-28T13:33:14.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.482+0000 7f76bd800640 1 --2- 172.21.15.196:0/1298926055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76b80cc9f0 0x7f76b81f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.482+0000 7f76bd800640 1 --2- 172.21.15.196:0/1298926055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76b80cc570 0x7f76b81f7d00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.482+0000 7f76bd800640 1 --2- 172.21.15.196:0/1298926055 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7698003a00 0x7f76b81f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.482+0000 7f76bd800640 1 -- 172.21.15.196:0/1298926055 >> 172.21.15.196:0/1298926055 conn(0x7f76b809e570 msgr2=0x7f76b8160760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:14.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.483+0000 7f76bd800640 1 -- 172.21.15.196:0/1298926055 shutdown_connections 2024-04-28T13:33:14.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.483+0000 7f76bd800640 1 -- 172.21.15.196:0/1298926055 wait complete. 2024-04-28T13:33:14.503 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-28T13:33:14.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:14.503 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.3 2024-04-28T13:33:14.596 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:33:14.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:14.596 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-28T13:33:14.642 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:33:14.854 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.853+0000 7fc63ee00640 1 Processor -- start 2024-04-28T13:33:14.854 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.853+0000 7fc63ee00640 1 -- start start 2024-04-28T13:33:14.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.854+0000 7fc63ee00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc638073fc0 0x7fc638072020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:14.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.855+0000 7fc63ee00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc638162e50 0x7fc638163290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:14.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.855+0000 7fc63ee00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc6380d0b90 0x7fc6380d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:14.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.855+0000 7fc63ee00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc638072560 con 0x7fc6380d0b90 2024-04-28T13:33:14.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.855+0000 7fc63ee00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc638163910 con 0x7fc638162e50 2024-04-28T13:33:14.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.855+0000 7fc63ee00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc638163bf0 con 0x7fc638073fc0 2024-04-28T13:33:14.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.855+0000 7fc63ce00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc6380d0b90 0x7fc6380d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:14.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.855+0000 7fc63ce00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc6380d0b90 0x7fc6380d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34848/0 (socket says 172.21.15.196:34848) 2024-04-28T13:33:14.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.855+0000 7fc63ce00640 1 -- 172.21.15.196:0/311183046 learned_addr learned my addr 172.21.15.196:0/311183046 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:14.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.855+0000 7fc637e00640 1 --2- 172.21.15.196:0/311183046 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc638073fc0 0x7fc638072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:14.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.855+0000 7fc637400640 1 --2- 172.21.15.196:0/311183046 >> v2:172.21.15.203:3300/0 conn(0x7fc638162e50 0x7fc638163290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:14.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.855+0000 7fc63ce00640 1 -- 172.21.15.196:0/311183046 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc638073fc0 msgr2=0x7fc638072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:14.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.855+0000 7fc63ce00640 1 --2- 172.21.15.196:0/311183046 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc638073fc0 0x7fc638072020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.855+0000 7fc63ce00640 1 -- 172.21.15.196:0/311183046 >> v2:172.21.15.203:3300/0 conn(0x7fc638162e50 msgr2=0x7fc638163290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:14.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.855+0000 7fc63ce00640 1 --2- 172.21.15.196:0/311183046 >> v2:172.21.15.203:3300/0 conn(0x7fc638162e50 0x7fc638163290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.855+0000 7fc63ce00640 1 -- 172.21.15.196:0/311183046 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63806ea50 con 0x7fc6380d0b90 2024-04-28T13:33:14.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.856+0000 7fc637e00640 1 --2- 172.21.15.196:0/311183046 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc638073fc0 0x7fc638072020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:14.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.856+0000 7fc63ce00640 1 --2- 172.21.15.196:0/311183046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6380d0b90 0x7fc6380d0fe0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc6380d1e70 tx=0x7fc63002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9367de7ac2e1a34d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:14.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.857+0000 7fc636a00640 1 -- 172.21.15.196:0/311183046 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630036070 con 0x7fc6380d0b90 2024-04-28T13:33:14.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.857+0000 7fc636a00640 1 -- 172.21.15.196:0/311183046 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc630031600 con 0x7fc6380d0b90 2024-04-28T13:33:14.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.857+0000 7fc636a00640 1 -- 172.21.15.196:0/311183046 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6300319d0 con 0x7fc6380d0b90 2024-04-28T13:33:14.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.858+0000 7fc63ee00640 1 -- 172.21.15.196:0/311183046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6380d0b90 msgr2=0x7fc6380d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:14.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.858+0000 7fc63ee00640 1 --2- 172.21.15.196:0/311183046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6380d0b90 0x7fc6380d0fe0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc6380d1e70 tx=0x7fc63002a730 comp rx=0 tx=0).stop 2024-04-28T13:33:14.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.858+0000 7fc63ee00640 1 -- 172.21.15.196:0/311183046 shutdown_connections 2024-04-28T13:33:14.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.858+0000 7fc63ee00640 1 --2- 172.21.15.196:0/311183046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6380d0b90 0x7fc6380d0fe0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.858+0000 7fc63ee00640 1 --2- 172.21.15.196:0/311183046 >> v2:172.21.15.203:3300/0 conn(0x7fc638162e50 0x7fc638163290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.858+0000 7fc63ee00640 1 --2- 172.21.15.196:0/311183046 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc638073fc0 0x7fc638072020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.858+0000 7fc63ee00640 1 -- 172.21.15.196:0/311183046 >> 172.21.15.196:0/311183046 conn(0x7fc6380cd830 msgr2=0x7fc6380cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:14.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.858+0000 7fc63ee00640 1 -- 172.21.15.196:0/311183046 shutdown_connections 2024-04-28T13:33:14.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.858+0000 7fc63ee00640 1 -- 172.21.15.196:0/311183046 wait complete. 2024-04-28T13:33:14.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.860+0000 7fc63ee00640 1 Processor -- start 2024-04-28T13:33:14.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.860+0000 7fc63ee00640 1 -- start start 2024-04-28T13:33:14.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.861+0000 7fc63ee00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc638073fc0 0x7fc6380b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:14.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.861+0000 7fc63ee00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6380d0b90 0x7fc63815ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:14.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.861+0000 7fc63ee00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638162e50 0x7fc6380b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:14.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.861+0000 7fc63ee00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6380d1520 con 0x7fc638162e50 2024-04-28T13:33:14.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.861+0000 7fc63ee00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc638148de0 con 0x7fc638073fc0 2024-04-28T13:33:14.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.861+0000 7fc63ee00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc6380743a0 con 0x7fc6380d0b90 2024-04-28T13:33:14.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.862+0000 7fc63ce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638162e50 0x7fc6380b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:14.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.862+0000 7fc63ce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638162e50 0x7fc6380b63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34856/0 (socket says 172.21.15.196:34856) 2024-04-28T13:33:14.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.862+0000 7fc63ce00640 1 -- 172.21.15.196:0/2325913321 learned_addr learned my addr 172.21.15.196:0/2325913321 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:14.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.862+0000 7fc637e00640 1 --2- 172.21.15.196:0/2325913321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc638073fc0 0x7fc6380b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:14.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.862+0000 7fc637400640 1 --2- 172.21.15.196:0/2325913321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6380d0b90 0x7fc63815ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:14.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.862+0000 7fc63ce00640 1 -- 172.21.15.196:0/2325913321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6380d0b90 msgr2=0x7fc63815ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:14.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.862+0000 7fc63ce00640 1 --2- 172.21.15.196:0/2325913321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6380d0b90 0x7fc63815ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.862+0000 7fc63ce00640 1 -- 172.21.15.196:0/2325913321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc638073fc0 msgr2=0x7fc6380b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:14.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.862+0000 7fc63ce00640 1 --2- 172.21.15.196:0/2325913321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc638073fc0 0x7fc6380b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:14.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.862+0000 7fc63ce00640 1 -- 172.21.15.196:0/2325913321 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63815f2c0 con 0x7fc638162e50 2024-04-28T13:33:14.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.862+0000 7fc63ce00640 1 --2- 172.21.15.196:0/2325913321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638162e50 0x7fc6380b63c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc630009f40 tx=0x7fc630004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:14.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.863+0000 7fc62fe00640 1 -- 172.21.15.196:0/2325913321 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630041030 con 0x7fc638162e50 2024-04-28T13:33:14.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.863+0000 7fc62fe00640 1 -- 172.21.15.196:0/2325913321 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc630031400 con 0x7fc638162e50 2024-04-28T13:33:14.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.863+0000 7fc62fe00640 1 -- 172.21.15.196:0/2325913321 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630036040 con 0x7fc638162e50 2024-04-28T13:33:14.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.863+0000 7fc63ee00640 1 -- 172.21.15.196:0/2325913321 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc638133320 con 0x7fc638162e50 2024-04-28T13:33:14.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.863+0000 7fc63ee00640 1 -- 172.21.15.196:0/2325913321 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6381338e0 con 0x7fc638162e50 2024-04-28T13:33:14.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.864+0000 7fc63ee00640 1 -- 172.21.15.196:0/2325913321 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc63806a290 con 0x7fc638162e50 2024-04-28T13:33:14.866 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.865+0000 7fc62fe00640 1 -- 172.21.15.196:0/2325913321 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc630044070 con 0x7fc638162e50 2024-04-28T13:33:14.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.866+0000 7fc62fe00640 1 --2- 172.21.15.196:0/2325913321 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc61806c980 0x7fc61806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:14.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.866+0000 7fc62fe00640 1 -- 172.21.15.196:0/2325913321 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fc6300ac670 con 0x7fc638162e50 2024-04-28T13:33:14.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.867+0000 7fc637e00640 1 --2- 172.21.15.196:0/2325913321 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc61806c980 0x7fc61806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:14.868 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.868+0000 7fc637e00640 1 --2- 172.21.15.196:0/2325913321 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc61806c980 0x7fc61806ee40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc6380c6fd0 tx=0x7fc628008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:14.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:14.870+0000 7fc62fe00640 1 -- 172.21.15.196:0/2325913321 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc63004b070 con 0x7fc638162e50 2024-04-28T13:33:15.112 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.111+0000 7fc63ee00640 1 -- 172.21.15.196:0/2325913321 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc638148930 con 0x7fc638162e50 2024-04-28T13:33:15.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.112+0000 7fc62fe00640 1 -- 172.21.15.196:0/2325913321 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5191 (secure 0 0 0) 0x7fc63007a8d0 con 0x7fc638162e50 2024-04-28T13:33:15.113 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:15.113 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/931076983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":931076983},{"type":"v1","addr":"172.21.15.196:6834","nonce":931076983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1114474485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1114474485},{"type":"v1","addr":"172.21.15.196:6835","nonce":1114474485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2070123371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2070123371},{"type":"v1","addr":"172.21.15.203:6837","nonce":2070123371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:32:31.669472+0000","modified":"2024-04-28T13:33:02.550473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4694},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4694":{"gid":4694,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6836/1998460400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1998460400},{"type":"v1","addr":"172.21.15.203:6836","nonce":1998460400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4694,"qdb_cluster":[4694]},"id":2}]} 2024-04-28T13:33:15.114 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 22 2024-04-28T13:33:15.116 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.115+0000 7fc63ee00640 1 -- 172.21.15.196:0/2325913321 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc61806c980 msgr2=0x7fc61806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:15.116 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.115+0000 7fc63ee00640 1 --2- 172.21.15.196:0/2325913321 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc61806c980 0x7fc61806ee40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc6380c6fd0 tx=0x7fc628008040 comp rx=0 tx=0).stop 2024-04-28T13:33:15.116 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.115+0000 7fc63ee00640 1 -- 172.21.15.196:0/2325913321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638162e50 msgr2=0x7fc6380b63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:15.116 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.115+0000 7fc63ee00640 1 --2- 172.21.15.196:0/2325913321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638162e50 0x7fc6380b63c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc630009f40 tx=0x7fc630004a40 comp rx=0 tx=0).stop 2024-04-28T13:33:15.116 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.115+0000 7fc63ee00640 1 -- 172.21.15.196:0/2325913321 shutdown_connections 2024-04-28T13:33:15.116 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.115+0000 7fc63ee00640 1 --2- 172.21.15.196:0/2325913321 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc61806c980 0x7fc61806ee40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.115+0000 7fc63ee00640 1 --2- 172.21.15.196:0/2325913321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638162e50 0x7fc6380b63c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.115+0000 7fc63ee00640 1 --2- 172.21.15.196:0/2325913321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6380d0b90 0x7fc63815ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.115+0000 7fc63ee00640 1 --2- 172.21.15.196:0/2325913321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc638073fc0 0x7fc6380b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.115+0000 7fc63ee00640 1 -- 172.21.15.196:0/2325913321 >> 172.21.15.196:0/2325913321 conn(0x7fc6380cd830 msgr2=0x7fc638161810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:15.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.115+0000 7fc63ee00640 1 -- 172.21.15.196:0/2325913321 shutdown_connections 2024-04-28T13:33:15.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.115+0000 7fc63ee00640 1 -- 172.21.15.196:0/2325913321 wait complete. 2024-04-28T13:33:15.135 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:33:15.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.335+0000 7f9a2d600640 1 Processor -- start 2024-04-28T13:33:15.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.335+0000 7f9a2d600640 1 -- start start 2024-04-28T13:33:15.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.337+0000 7f9a2d600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9a280d1760 0x7f9a280b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:15.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.337+0000 7f9a2d600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a2806dc70 con 0x7f9a2815f2e0 2024-04-28T13:33:15.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.337+0000 7f9a2d600640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a2806df50 con 0x7f9a2815ef00 2024-04-28T13:33:15.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.337+0000 7f9a2d600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9a2806e230 con 0x7f9a280d1760 2024-04-28T13:33:15.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.338+0000 7f9a27400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f9a2815f2e0 0x7f9a280d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56992/0 (socket says 172.21.15.196:56992) 2024-04-28T13:33:15.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.338+0000 7f9a27400640 1 -- 172.21.15.196:0/2203514892 learned_addr learned my addr 172.21.15.196:0/2203514892 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:15.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.338+0000 7f9a2cc00640 1 --2- 172.21.15.196:0/2203514892 >> v2:172.21.15.203:3300/0 conn(0x7f9a280d1760 0x7f9a280b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:15.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.338+0000 7f9a26a00640 1 -- 172.21.15.196:0/2203514892 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 932867051 0 0) 0x7f9a2806dc70 con 0x7f9a2815f2e0 2024-04-28T13:33:15.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.338+0000 7f9a26a00640 1 -- 172.21.15.196:0/2203514892 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a0c003520 con 0x7f9a2815f2e0 2024-04-28T13:33:15.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.339+0000 7f9a2cc00640 1 -- 172.21.15.196:0/2203514892 >> v1:172.21.15.203:6789/0 conn(0x7f9a2815ef00 legacy=0x7f9a2806f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:15.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.339+0000 7f9a2cc00640 1 -- 172.21.15.196:0/2203514892 >> v1:172.21.15.196:6789/0 conn(0x7f9a2815f2e0 legacy=0x7f9a280d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:15.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.339+0000 7f9a2cc00640 1 -- 172.21.15.196:0/2203514892 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2806e510 con 0x7f9a280d1760 2024-04-28T13:33:15.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.340+0000 7f9a2cc00640 1 --2- 172.21.15.196:0/2203514892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a280d1760 0x7f9a280b5c10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009d90 tx=0x7f9a2002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eca81cb4ef4cb035 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:15.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.341+0000 7f9a26a00640 1 -- 172.21.15.196:0/2203514892 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a20007740 con 0x7f9a280d1760 2024-04-28T13:33:15.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.341+0000 7f9a26a00640 1 -- 172.21.15.196:0/2203514892 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a200078e0 con 0x7f9a280d1760 2024-04-28T13:33:15.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.341+0000 7f9a26a00640 1 -- 172.21.15.196:0/2203514892 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a20007cb0 con 0x7f9a280d1760 2024-04-28T13:33:15.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.341+0000 7f9a2d600640 1 -- 172.21.15.196:0/2203514892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a280d1760 msgr2=0x7f9a280b5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:15.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.341+0000 7f9a2d600640 1 --2- 172.21.15.196:0/2203514892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a280d1760 0x7f9a280b5c10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009d90 tx=0x7f9a2002a660 comp rx=0 tx=0).stop 2024-04-28T13:33:15.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.342+0000 7f9a2d600640 1 -- 172.21.15.196:0/2203514892 shutdown_connections 2024-04-28T13:33:15.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.342+0000 7f9a2d600640 1 --2- 172.21.15.196:0/2203514892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a280d1760 0x7f9a280b5c10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.342+0000 7f9a2d600640 1 -- 172.21.15.196:0/2203514892 >> 172.21.15.196:0/2203514892 conn(0x7f9a2806c6c0 msgr2=0x7f9a2806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:15.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.342+0000 7f9a2d600640 1 -- 172.21.15.196:0/2203514892 shutdown_connections 2024-04-28T13:33:15.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.342+0000 7f9a2d600640 1 -- 172.21.15.196:0/2203514892 wait complete. 2024-04-28T13:33:15.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.343+0000 7f9a2d600640 1 Processor -- start 2024-04-28T13:33:15.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.344+0000 7f9a2d600640 1 -- start start 2024-04-28T13:33:15.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.344+0000 7f9a2d600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a280d1760 0x7f9a281507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:15.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.344+0000 7f9a2d600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2815ef00 0x7f9a28160100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:15.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.345+0000 7f9a2d600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a2815f2e0 0x7f9a2814e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:15.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.345+0000 7f9a2d600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a28169df0 con 0x7f9a2815ef00 2024-04-28T13:33:15.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.345+0000 7f9a2d600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9a28073320 con 0x7f9a2815f2e0 2024-04-28T13:33:15.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.345+0000 7f9a2d600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9a28169f70 con 0x7f9a280d1760 2024-04-28T13:33:15.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.345+0000 7f9a27400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2815ef00 0x7f9a28160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:15.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.345+0000 7f9a27400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2815ef00 0x7f9a28160100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34868/0 (socket says 172.21.15.196:34868) 2024-04-28T13:33:15.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.345+0000 7f9a27400640 1 -- 172.21.15.196:0/19569395 learned_addr learned my addr 172.21.15.196:0/19569395 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:15.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.345+0000 7f9a27400640 1 -- 172.21.15.196:0/19569395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a280d1760 msgr2=0x7f9a281507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:15.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.345+0000 7f9a27400640 1 --2- 172.21.15.196:0/19569395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a280d1760 0x7f9a281507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.345+0000 7f9a27400640 1 -- 172.21.15.196:0/19569395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a2815f2e0 msgr2=0x7f9a2814e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:15.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.345+0000 7f9a27400640 1 --2- 172.21.15.196:0/19569395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a2815f2e0 0x7f9a2814e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.345+0000 7f9a27400640 1 -- 172.21.15.196:0/19569395 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a28160640 con 0x7f9a2815ef00 2024-04-28T13:33:15.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.346+0000 7f9a27400640 1 --2- 172.21.15.196:0/19569395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2815ef00 0x7f9a28160100 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400c500 tx=0x7f9a1400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:15.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.346+0000 7f9a1fe00640 1 -- 172.21.15.196:0/19569395 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14017070 con 0x7f9a2815ef00 2024-04-28T13:33:15.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.346+0000 7f9a1fe00640 1 -- 172.21.15.196:0/19569395 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a14005ce0 con 0x7f9a2815ef00 2024-04-28T13:33:15.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.346+0000 7f9a1fe00640 1 -- 172.21.15.196:0/19569395 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14007330 con 0x7f9a2815ef00 2024-04-28T13:33:15.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.346+0000 7f9a2d600640 1 -- 172.21.15.196:0/19569395 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2809be20 con 0x7f9a2815ef00 2024-04-28T13:33:15.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.346+0000 7f9a2d600640 1 -- 172.21.15.196:0/19569395 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2809c3e0 con 0x7f9a2815ef00 2024-04-28T13:33:15.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.347+0000 7f9a2d600640 1 -- 172.21.15.196:0/19569395 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a28069ae0 con 0x7f9a2815ef00 2024-04-28T13:33:15.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.348+0000 7f9a1fe00640 1 -- 172.21.15.196:0/19569395 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f9a1401a070 con 0x7f9a2815ef00 2024-04-28T13:33:15.350 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.349+0000 7f9a1fe00640 1 --2- 172.21.15.196:0/19569395 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a0806c9b0 0x7f9a0806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:15.350 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.350+0000 7f9a1fe00640 1 -- 172.21.15.196:0/19569395 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f9a1408b0b0 con 0x7f9a2815ef00 2024-04-28T13:33:15.351 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.350+0000 7f9a27e00640 1 --2- 172.21.15.196:0/19569395 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a0806c9b0 0x7f9a0806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:15.352 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.351+0000 7f9a27e00640 1 --2- 172.21.15.196:0/19569395 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a0806c9b0 0x7f9a0806ee70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9a18000f60 tx=0x7f9a180079d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:15.355 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.355+0000 7f9a1fe00640 1 -- 172.21.15.196:0/19569395 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a28069ae0 con 0x7f9a2815ef00 2024-04-28T13:33:15.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.555+0000 7f9a2d600640 1 -- 172.21.15.196:0/19569395 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a28150d40 con 0x7f9a2815ef00 2024-04-28T13:33:15.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.557+0000 7f9a1fe00640 1 -- 172.21.15.196:0/19569395 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16834 (secure 0 0 0) 0x7f9a28160640 con 0x7f9a2815ef00 2024-04-28T13:33:15.558 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:15.558 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":34,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:58.527505+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:32:29.352348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-28T13:32:30.637498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6836/459640695":"2024-04-29T13:32:58.527451+0000","172.21.15.203:6835/459640695":"2024-04-29T13:32:58.527451+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:33:15.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.560+0000 7f9a2d600640 1 -- 172.21.15.196:0/19569395 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a0806c9b0 msgr2=0x7f9a0806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:15.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.560+0000 7f9a2d600640 1 --2- 172.21.15.196:0/19569395 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a0806c9b0 0x7f9a0806ee70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9a18000f60 tx=0x7f9a180079d0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.560+0000 7f9a2d600640 1 -- 172.21.15.196:0/19569395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2815ef00 msgr2=0x7f9a28160100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:15.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.560+0000 7f9a2d600640 1 --2- 172.21.15.196:0/19569395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2815ef00 0x7f9a28160100 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400c500 tx=0x7f9a1400c9c0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.560+0000 7f9a2d600640 1 -- 172.21.15.196:0/19569395 shutdown_connections 2024-04-28T13:33:15.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.560+0000 7f9a2d600640 1 --2- 172.21.15.196:0/19569395 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9a0806c9b0 0x7f9a0806ee70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.560+0000 7f9a2d600640 1 --2- 172.21.15.196:0/19569395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a2815f2e0 0x7f9a2814e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.560+0000 7f9a2d600640 1 --2- 172.21.15.196:0/19569395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2815ef00 0x7f9a28160100 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.560+0000 7f9a2d600640 1 --2- 172.21.15.196:0/19569395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a280d1760 0x7f9a281507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.560+0000 7f9a2d600640 1 -- 172.21.15.196:0/19569395 >> 172.21.15.196:0/19569395 conn(0x7f9a2806c6c0 msgr2=0x7f9a28075390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:15.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.560+0000 7f9a2d600640 1 -- 172.21.15.196:0/19569395 shutdown_connections 2024-04-28T13:33:15.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.560+0000 7f9a2d600640 1 -- 172.21.15.196:0/19569395 wait complete. 2024-04-28T13:33:15.582 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:32:29.352348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:32:30.637498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:33:15.582 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:33:15.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.782+0000 7f8f6ec00640 1 Processor -- start 2024-04-28T13:33:15.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.782+0000 7f8f6ec00640 1 -- start start 2024-04-28T13:33:15.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.784+0000 7f8f6ec00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8f680d10a0 0x7f8f680b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:15.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.784+0000 7f8f6ec00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8f680d1480 0x7f8f68160c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:15.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.784+0000 7f8f6ec00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f6815d700 con 0x7f8f6806e8f0 2024-04-28T13:33:15.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.784+0000 7f8f6ec00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8f6815e310 con 0x7f8f680d10a0 2024-04-28T13:33:15.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.784+0000 7f8f6ec00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f8f6815d9e0 con 0x7f8f680d1480 2024-04-28T13:33:15.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.784+0000 7f8f6d800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8f680d10a0 0x7f8f680b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:15.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.784+0000 7f8f6d800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8f680d10a0 0x7f8f680b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34870/0 (socket says 172.21.15.196:34870) 2024-04-28T13:33:15.785 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.784+0000 7f8f6d800640 1 -- 172.21.15.196:0/2053791171 learned_addr learned my addr 172.21.15.196:0/2053791171 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:15.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.784+0000 7f8f6e200640 1 --2- 172.21.15.196:0/2053791171 >> v2:172.21.15.203:3300/0 conn(0x7f8f680d1480 0x7f8f68160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:15.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.784+0000 7f8f6d800640 1 -- 172.21.15.196:0/2053791171 >> v2:172.21.15.203:3300/0 conn(0x7f8f680d1480 msgr2=0x7f8f68160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:15.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.784+0000 7f8f6d800640 1 --2- 172.21.15.196:0/2053791171 >> v2:172.21.15.203:3300/0 conn(0x7f8f680d1480 0x7f8f68160c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.784+0000 7f8f6d800640 1 -- 172.21.15.196:0/2053791171 >> v1:172.21.15.203:6789/0 conn(0x7f8f6806e8f0 legacy=0x7f8f680b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:15.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.784+0000 7f8f6d800640 1 -- 172.21.15.196:0/2053791171 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6815dcc0 con 0x7f8f680d10a0 2024-04-28T13:33:15.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.785+0000 7f8f6d800640 1 --2- 172.21.15.196:0/2053791171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f680d10a0 0x7f8f680b3c90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8f6815b030 tx=0x7f8f5c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23897fc5a24966e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:15.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.786+0000 7f8f67e00640 1 -- 172.21.15.196:0/2053791171 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c037070 con 0x7f8f680d10a0 2024-04-28T13:33:15.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.786+0000 7f8f67e00640 1 -- 172.21.15.196:0/2053791171 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f5c032440 con 0x7f8f680d10a0 2024-04-28T13:33:15.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.786+0000 7f8f67e00640 1 -- 172.21.15.196:0/2053791171 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c032810 con 0x7f8f680d10a0 2024-04-28T13:33:15.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.786+0000 7f8f6ec00640 1 -- 172.21.15.196:0/2053791171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f680d10a0 msgr2=0x7f8f680b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:15.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.786+0000 7f8f6ec00640 1 --2- 172.21.15.196:0/2053791171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f680d10a0 0x7f8f680b3c90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8f6815b030 tx=0x7f8f5c02a1a0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.787+0000 7f8f6ec00640 1 -- 172.21.15.196:0/2053791171 shutdown_connections 2024-04-28T13:33:15.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.787+0000 7f8f6ec00640 1 --2- 172.21.15.196:0/2053791171 >> v2:172.21.15.203:3300/0 conn(0x7f8f680d1480 0x7f8f68160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.787+0000 7f8f6ec00640 1 --2- 172.21.15.196:0/2053791171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f680d10a0 0x7f8f680b3c90 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.787+0000 7f8f6ec00640 1 -- 172.21.15.196:0/2053791171 >> 172.21.15.196:0/2053791171 conn(0x7f8f6806c6c0 msgr2=0x7f8f6806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:15.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.787+0000 7f8f6ec00640 1 -- 172.21.15.196:0/2053791171 shutdown_connections 2024-04-28T13:33:15.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.787+0000 7f8f6ec00640 1 -- 172.21.15.196:0/2053791171 wait complete. 2024-04-28T13:33:15.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.789+0000 7f8f6ec00640 1 Processor -- start 2024-04-28T13:33:15.789 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.789+0000 7f8f6ec00640 1 -- start start 2024-04-28T13:33:15.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.790+0000 7f8f6ec00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f6806e8f0 0x7f8f6814dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.790+0000 7f8f6ec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f680d10a0 0x7f8f68155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.790+0000 7f8f6ec00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f680d1480 0x7f8f68071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.790+0000 7f8f6ec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f680d1a30 con 0x7f8f680d10a0 2024-04-28T13:33:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.790+0000 7f8f6ec00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8f680d18b0 con 0x7f8f6806e8f0 2024-04-28T13:33:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.790+0000 7f8f6ec00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8f681611d0 con 0x7f8f680d1480 2024-04-28T13:33:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.790+0000 7f8f6ce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f680d10a0 0x7f8f68155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.790+0000 7f8f6ce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f680d10a0 0x7f8f68155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34886/0 (socket says 172.21.15.196:34886) 2024-04-28T13:33:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.790+0000 7f8f6ce00640 1 -- 172.21.15.196:0/1148333428 learned_addr learned my addr 172.21.15.196:0/1148333428 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.790+0000 7f8f6d800640 1 --2- 172.21.15.196:0/1148333428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f6806e8f0 0x7f8f6814dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.790+0000 7f8f6e200640 1 --2- 172.21.15.196:0/1148333428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f680d1480 0x7f8f68071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.791+0000 7f8f6ce00640 1 -- 172.21.15.196:0/1148333428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f680d1480 msgr2=0x7f8f68071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.791+0000 7f8f6ce00640 1 --2- 172.21.15.196:0/1148333428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f680d1480 0x7f8f68071ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.791+0000 7f8f6ce00640 1 -- 172.21.15.196:0/1148333428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f6806e8f0 msgr2=0x7f8f6814dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.791+0000 7f8f6ce00640 1 --2- 172.21.15.196:0/1148333428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f6806e8f0 0x7f8f6814dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.791+0000 7f8f6ce00640 1 -- 172.21.15.196:0/1148333428 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f68072ee0 con 0x7f8f680d10a0 2024-04-28T13:33:15.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.791+0000 7f8f6e200640 1 --2- 172.21.15.196:0/1148333428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f680d1480 0x7f8f68071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:15.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.791+0000 7f8f6ce00640 1 --2- 172.21.15.196:0/1148333428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f680d10a0 0x7f8f68155ba0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400c410 tx=0x7f8f5400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:15.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.791+0000 7f8f65600640 1 -- 172.21.15.196:0/1148333428 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54017070 con 0x7f8f680d10a0 2024-04-28T13:33:15.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.791+0000 7f8f65600640 1 -- 172.21.15.196:0/1148333428 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f540035f0 con 0x7f8f680d10a0 2024-04-28T13:33:15.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.791+0000 7f8f65600640 1 -- 172.21.15.196:0/1148333428 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54003a20 con 0x7f8f680d10a0 2024-04-28T13:33:15.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.791+0000 7f8f6ec00640 1 -- 172.21.15.196:0/1148333428 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f68072920 con 0x7f8f680d10a0 2024-04-28T13:33:15.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.791+0000 7f8f6ec00640 1 -- 172.21.15.196:0/1148333428 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f680ca0b0 con 0x7f8f680d10a0 2024-04-28T13:33:15.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.793+0000 7f8f6ec00640 1 -- 172.21.15.196:0/1148333428 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f68050730 con 0x7f8f680d10a0 2024-04-28T13:33:15.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.794+0000 7f8f65600640 1 -- 172.21.15.196:0/1148333428 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8f54003bc0 con 0x7f8f680d10a0 2024-04-28T13:33:15.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.795+0000 7f8f65600640 1 --2- 172.21.15.196:0/1148333428 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8f4806c980 0x7f8f4806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:15.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.795+0000 7f8f65600640 1 -- 172.21.15.196:0/1148333428 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f8f5408aff0 con 0x7f8f680d10a0 2024-04-28T13:33:15.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.796+0000 7f8f6d800640 1 --2- 172.21.15.196:0/1148333428 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8f4806c980 0x7f8f4806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:15.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.797+0000 7f8f6d800640 1 --2- 172.21.15.196:0/1148333428 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8f4806c980 0x7f8f4806ee40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c034000 tx=0x7f8f5c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:15.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.799+0000 7f8f65600640 1 -- 172.21.15.196:0/1148333428 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f540592d0 con 0x7f8f680d10a0 2024-04-28T13:33:16.000 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:15.999+0000 7f8f6ec00640 1 -- 172.21.15.196:0/1148333428 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8f6806fc10 con 0x7f8f680d10a0 2024-04-28T13:33:16.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.000+0000 7f8f65600640 1 -- 172.21.15.196:0/1148333428 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f8f540010c0 con 0x7f8f680d10a0 2024-04-28T13:33:16.001 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:33:16.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.003+0000 7f8f6ec00640 1 -- 172.21.15.196:0/1148333428 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8f4806c980 msgr2=0x7f8f4806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:16.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.003+0000 7f8f6ec00640 1 --2- 172.21.15.196:0/1148333428 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8f4806c980 0x7f8f4806ee40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c034000 tx=0x7f8f5c02e000 comp rx=0 tx=0).stop 2024-04-28T13:33:16.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.003+0000 7f8f6ec00640 1 -- 172.21.15.196:0/1148333428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f680d10a0 msgr2=0x7f8f68155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:16.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.003+0000 7f8f6ec00640 1 --2- 172.21.15.196:0/1148333428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f680d10a0 0x7f8f68155ba0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400c410 tx=0x7f8f5400c8d0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.003+0000 7f8f6ec00640 1 -- 172.21.15.196:0/1148333428 shutdown_connections 2024-04-28T13:33:16.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.003+0000 7f8f6ec00640 1 --2- 172.21.15.196:0/1148333428 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8f4806c980 0x7f8f4806ee40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.003+0000 7f8f6ec00640 1 --2- 172.21.15.196:0/1148333428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f680d1480 0x7f8f68071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.003+0000 7f8f6ec00640 1 --2- 172.21.15.196:0/1148333428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f680d10a0 0x7f8f68155ba0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.003+0000 7f8f6ec00640 1 --2- 172.21.15.196:0/1148333428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f6806e8f0 0x7f8f6814dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.003+0000 7f8f6ec00640 1 -- 172.21.15.196:0/1148333428 >> 172.21.15.196:0/1148333428 conn(0x7f8f6806c6c0 msgr2=0x7f8f6815ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:16.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.003+0000 7f8f6ec00640 1 -- 172.21.15.196:0/1148333428 shutdown_connections 2024-04-28T13:33:16.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.003+0000 7f8f6ec00640 1 -- 172.21.15.196:0/1148333428 wait complete. 2024-04-28T13:33:16.023 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:33:16.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.224+0000 7efc9ae00640 1 Processor -- start 2024-04-28T13:33:16.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.224+0000 7efc9ae00640 1 -- start start 2024-04-28T13:33:16.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.226+0000 7efc9ae00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efc94143840 0x7efc940b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:16.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.226+0000 7efc9ae00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc940b4520 0x7efc940b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:16.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.226+0000 7efc9ae00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc9415dca0 con 0x7efc940b53f0 2024-04-28T13:33:16.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.226+0000 7efc9ae00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7efc94143c20 con 0x7efc94143840 2024-04-28T13:33:16.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.226+0000 7efc9ae00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efc9415df80 con 0x7efc940b4520 2024-04-28T13:33:16.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.226+0000 7efc93e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efc94143840 0x7efc940b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:16.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.226+0000 7efc93e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efc94143840 0x7efc940b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34888/0 (socket says 172.21.15.196:34888) 2024-04-28T13:33:16.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.226+0000 7efc93e00640 1 -- 172.21.15.196:0/1869103694 learned_addr learned my addr 172.21.15.196:0/1869103694 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:16.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.227+0000 7efc93e00640 1 -- 172.21.15.196:0/1869103694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc940b4520 msgr2=0x7efc940b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:16.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.227+0000 7efc8b400640 1 --2- 172.21.15.196:0/1869103694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc940b4520 0x7efc940b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:16.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.227+0000 7efc93e00640 1 --2- 172.21.15.196:0/1869103694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc940b4520 0x7efc940b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.227+0000 7efc93e00640 1 -- 172.21.15.196:0/1869103694 >> v1:172.21.15.203:6789/0 conn(0x7efc940b53f0 legacy=0x7efc94160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:16.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.227+0000 7efc93e00640 1 -- 172.21.15.196:0/1869103694 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9415e260 con 0x7efc94143840 2024-04-28T13:33:16.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.227+0000 7efc8b400640 1 --2- 172.21.15.196:0/1869103694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc940b4520 0x7efc940b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:16.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.228+0000 7efc93e00640 1 --2- 172.21.15.196:0/1869103694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc94143840 0x7efc940b3fe0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efc84009e50 tx=0x7efc8402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827ed4679f5cadd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:16.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.228+0000 7efc93400640 1 -- 172.21.15.196:0/1869103694 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84036070 con 0x7efc94143840 2024-04-28T13:33:16.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.228+0000 7efc93400640 1 -- 172.21.15.196:0/1869103694 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc84031600 con 0x7efc94143840 2024-04-28T13:33:16.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.228+0000 7efc93400640 1 -- 172.21.15.196:0/1869103694 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc840319d0 con 0x7efc94143840 2024-04-28T13:33:16.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.229+0000 7efc9ae00640 1 -- 172.21.15.196:0/1869103694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc94143840 msgr2=0x7efc940b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:16.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.229+0000 7efc9ae00640 1 --2- 172.21.15.196:0/1869103694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc94143840 0x7efc940b3fe0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efc84009e50 tx=0x7efc8402a6d0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.229+0000 7efc9ae00640 1 -- 172.21.15.196:0/1869103694 shutdown_connections 2024-04-28T13:33:16.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.229+0000 7efc9ae00640 1 --2- 172.21.15.196:0/1869103694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc940b4520 0x7efc940b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.229+0000 7efc9ae00640 1 --2- 172.21.15.196:0/1869103694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc94143840 0x7efc940b3fe0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.229+0000 7efc9ae00640 1 -- 172.21.15.196:0/1869103694 >> 172.21.15.196:0/1869103694 conn(0x7efc940cd8a0 msgr2=0x7efc940cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:16.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.229+0000 7efc9ae00640 1 -- 172.21.15.196:0/1869103694 shutdown_connections 2024-04-28T13:33:16.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.229+0000 7efc9ae00640 1 -- 172.21.15.196:0/1869103694 wait complete. 2024-04-28T13:33:16.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.231+0000 7efc9ae00640 1 Processor -- start 2024-04-28T13:33:16.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.231+0000 7efc9ae00640 1 -- start start 2024-04-28T13:33:16.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.232+0000 7efc9ae00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc940b4520 0x7efc941f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:16.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.232+0000 7efc9ae00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc940b53f0 0x7efc941e9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:16.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.232+0000 7efc9ae00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc94143840 0x7efc941ea500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:16.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.232+0000 7efc9ae00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efc9415da40 con 0x7efc940b53f0 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.232+0000 7efc9ae00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efc9415d8c0 con 0x7efc94143840 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.232+0000 7efc9ae00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efc940d0000 con 0x7efc940b4520 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.232+0000 7efc8b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc940b53f0 0x7efc941e9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.232+0000 7efc8b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc940b53f0 0x7efc941e9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34904/0 (socket says 172.21.15.196:34904) 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.232+0000 7efc8b400640 1 -- 172.21.15.196:0/2882615153 learned_addr learned my addr 172.21.15.196:0/2882615153 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.233+0000 7efc93e00640 1 --2- 172.21.15.196:0/2882615153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc940b4520 0x7efc941f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.233+0000 7efc98e00640 1 --2- 172.21.15.196:0/2882615153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc94143840 0x7efc941ea500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.233+0000 7efc8b400640 1 -- 172.21.15.196:0/2882615153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc940b4520 msgr2=0x7efc941f5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.233+0000 7efc8b400640 1 --2- 172.21.15.196:0/2882615153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc940b4520 0x7efc941f5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.233+0000 7efc8b400640 1 -- 172.21.15.196:0/2882615153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc94143840 msgr2=0x7efc941ea500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.233+0000 7efc8b400640 1 --2- 172.21.15.196:0/2882615153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc94143840 0x7efc941ea500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.233+0000 7efc8b400640 1 -- 172.21.15.196:0/2882615153 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc941e6cb0 con 0x7efc940b53f0 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.233+0000 7efc8b400640 1 --2- 172.21.15.196:0/2882615153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc940b53f0 0x7efc941e9fc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00c7f0 tx=0x7efc7c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.233+0000 7efc90c00640 1 -- 172.21.15.196:0/2882615153 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c015070 con 0x7efc940b53f0 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.233+0000 7efc90c00640 1 -- 172.21.15.196:0/2882615153 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc7c0079d0 con 0x7efc940b53f0 2024-04-28T13:33:16.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.233+0000 7efc90c00640 1 -- 172.21.15.196:0/2882615153 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c007da0 con 0x7efc940b53f0 2024-04-28T13:33:16.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.233+0000 7efc9ae00640 1 -- 172.21.15.196:0/2882615153 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc941e6f50 con 0x7efc940b53f0 2024-04-28T13:33:16.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.234+0000 7efc9ae00640 1 -- 172.21.15.196:0/2882615153 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc941e7510 con 0x7efc940b53f0 2024-04-28T13:33:16.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.235+0000 7efc9ae00640 1 -- 172.21.15.196:0/2882615153 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc941f6570 con 0x7efc940b53f0 2024-04-28T13:33:16.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.236+0000 7efc90c00640 1 -- 172.21.15.196:0/2882615153 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7efc7c018070 con 0x7efc940b53f0 2024-04-28T13:33:16.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.237+0000 7efc90c00640 1 --2- 172.21.15.196:0/2882615153 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efc7006c980 0x7efc7006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:16.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.237+0000 7efc90c00640 1 -- 172.21.15.196:0/2882615153 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7efc7c08bc40 con 0x7efc940b53f0 2024-04-28T13:33:16.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.237+0000 7efc93e00640 1 --2- 172.21.15.196:0/2882615153 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efc7006c980 0x7efc7006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:16.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.238+0000 7efc93e00640 1 --2- 172.21.15.196:0/2882615153 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efc7006c980 0x7efc7006ee40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efc84033000 tx=0x7efc84009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:16.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.242+0000 7efc90c00640 1 -- 172.21.15.196:0/2882615153 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efc7c059f10 con 0x7efc940b53f0 2024-04-28T13:33:16.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.439+0000 7efc9ae00640 1 -- 172.21.15.196:0/2882615153 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efc9406a290 con 0x7efc940b53f0 2024-04-28T13:33:16.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.440+0000 7efc90c00640 1 -- 172.21.15.196:0/2882615153 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7efc7c059810 con 0x7efc940b53f0 2024-04-28T13:33:16.441 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:33:16.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.443+0000 7efc9ae00640 1 -- 172.21.15.196:0/2882615153 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efc7006c980 msgr2=0x7efc7006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:16.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.443+0000 7efc9ae00640 1 --2- 172.21.15.196:0/2882615153 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efc7006c980 0x7efc7006ee40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efc84033000 tx=0x7efc84009000 comp rx=0 tx=0).stop 2024-04-28T13:33:16.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.443+0000 7efc9ae00640 1 -- 172.21.15.196:0/2882615153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc940b53f0 msgr2=0x7efc941e9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:16.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.443+0000 7efc9ae00640 1 --2- 172.21.15.196:0/2882615153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc940b53f0 0x7efc941e9fc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00c7f0 tx=0x7efc7c00cbf0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.443+0000 7efc9ae00640 1 -- 172.21.15.196:0/2882615153 shutdown_connections 2024-04-28T13:33:16.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.443+0000 7efc9ae00640 1 --2- 172.21.15.196:0/2882615153 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efc7006c980 0x7efc7006ee40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.446 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.443+0000 7efc9ae00640 1 --2- 172.21.15.196:0/2882615153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc94143840 0x7efc941ea500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.443+0000 7efc9ae00640 1 --2- 172.21.15.196:0/2882615153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc940b53f0 0x7efc941e9fc0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.443+0000 7efc9ae00640 1 --2- 172.21.15.196:0/2882615153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc940b4520 0x7efc941f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.443+0000 7efc9ae00640 1 -- 172.21.15.196:0/2882615153 >> 172.21.15.196:0/2882615153 conn(0x7efc940cd8a0 msgr2=0x7efc9406f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:16.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.443+0000 7efc9ae00640 1 -- 172.21.15.196:0/2882615153 shutdown_connections 2024-04-28T13:33:16.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.443+0000 7efc9ae00640 1 -- 172.21.15.196:0/2882615153 wait complete. 2024-04-28T13:33:16.461 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:33:16.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.659+0000 7f5f01000640 1 Processor -- start 2024-04-28T13:33:16.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.660+0000 7f5f01000640 1 -- start start 2024-04-28T13:33:16.663 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.661+0000 7f5f01000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5efc06f340 0x7f5efc0b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:16.663 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.661+0000 7f5f01000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5efc0b3e50 0x7f5efc0720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:16.663 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.661+0000 7f5f01000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5efc0b46a0 con 0x7f5efc0d0300 2024-04-28T13:33:16.663 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.661+0000 7f5f01000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5efc0b5db0 con 0x7f5efc06f340 2024-04-28T13:33:16.663 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.661+0000 7f5f01000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5efc0b6090 con 0x7f5efc0b3e50 2024-04-28T13:33:16.663 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.662+0000 7f5ef9e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5efc06f340 0x7f5efc0b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:16.663 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.662+0000 7f5ef9e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5efc06f340 0x7f5efc0b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34912/0 (socket says 172.21.15.196:34912) 2024-04-28T13:33:16.663 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.662+0000 7f5ef9400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f5efc0d0300 0x7f5efc06f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:56998/0 (socket says 172.21.15.196:56998) 2024-04-28T13:33:16.664 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.662+0000 7f5ef9e00640 1 -- 172.21.15.196:0/722966617 learned_addr learned my addr 172.21.15.196:0/722966617 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:16.664 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.662+0000 7f5efa800640 1 --2- 172.21.15.196:0/722966617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5efc0b3e50 0x7f5efc0720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:16.664 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.662+0000 7f5ef9e00640 1 -- 172.21.15.196:0/722966617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5efc0b3e50 msgr2=0x7f5efc0720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:16.664 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.662+0000 7f5ef9e00640 1 --2- 172.21.15.196:0/722966617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5efc0b3e50 0x7f5efc0720f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.664 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.662+0000 7f5ef9e00640 1 -- 172.21.15.196:0/722966617 >> v1:172.21.15.196:6789/0 conn(0x7f5efc0d0300 legacy=0x7f5efc06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:16.664 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.662+0000 7f5ef9e00640 1 -- 172.21.15.196:0/722966617 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5efc0b6370 con 0x7f5efc06f340 2024-04-28T13:33:16.664 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.662+0000 7f5ef8a00640 1 -- 172.21.15.196:0/722966617 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1417798005 0 0) 0x7f5efc0b46a0 con 0x7f5efc0d0300 2024-04-28T13:33:16.664 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.663+0000 7f5ef9e00640 1 --2- 172.21.15.196:0/722966617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5efc06f340 0x7f5efc0b6b70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009870 tx=0x7f5eec02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4319129eb9c7c8bd server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:16.665 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.663+0000 7f5ef8a00640 1 -- 172.21.15.196:0/722966617 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec037070 con 0x7f5efc06f340 2024-04-28T13:33:16.665 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.664+0000 7f5ef8a00640 1 -- 172.21.15.196:0/722966617 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eec032440 con 0x7f5efc06f340 2024-04-28T13:33:16.665 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.664+0000 7f5ef8a00640 1 -- 172.21.15.196:0/722966617 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec032810 con 0x7f5efc06f340 2024-04-28T13:33:16.666 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.664+0000 7f5f01000640 1 -- 172.21.15.196:0/722966617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5efc06f340 msgr2=0x7f5efc0b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:16.666 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.664+0000 7f5f01000640 1 --2- 172.21.15.196:0/722966617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5efc06f340 0x7f5efc0b6b70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009870 tx=0x7f5eec02a130 comp rx=0 tx=0).stop 2024-04-28T13:33:16.666 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.664+0000 7f5f01000640 1 -- 172.21.15.196:0/722966617 shutdown_connections 2024-04-28T13:33:16.666 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.664+0000 7f5f01000640 1 --2- 172.21.15.196:0/722966617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5efc0b3e50 0x7f5efc0720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.666 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.664+0000 7f5f01000640 1 --2- 172.21.15.196:0/722966617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5efc06f340 0x7f5efc0b6b70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.666 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.664+0000 7f5f01000640 1 -- 172.21.15.196:0/722966617 >> 172.21.15.196:0/722966617 conn(0x7f5efc0cd8a0 msgr2=0x7f5efc0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:16.666 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.664+0000 7f5f01000640 1 -- 172.21.15.196:0/722966617 shutdown_connections 2024-04-28T13:33:16.666 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.664+0000 7f5f01000640 1 -- 172.21.15.196:0/722966617 wait complete. 2024-04-28T13:33:16.667 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.666+0000 7f5f01000640 1 Processor -- start 2024-04-28T13:33:16.667 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.666+0000 7f5f01000640 1 -- start start 2024-04-28T13:33:16.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.667+0000 7f5f01000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5efc06f340 0x7f5efc0cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:16.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.667+0000 7f5f01000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5efc0b3e50 0x7f5efc0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:16.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.667+0000 7f5f01000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5efc0d0300 0x7f5efc0cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:16.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.667+0000 7f5f01000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc0b4460 con 0x7f5efc06f340 2024-04-28T13:33:16.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.667+0000 7f5f01000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc0b42e0 con 0x7f5efc0b3e50 2024-04-28T13:33:16.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.667+0000 7f5f01000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5efc142a30 con 0x7f5efc0d0300 2024-04-28T13:33:16.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.667+0000 7f5ef9e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5efc06f340 0x7f5efc0cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:16.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.667+0000 7f5ef9e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5efc06f340 0x7f5efc0cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34914/0 (socket says 172.21.15.196:34914) 2024-04-28T13:33:16.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.667+0000 7f5ef9e00640 1 -- 172.21.15.196:0/3761680543 learned_addr learned my addr 172.21.15.196:0/3761680543 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:16.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.668+0000 7f5ef9e00640 1 -- 172.21.15.196:0/3761680543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5efc0d0300 msgr2=0x7f5efc0cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:16.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.668+0000 7f5efa800640 1 --2- 172.21.15.196:0/3761680543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5efc0d0300 0x7f5efc0cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:16.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.668+0000 7f5ef9400640 1 --2- 172.21.15.196:0/3761680543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5efc0b3e50 0x7f5efc0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:16.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.668+0000 7f5ef9e00640 1 --2- 172.21.15.196:0/3761680543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5efc0d0300 0x7f5efc0cd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.668+0000 7f5ef9e00640 1 -- 172.21.15.196:0/3761680543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5efc0b3e50 msgr2=0x7f5efc0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:16.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.668+0000 7f5ef9e00640 1 --2- 172.21.15.196:0/3761680543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5efc0b3e50 0x7f5efc0ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.668+0000 7f5ef9e00640 1 -- 172.21.15.196:0/3761680543 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5efc0cb760 con 0x7f5efc06f340 2024-04-28T13:33:16.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.668+0000 7f5ef9e00640 1 --2- 172.21.15.196:0/3761680543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5efc06f340 0x7f5efc0cb220 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5eec035040 tx=0x7f5eec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:16.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.668+0000 7f5ef2000640 1 -- 172.21.15.196:0/3761680543 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec037040 con 0x7f5efc06f340 2024-04-28T13:33:16.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.668+0000 7f5ef2000640 1 -- 172.21.15.196:0/3761680543 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eec004230 con 0x7f5efc06f340 2024-04-28T13:33:16.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.669+0000 7f5ef2000640 1 -- 172.21.15.196:0/3761680543 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec003610 con 0x7f5efc06f340 2024-04-28T13:33:16.671 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.669+0000 7f5f01000640 1 -- 172.21.15.196:0/3761680543 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5efc0c6e00 con 0x7f5efc06f340 2024-04-28T13:33:16.671 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.669+0000 7f5f01000640 1 -- 172.21.15.196:0/3761680543 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5efc0c73c0 con 0x7f5efc06f340 2024-04-28T13:33:16.671 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.670+0000 7f5f01000640 1 -- 172.21.15.196:0/3761680543 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5efc06a290 con 0x7f5efc06f340 2024-04-28T13:33:16.671 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.671+0000 7f5ef2000640 1 -- 172.21.15.196:0/3761680543 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f5eec0037b0 con 0x7f5efc06f340 2024-04-28T13:33:16.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.671+0000 7f5ef2000640 1 --2- 172.21.15.196:0/3761680543 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5ed806c980 0x7f5ed806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:16.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.672+0000 7f5ef2000640 1 -- 172.21.15.196:0/3761680543 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f5eec0acc50 con 0x7f5efc06f340 2024-04-28T13:33:16.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.672+0000 7f5ef9400640 1 --2- 172.21.15.196:0/3761680543 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5ed806c980 0x7f5ed806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:16.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.673+0000 7f5ef9400640 1 --2- 172.21.15.196:0/3761680543 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5ed806c980 0x7f5ed806ee40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4007fd0 tx=0x7f5ee4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:16.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.676+0000 7f5ef2000640 1 -- 172.21.15.196:0/3761680543 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5eec07afa0 con 0x7f5efc06f340 2024-04-28T13:33:16.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.879+0000 7f5f01000640 1 -- 172.21.15.196:0/3761680543 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5efc0b46a0 con 0x7f5efc06f340 2024-04-28T13:33:16.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.880+0000 7f5ef2000640 1 -- 172.21.15.196:0/3761680543 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f5eec07a8a0 con 0x7f5efc06f340 2024-04-28T13:33:16.881 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:33:16.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.883+0000 7f5f01000640 1 -- 172.21.15.196:0/3761680543 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5ed806c980 msgr2=0x7f5ed806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:16.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.883+0000 7f5f01000640 1 --2- 172.21.15.196:0/3761680543 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5ed806c980 0x7f5ed806ee40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4007fd0 tx=0x7f5ee4009040 comp rx=0 tx=0).stop 2024-04-28T13:33:16.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.883+0000 7f5f01000640 1 -- 172.21.15.196:0/3761680543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5efc06f340 msgr2=0x7f5efc0cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:16.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.883+0000 7f5f01000640 1 --2- 172.21.15.196:0/3761680543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5efc06f340 0x7f5efc0cb220 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5eec035040 tx=0x7f5eec0046c0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.883+0000 7f5f01000640 1 -- 172.21.15.196:0/3761680543 shutdown_connections 2024-04-28T13:33:16.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.883+0000 7f5f01000640 1 --2- 172.21.15.196:0/3761680543 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5ed806c980 0x7f5ed806ee40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.883+0000 7f5f01000640 1 --2- 172.21.15.196:0/3761680543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5efc0d0300 0x7f5efc0cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.883+0000 7f5f01000640 1 --2- 172.21.15.196:0/3761680543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5efc0b3e50 0x7f5efc0ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.883+0000 7f5f01000640 1 --2- 172.21.15.196:0/3761680543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5efc06f340 0x7f5efc0cb220 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:16.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.883+0000 7f5f01000640 1 -- 172.21.15.196:0/3761680543 >> 172.21.15.196:0/3761680543 conn(0x7f5efc0cd8a0 msgr2=0x7f5efc143490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:16.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.883+0000 7f5f01000640 1 -- 172.21.15.196:0/3761680543 shutdown_connections 2024-04-28T13:33:16.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:16.883+0000 7f5f01000640 1 -- 172.21.15.196:0/3761680543 wait complete. 2024-04-28T13:33:16.903 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:33:17.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.100+0000 7f1e93c00640 1 Processor -- start 2024-04-28T13:33:17.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.100+0000 7f1e93c00640 1 -- start start 2024-04-28T13:33:17.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.102+0000 7f1e93c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1e8c06fb50 0x7f1e8c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:17.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.102+0000 7f1e93c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e8c073250 0x7f1e8c15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:17.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.102+0000 7f1e93c00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e8c073a40 con 0x7f1e8c145870 2024-04-28T13:33:17.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.102+0000 7f1e93c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1e8c15d8d0 con 0x7f1e8c06fb50 2024-04-28T13:33:17.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.102+0000 7f1e93c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1e8c15dbb0 con 0x7f1e8c073250 2024-04-28T13:33:17.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.102+0000 7f1e8be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1e8c06fb50 0x7f1e8c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:17.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.102+0000 7f1e8be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1e8c06fb50 0x7f1e8c072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34926/0 (socket says 172.21.15.196:34926) 2024-04-28T13:33:17.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.102+0000 7f1e8be00640 1 -- 172.21.15.196:0/3912271879 learned_addr learned my addr 172.21.15.196:0/3912271879 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:17.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.102+0000 7f1e8be00640 1 -- 172.21.15.196:0/3912271879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e8c073250 msgr2=0x7f1e8c15e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:17.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.102+0000 7f1e91c00640 1 --2- 172.21.15.196:0/3912271879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e8c073250 0x7f1e8c15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:17.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.103+0000 7f1e8be00640 1 --2- 172.21.15.196:0/3912271879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e8c073250 0x7f1e8c15e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.103+0000 7f1e8be00640 1 -- 172.21.15.196:0/3912271879 >> v1:172.21.15.203:6789/0 conn(0x7f1e8c145870 legacy=0x7f1e8c06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:17.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.103+0000 7f1e8be00640 1 -- 172.21.15.196:0/3912271879 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c15de90 con 0x7f1e8c06fb50 2024-04-28T13:33:17.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.103+0000 7f1e91c00640 1 --2- 172.21.15.196:0/3912271879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e8c073250 0x7f1e8c15e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:17.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.104+0000 7f1e8be00640 1 --2- 172.21.15.196:0/3912271879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e8c06fb50 0x7f1e8c072d10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c070530 tx=0x7f1e7402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb2d931b0d87214 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:17.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.104+0000 7f1e8b400640 1 -- 172.21.15.196:0/3912271879 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74032750 con 0x7f1e8c06fb50 2024-04-28T13:33:17.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.104+0000 7f1e8b400640 1 -- 172.21.15.196:0/3912271879 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e74030050 con 0x7f1e8c06fb50 2024-04-28T13:33:17.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.104+0000 7f1e8b400640 1 -- 172.21.15.196:0/3912271879 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74032b10 con 0x7f1e8c06fb50 2024-04-28T13:33:17.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.104+0000 7f1e93c00640 1 -- 172.21.15.196:0/3912271879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e8c06fb50 msgr2=0x7f1e8c072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:17.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.104+0000 7f1e93c00640 1 --2- 172.21.15.196:0/3912271879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e8c06fb50 0x7f1e8c072d10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c070530 tx=0x7f1e7402c890 comp rx=0 tx=0).stop 2024-04-28T13:33:17.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.105+0000 7f1e93c00640 1 -- 172.21.15.196:0/3912271879 shutdown_connections 2024-04-28T13:33:17.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.105+0000 7f1e93c00640 1 --2- 172.21.15.196:0/3912271879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e8c073250 0x7f1e8c15e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.105+0000 7f1e93c00640 1 --2- 172.21.15.196:0/3912271879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e8c06fb50 0x7f1e8c072d10 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.105+0000 7f1e93c00640 1 -- 172.21.15.196:0/3912271879 >> 172.21.15.196:0/3912271879 conn(0x7f1e8c0cd830 msgr2=0x7f1e8c0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:17.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.105+0000 7f1e93c00640 1 -- 172.21.15.196:0/3912271879 shutdown_connections 2024-04-28T13:33:17.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.105+0000 7f1e93c00640 1 -- 172.21.15.196:0/3912271879 wait complete. 2024-04-28T13:33:17.107 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.106+0000 7f1e93c00640 1 Processor -- start 2024-04-28T13:33:17.107 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.106+0000 7f1e93c00640 1 -- start start 2024-04-28T13:33:17.109 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.107+0000 7f1e93c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e8c06fb50 0x7f1e8c1f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:17.109 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.107+0000 7f1e93c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e8c073250 0x7f1e8c140490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:17.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.107+0000 7f1e93c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e8c145870 0x7f1e8c1409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:17.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.107+0000 7f1e93c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c0736c0 con 0x7f1e8c145870 2024-04-28T13:33:17.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.107+0000 7f1e93c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c145c50 con 0x7f1e8c06fb50 2024-04-28T13:33:17.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.107+0000 7f1e93c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1e8c15e8f0 con 0x7f1e8c073250 2024-04-28T13:33:17.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.108+0000 7f1e91c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e8c145870 0x7f1e8c1409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:17.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.108+0000 7f1e91c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e8c145870 0x7f1e8c1409d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34934/0 (socket says 172.21.15.196:34934) 2024-04-28T13:33:17.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.108+0000 7f1e91c00640 1 -- 172.21.15.196:0/670623108 learned_addr learned my addr 172.21.15.196:0/670623108 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:17.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.108+0000 7f1e91200640 1 --2- 172.21.15.196:0/670623108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e8c06fb50 0x7f1e8c1f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:17.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.108+0000 7f1e91c00640 1 -- 172.21.15.196:0/670623108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e8c073250 msgr2=0x7f1e8c140490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:17.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.108+0000 7f1e91c00640 1 --2- 172.21.15.196:0/670623108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e8c073250 0x7f1e8c140490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.108+0000 7f1e91c00640 1 -- 172.21.15.196:0/670623108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e8c06fb50 msgr2=0x7f1e8c1f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:17.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.108+0000 7f1e91c00640 1 --2- 172.21.15.196:0/670623108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e8c06fb50 0x7f1e8c1f3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.108+0000 7f1e91c00640 1 -- 172.21.15.196:0/670623108 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c0b1630 con 0x7f1e8c145870 2024-04-28T13:33:17.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.108+0000 7f1e91200640 1 --2- 172.21.15.196:0/670623108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e8c06fb50 0x7f1e8c1f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:17.111 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.108+0000 7f1e91c00640 1 --2- 172.21.15.196:0/670623108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e8c145870 0x7f1e8c1409d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400ce40 tx=0x7f1e8400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:17.111 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.109+0000 7f1e88c00640 1 -- 172.21.15.196:0/670623108 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8401a070 con 0x7f1e8c145870 2024-04-28T13:33:17.111 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.109+0000 7f1e88c00640 1 -- 172.21.15.196:0/670623108 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8400d840 con 0x7f1e8c145870 2024-04-28T13:33:17.111 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.109+0000 7f1e93c00640 1 -- 172.21.15.196:0/670623108 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8c0b1910 con 0x7f1e8c145870 2024-04-28T13:33:17.111 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.109+0000 7f1e88c00640 1 -- 172.21.15.196:0/670623108 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8400dc30 con 0x7f1e8c145870 2024-04-28T13:33:17.111 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.109+0000 7f1e93c00640 1 -- 172.21.15.196:0/670623108 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8c0b1ed0 con 0x7f1e8c145870 2024-04-28T13:33:17.111 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.110+0000 7f1e93c00640 1 -- 172.21.15.196:0/670623108 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e8c06a290 con 0x7f1e8c145870 2024-04-28T13:33:17.111 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.111+0000 7f1e88c00640 1 -- 172.21.15.196:0/670623108 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f1e8401d070 con 0x7f1e8c145870 2024-04-28T13:33:17.112 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.112+0000 7f1e88c00640 1 --2- 172.21.15.196:0/670623108 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1e6806c980 0x7f1e6806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:17.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.112+0000 7f1e88c00640 1 -- 172.21.15.196:0/670623108 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f1e8408c410 con 0x7f1e8c145870 2024-04-28T13:33:17.113 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.113+0000 7f1e91200640 1 --2- 172.21.15.196:0/670623108 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1e6806c980 0x7f1e6806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:17.114 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.114+0000 7f1e91200640 1 --2- 172.21.15.196:0/670623108 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1e6806c980 0x7f1e6806ee40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0b0100 tx=0x7f1e7c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:17.118 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.117+0000 7f1e88c00640 1 -- 172.21.15.196:0/670623108 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e8405a7f0 con 0x7f1e8c145870 2024-04-28T13:33:17.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.358+0000 7f1e93c00640 1 -- 172.21.15.196:0/670623108 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e8c0696f0 con 0x7f1e8c145870 2024-04-28T13:33:17.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.359+0000 7f1e88c00640 1 -- 172.21.15.196:0/670623108 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5191 (secure 0 0 0) 0x7f1e8405a0f0 con 0x7f1e8c145870 2024-04-28T13:33:17.361 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:17.361 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/931076983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":931076983},{"type":"v1","addr":"172.21.15.196:6834","nonce":931076983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1114474485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1114474485},{"type":"v1","addr":"172.21.15.196:6835","nonce":1114474485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2070123371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2070123371},{"type":"v1","addr":"172.21.15.203:6837","nonce":2070123371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:32:31.669472+0000","modified":"2024-04-28T13:33:02.550473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4694},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4694":{"gid":4694,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6836/1998460400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1998460400},{"type":"v1","addr":"172.21.15.203:6836","nonce":1998460400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4694,"qdb_cluster":[4694]},"id":2}]} 2024-04-28T13:33:17.361 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 22 2024-04-28T13:33:17.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.363+0000 7f1e93c00640 1 -- 172.21.15.196:0/670623108 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1e6806c980 msgr2=0x7f1e6806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:17.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.363+0000 7f1e93c00640 1 --2- 172.21.15.196:0/670623108 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1e6806c980 0x7f1e6806ee40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0b0100 tx=0x7f1e7c008040 comp rx=0 tx=0).stop 2024-04-28T13:33:17.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.363+0000 7f1e93c00640 1 -- 172.21.15.196:0/670623108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e8c145870 msgr2=0x7f1e8c1409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:17.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.363+0000 7f1e93c00640 1 --2- 172.21.15.196:0/670623108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e8c145870 0x7f1e8c1409d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400ce40 tx=0x7f1e8400a710 comp rx=0 tx=0).stop 2024-04-28T13:33:17.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.363+0000 7f1e93c00640 1 -- 172.21.15.196:0/670623108 shutdown_connections 2024-04-28T13:33:17.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.363+0000 7f1e93c00640 1 --2- 172.21.15.196:0/670623108 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1e6806c980 0x7f1e6806ee40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.363+0000 7f1e93c00640 1 --2- 172.21.15.196:0/670623108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e8c145870 0x7f1e8c1409d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.363+0000 7f1e93c00640 1 --2- 172.21.15.196:0/670623108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e8c073250 0x7f1e8c140490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.363+0000 7f1e93c00640 1 --2- 172.21.15.196:0/670623108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e8c06fb50 0x7f1e8c1f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.363+0000 7f1e93c00640 1 -- 172.21.15.196:0/670623108 >> 172.21.15.196:0/670623108 conn(0x7f1e8c0cd830 msgr2=0x7f1e8c0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:17.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.363+0000 7f1e93c00640 1 -- 172.21.15.196:0/670623108 shutdown_connections 2024-04-28T13:33:17.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.363+0000 7f1e93c00640 1 -- 172.21.15.196:0/670623108 wait complete. 2024-04-28T13:33:17.382 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:33:17.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.582+0000 7f1c83600640 1 Processor -- start 2024-04-28T13:33:17.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.582+0000 7f1c83600640 1 -- start start 2024-04-28T13:33:17.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.584+0000 7f1c83600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c7c073fc0 0x7f1c7c072020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:17.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.584+0000 7f1c83600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1c7c0d12a0 0x7f1c7c163290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:17.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.584+0000 7f1c83600640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c7c072560 con 0x7f1c7c162e50 2024-04-28T13:33:17.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.584+0000 7f1c83600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1c7c163910 con 0x7f1c7c0d12a0 2024-04-28T13:33:17.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.584+0000 7f1c83600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1c7c163bf0 con 0x7f1c7c073fc0 2024-04-28T13:33:17.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.584+0000 7f1c81600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1c7c0d12a0 0x7f1c7c163290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:17.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.584+0000 7f1c81600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1c7c0d12a0 0x7f1c7c163290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34950/0 (socket says 172.21.15.196:34950) 2024-04-28T13:33:17.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.584+0000 7f1c81600640 1 -- 172.21.15.196:0/110167474 learned_addr learned my addr 172.21.15.196:0/110167474 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:17.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.584+0000 7f1c80c00640 1 --2- 172.21.15.196:0/110167474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c7c073fc0 0x7f1c7c072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:17.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.585+0000 7f1c81600640 1 -- 172.21.15.196:0/110167474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c7c073fc0 msgr2=0x7f1c7c072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:17.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.585+0000 7f1c81600640 1 --2- 172.21.15.196:0/110167474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c7c073fc0 0x7f1c7c072020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.585+0000 7f1c81600640 1 -- 172.21.15.196:0/110167474 >> v1:172.21.15.203:6789/0 conn(0x7f1c7c162e50 legacy=0x7f1c7c0d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:17.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.585+0000 7f1c81600640 1 -- 172.21.15.196:0/110167474 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7c06ea50 con 0x7f1c7c0d12a0 2024-04-28T13:33:17.587 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.586+0000 7f1c81600640 1 --2- 172.21.15.196:0/110167474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c7c0d12a0 0x7f1c7c163290 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009870 tx=0x7f1c7402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602d706a8daaa92f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:17.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.586+0000 7f1c7b400640 1 -- 172.21.15.196:0/110167474 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74037070 con 0x7f1c7c0d12a0 2024-04-28T13:33:17.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.586+0000 7f1c7b400640 1 -- 172.21.15.196:0/110167474 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c74032440 con 0x7f1c7c0d12a0 2024-04-28T13:33:17.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.586+0000 7f1c7b400640 1 -- 172.21.15.196:0/110167474 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74032810 con 0x7f1c7c0d12a0 2024-04-28T13:33:17.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.587+0000 7f1c83600640 1 -- 172.21.15.196:0/110167474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c7c0d12a0 msgr2=0x7f1c7c163290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:17.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.587+0000 7f1c83600640 1 --2- 172.21.15.196:0/110167474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c7c0d12a0 0x7f1c7c163290 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009870 tx=0x7f1c7402a2a0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.587+0000 7f1c83600640 1 -- 172.21.15.196:0/110167474 shutdown_connections 2024-04-28T13:33:17.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.587+0000 7f1c83600640 1 --2- 172.21.15.196:0/110167474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c7c0d12a0 0x7f1c7c163290 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.587+0000 7f1c83600640 1 --2- 172.21.15.196:0/110167474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c7c073fc0 0x7f1c7c072020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.587+0000 7f1c83600640 1 -- 172.21.15.196:0/110167474 >> 172.21.15.196:0/110167474 conn(0x7f1c7c0cd830 msgr2=0x7f1c7c0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:17.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.587+0000 7f1c83600640 1 -- 172.21.15.196:0/110167474 shutdown_connections 2024-04-28T13:33:17.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.587+0000 7f1c83600640 1 -- 172.21.15.196:0/110167474 wait complete. 2024-04-28T13:33:17.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.588+0000 7f1c83600640 1 Processor -- start 2024-04-28T13:33:17.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.589+0000 7f1c83600640 1 -- start start 2024-04-28T13:33:17.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.589+0000 7f1c83600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c7c073fc0 0x7f1c7c0b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:17.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.590+0000 7f1c83600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c7c0d12a0 0x7f1c7c1f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:17.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.590+0000 7f1c83600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c7c162e50 0x7f1c7c0b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:17.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.590+0000 7f1c83600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c0743a0 con 0x7f1c7c073fc0 2024-04-28T13:33:17.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.590+0000 7f1c83600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c148de0 con 0x7f1c7c162e50 2024-04-28T13:33:17.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.590+0000 7f1c83600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1c7c0b1860 con 0x7f1c7c0d12a0 2024-04-28T13:33:17.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.590+0000 7f1c80c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c7c073fc0 0x7f1c7c0b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:17.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.590+0000 7f1c80c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c7c073fc0 0x7f1c7c0b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34952/0 (socket says 172.21.15.196:34952) 2024-04-28T13:33:17.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.590+0000 7f1c80c00640 1 -- 172.21.15.196:0/4003583310 learned_addr learned my addr 172.21.15.196:0/4003583310 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:17.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.590+0000 7f1c80c00640 1 -- 172.21.15.196:0/4003583310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c7c0d12a0 msgr2=0x7f1c7c1f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:17.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.590+0000 7f1c80c00640 1 --2- 172.21.15.196:0/4003583310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c7c0d12a0 0x7f1c7c1f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.590+0000 7f1c80c00640 1 -- 172.21.15.196:0/4003583310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c7c162e50 msgr2=0x7f1c7c0b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:17.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.590+0000 7f1c80c00640 1 --2- 172.21.15.196:0/4003583310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c7c162e50 0x7f1c7c0b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.590+0000 7f1c80c00640 1 -- 172.21.15.196:0/4003583310 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7c0b1610 con 0x7f1c7c073fc0 2024-04-28T13:33:17.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.591+0000 7f1c80c00640 1 --2- 172.21.15.196:0/4003583310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c7c073fc0 0x7f1c7c0b10d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00d800 tx=0x7f1c6c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:17.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.591+0000 7f1c78c00640 1 -- 172.21.15.196:0/4003583310 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00b6c0 con 0x7f1c7c073fc0 2024-04-28T13:33:17.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.591+0000 7f1c78c00640 1 -- 172.21.15.196:0/4003583310 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c6c019070 con 0x7f1c7c073fc0 2024-04-28T13:33:17.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.591+0000 7f1c83600640 1 -- 172.21.15.196:0/4003583310 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7c1e99f0 con 0x7f1c7c073fc0 2024-04-28T13:33:17.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.591+0000 7f1c78c00640 1 -- 172.21.15.196:0/4003583310 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00ba80 con 0x7f1c7c073fc0 2024-04-28T13:33:17.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.591+0000 7f1c83600640 1 -- 172.21.15.196:0/4003583310 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7c0b1ab0 con 0x7f1c7c073fc0 2024-04-28T13:33:17.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.592+0000 7f1c83600640 1 -- 172.21.15.196:0/4003583310 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c7c06a290 con 0x7f1c7c073fc0 2024-04-28T13:33:17.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.593+0000 7f1c78c00640 1 -- 172.21.15.196:0/4003583310 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f1c6c00bc20 con 0x7f1c7c073fc0 2024-04-28T13:33:17.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.594+0000 7f1c78c00640 1 --2- 172.21.15.196:0/4003583310 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1c5806c980 0x7f1c5806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:17.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.594+0000 7f1c78c00640 1 -- 172.21.15.196:0/4003583310 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f1c6c08c1b0 con 0x7f1c7c073fc0 2024-04-28T13:33:17.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.595+0000 7f1c7be00640 1 --2- 172.21.15.196:0/4003583310 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1c5806c980 0x7f1c5806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:17.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.596+0000 7f1c7be00640 1 --2- 172.21.15.196:0/4003583310 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1c5806c980 0x7f1c5806ee40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0b38d0 tx=0x7f1c64009230 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:17.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.599+0000 7f1c78c00640 1 -- 172.21.15.196:0/4003583310 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1c6c01c360 con 0x7f1c7c073fc0 2024-04-28T13:33:17.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.804+0000 7f1c83600640 1 -- 172.21.15.196:0/4003583310 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c7c0cd610 con 0x7f1c7c073fc0 2024-04-28T13:33:17.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.806+0000 7f1c78c00640 1 -- 172.21.15.196:0/4003583310 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16834 (secure 0 0 0) 0x7f1c6c05a490 con 0x7f1c7c073fc0 2024-04-28T13:33:17.807 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:17.807 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":34,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:58.527505+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:32:29.352348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-28T13:32:30.637498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6836/459640695":"2024-04-29T13:32:58.527451+0000","172.21.15.203:6835/459640695":"2024-04-29T13:32:58.527451+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:33:17.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.809+0000 7f1c83600640 1 -- 172.21.15.196:0/4003583310 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1c5806c980 msgr2=0x7f1c5806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:17.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.809+0000 7f1c83600640 1 --2- 172.21.15.196:0/4003583310 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1c5806c980 0x7f1c5806ee40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0b38d0 tx=0x7f1c64009230 comp rx=0 tx=0).stop 2024-04-28T13:33:17.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.809+0000 7f1c83600640 1 -- 172.21.15.196:0/4003583310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c7c073fc0 msgr2=0x7f1c7c0b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:17.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.809+0000 7f1c83600640 1 --2- 172.21.15.196:0/4003583310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c7c073fc0 0x7f1c7c0b10d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00d800 tx=0x7f1c6c00dcc0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.809+0000 7f1c83600640 1 -- 172.21.15.196:0/4003583310 shutdown_connections 2024-04-28T13:33:17.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.809+0000 7f1c83600640 1 --2- 172.21.15.196:0/4003583310 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1c5806c980 0x7f1c5806ee40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.810 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.809+0000 7f1c83600640 1 --2- 172.21.15.196:0/4003583310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c7c162e50 0x7f1c7c0b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.809+0000 7f1c83600640 1 --2- 172.21.15.196:0/4003583310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c7c0d12a0 0x7f1c7c1f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.809+0000 7f1c83600640 1 --2- 172.21.15.196:0/4003583310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c7c073fc0 0x7f1c7c0b10d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:17.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.809+0000 7f1c83600640 1 -- 172.21.15.196:0/4003583310 >> 172.21.15.196:0/4003583310 conn(0x7f1c7c0cd830 msgr2=0x7f1c7c15f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:17.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.809+0000 7f1c83600640 1 -- 172.21.15.196:0/4003583310 shutdown_connections 2024-04-28T13:33:17.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:17.809+0000 7f1c83600640 1 -- 172.21.15.196:0/4003583310 wait complete. 2024-04-28T13:33:17.830 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-28T13:33:17.830 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:33:18.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.025+0000 7f4606000640 1 Processor -- start 2024-04-28T13:33:18.026 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.025+0000 7f4606000640 1 -- start start 2024-04-28T13:33:18.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.027+0000 7f4606000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4600072020 0x7f46000724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.027+0000 7f4606000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f46000b3de0 0x7f46000b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.027+0000 7f4606000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46000ceef0 con 0x7f460014c600 2024-04-28T13:33:18.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.027+0000 7f4606000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f460014c9e0 con 0x7f46000b3de0 2024-04-28T13:33:18.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.027+0000 7f4606000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f46000cf1d0 con 0x7f4600072020 2024-04-28T13:33:18.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.027+0000 7f45ffe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f46000b3de0 0x7f46000b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.027+0000 7f45ff400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f460014c600 0x7f46000d0f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:57006/0 (socket says 172.21.15.196:57006) 2024-04-28T13:33:18.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.027+0000 7f45ffe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f46000b3de0 0x7f46000b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34960/0 (socket says 172.21.15.196:34960) 2024-04-28T13:33:18.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.027+0000 7f45ff400640 1 -- 172.21.15.196:0/3348154512 learned_addr learned my addr 172.21.15.196:0/3348154512 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:18.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.028+0000 7f45f6a00640 1 --2- 172.21.15.196:0/3348154512 >> v2:172.21.15.203:3300/0 conn(0x7f4600072020 0x7f46000724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.028+0000 7f45ffe00640 1 -- 172.21.15.196:0/3348154512 >> v2:172.21.15.203:3300/0 conn(0x7f4600072020 msgr2=0x7f46000724b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:18.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.028+0000 7f45ffe00640 1 --2- 172.21.15.196:0/3348154512 >> v2:172.21.15.203:3300/0 conn(0x7f4600072020 0x7f46000724b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.028+0000 7f45ffe00640 1 -- 172.21.15.196:0/3348154512 >> v1:172.21.15.196:6789/0 conn(0x7f460014c600 legacy=0x7f46000d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.028+0000 7f45ffe00640 1 -- 172.21.15.196:0/3348154512 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46000cf4b0 con 0x7f46000b3de0 2024-04-28T13:33:18.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.028+0000 7f45fea00640 1 -- 172.21.15.196:0/3348154512 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1638913019 0 0) 0x7f46000ceef0 con 0x7f460014c600 2024-04-28T13:33:18.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.029+0000 7f45ffe00640 1 --2- 172.21.15.196:0/3348154512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46000b3de0 0x7f46000b4210 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4600071de0 tx=0x7f45f802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a58305d5e6b62c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:18.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.029+0000 7f45fea00640 1 -- 172.21.15.196:0/3348154512 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8007740 con 0x7f46000b3de0 2024-04-28T13:33:18.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.029+0000 7f45fea00640 1 -- 172.21.15.196:0/3348154512 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f80078e0 con 0x7f46000b3de0 2024-04-28T13:33:18.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.029+0000 7f45fea00640 1 -- 172.21.15.196:0/3348154512 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8007cb0 con 0x7f46000b3de0 2024-04-28T13:33:18.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.030+0000 7f4606000640 1 -- 172.21.15.196:0/3348154512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46000b3de0 msgr2=0x7f46000b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.030+0000 7f4606000640 1 --2- 172.21.15.196:0/3348154512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46000b3de0 0x7f46000b4210 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4600071de0 tx=0x7f45f802a680 comp rx=0 tx=0).stop 2024-04-28T13:33:18.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.030+0000 7f4606000640 1 -- 172.21.15.196:0/3348154512 shutdown_connections 2024-04-28T13:33:18.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.030+0000 7f4606000640 1 --2- 172.21.15.196:0/3348154512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46000b3de0 0x7f46000b4210 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.030+0000 7f4606000640 1 --2- 172.21.15.196:0/3348154512 >> v2:172.21.15.203:3300/0 conn(0x7f4600072020 0x7f46000724b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.030+0000 7f4606000640 1 -- 172.21.15.196:0/3348154512 >> 172.21.15.196:0/3348154512 conn(0x7f46000cd830 msgr2=0x7f46000cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:18.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.030+0000 7f4606000640 1 -- 172.21.15.196:0/3348154512 shutdown_connections 2024-04-28T13:33:18.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.030+0000 7f4606000640 1 -- 172.21.15.196:0/3348154512 wait complete. 2024-04-28T13:33:18.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.032+0000 7f4606000640 1 Processor -- start 2024-04-28T13:33:18.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.032+0000 7f4606000640 1 -- start start 2024-04-28T13:33:18.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.033+0000 7f4606000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4600072020 0x7f4600150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.033+0000 7f4606000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46000b3de0 0x7f46001fc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.033+0000 7f4606000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f460014c600 0x7f46001fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.033+0000 7f4606000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46000cff90 con 0x7f460014c600 2024-04-28T13:33:18.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.033+0000 7f4606000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f46000cfa70 con 0x7f4600072020 2024-04-28T13:33:18.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.033+0000 7f4606000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f46000d1630 con 0x7f46000b3de0 2024-04-28T13:33:18.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.033+0000 7f45ffe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f460014c600 0x7f46001fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.034+0000 7f45ffe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f460014c600 0x7f46001fcb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34966/0 (socket says 172.21.15.196:34966) 2024-04-28T13:33:18.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.034+0000 7f45ffe00640 1 -- 172.21.15.196:0/2668164732 learned_addr learned my addr 172.21.15.196:0/2668164732 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:18.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.034+0000 7f45ff400640 1 --2- 172.21.15.196:0/2668164732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4600072020 0x7f4600150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.034+0000 7f45ffe00640 1 -- 172.21.15.196:0/2668164732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46000b3de0 msgr2=0x7f46001fc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.034+0000 7f45ffe00640 1 --2- 172.21.15.196:0/2668164732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46000b3de0 0x7f46001fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.034+0000 7f45ffe00640 1 -- 172.21.15.196:0/2668164732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4600072020 msgr2=0x7f4600150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.034+0000 7f45ffe00640 1 --2- 172.21.15.196:0/2668164732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4600072020 0x7f4600150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.034+0000 7f45ffe00640 1 -- 172.21.15.196:0/2668164732 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46001516f0 con 0x7f460014c600 2024-04-28T13:33:18.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.034+0000 7f45ffe00640 1 --2- 172.21.15.196:0/2668164732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f460014c600 0x7f46001fcb50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f45f802ac70 tx=0x7f45f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:18.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.034+0000 7f45f7e00640 1 -- 172.21.15.196:0/2668164732 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8040070 con 0x7f460014c600 2024-04-28T13:33:18.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.035+0000 7f45f7e00640 1 -- 172.21.15.196:0/2668164732 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f8031bc0 con 0x7f460014c600 2024-04-28T13:33:18.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.035+0000 7f45f7e00640 1 -- 172.21.15.196:0/2668164732 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8009450 con 0x7f460014c600 2024-04-28T13:33:18.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.035+0000 7f4606000640 1 -- 172.21.15.196:0/2668164732 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46001519d0 con 0x7f460014c600 2024-04-28T13:33:18.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.035+0000 7f4606000640 1 -- 172.21.15.196:0/2668164732 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46001564c0 con 0x7f460014c600 2024-04-28T13:33:18.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.036+0000 7f4606000640 1 -- 172.21.15.196:0/2668164732 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f460006a290 con 0x7f460014c600 2024-04-28T13:33:18.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.037+0000 7f45f7e00640 1 -- 172.21.15.196:0/2668164732 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f45f8039040 con 0x7f460014c600 2024-04-28T13:33:18.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.038+0000 7f45f7e00640 1 --2- 172.21.15.196:0/2668164732 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f45dc06c980 0x7f45dc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.038+0000 7f45f7e00640 1 -- 172.21.15.196:0/2668164732 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f45f80ad430 con 0x7f460014c600 2024-04-28T13:33:18.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.038+0000 7f45ff400640 1 --2- 172.21.15.196:0/2668164732 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f45dc06c980 0x7f45dc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.040+0000 7f45ff400640 1 --2- 172.21.15.196:0/2668164732 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f45dc06c980 0x7f45dc06ee40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f45f0000f60 tx=0x7f45f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:18.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.043+0000 7f45f7e00640 1 -- 172.21.15.196:0/2668164732 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45f807b790 con 0x7f460014c600 2024-04-28T13:33:18.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.283+0000 7f4606000640 1 -- 172.21.15.196:0/2668164732 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46000d11c0 con 0x7f460014c600 2024-04-28T13:33:18.285 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.284+0000 7f45f7e00640 1 -- 172.21.15.196:0/2668164732 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5191 (secure 0 0 0) 0x7f45f807b090 con 0x7f460014c600 2024-04-28T13:33:18.285 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:18.286 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/931076983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":931076983},{"type":"v1","addr":"172.21.15.196:6834","nonce":931076983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1114474485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1114474485},{"type":"v1","addr":"172.21.15.196:6835","nonce":1114474485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2070123371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2070123371},{"type":"v1","addr":"172.21.15.203:6837","nonce":2070123371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:32:31.669472+0000","modified":"2024-04-28T13:33:02.550473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4694},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4694":{"gid":4694,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6836/1998460400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1998460400},{"type":"v1","addr":"172.21.15.203:6836","nonce":1998460400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4694,"qdb_cluster":[4694]},"id":2}]} 2024-04-28T13:33:18.286 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 22 2024-04-28T13:33:18.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.287+0000 7f4606000640 1 -- 172.21.15.196:0/2668164732 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f45dc06c980 msgr2=0x7f45dc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.287+0000 7f4606000640 1 --2- 172.21.15.196:0/2668164732 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f45dc06c980 0x7f45dc06ee40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f45f0000f60 tx=0x7f45f0009040 comp rx=0 tx=0).stop 2024-04-28T13:33:18.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.287+0000 7f4606000640 1 -- 172.21.15.196:0/2668164732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f460014c600 msgr2=0x7f46001fcb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.287+0000 7f4606000640 1 --2- 172.21.15.196:0/2668164732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f460014c600 0x7f46001fcb50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f45f802ac70 tx=0x7f45f8004a40 comp rx=0 tx=0).stop 2024-04-28T13:33:18.288 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.287+0000 7f4606000640 1 -- 172.21.15.196:0/2668164732 shutdown_connections 2024-04-28T13:33:18.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.287+0000 7f4606000640 1 --2- 172.21.15.196:0/2668164732 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f45dc06c980 0x7f45dc06ee40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.287+0000 7f4606000640 1 --2- 172.21.15.196:0/2668164732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f460014c600 0x7f46001fcb50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.287+0000 7f4606000640 1 --2- 172.21.15.196:0/2668164732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46000b3de0 0x7f46001fc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.287+0000 7f4606000640 1 --2- 172.21.15.196:0/2668164732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4600072020 0x7f4600150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.287+0000 7f4606000640 1 -- 172.21.15.196:0/2668164732 >> 172.21.15.196:0/2668164732 conn(0x7f46000cd830 msgr2=0x7f46000b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:18.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.287+0000 7f4606000640 1 -- 172.21.15.196:0/2668164732 shutdown_connections 2024-04-28T13:33:18.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.287+0000 7f4606000640 1 -- 172.21.15.196:0/2668164732 wait complete. 2024-04-28T13:33:18.307 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:33:18.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.508+0000 7f8362600640 1 Processor -- start 2024-04-28T13:33:18.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.508+0000 7f8362600640 1 -- start start 2024-04-28T13:33:18.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.509+0000 7f8362600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f835c0b5ee0 0x7f835c0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.510+0000 7f8362600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f835c15e380 con 0x7f835c09ec80 2024-04-28T13:33:18.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.510+0000 7f8362600640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f835c0b4b20 con 0x7f835c0d1760 2024-04-28T13:33:18.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.510+0000 7f8362600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f835c0b3f30 con 0x7f835c0b5ee0 2024-04-28T13:33:18.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.510+0000 7f8361200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f835c0b5ee0 0x7f835c0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.510+0000 7f8361200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f835c0b5ee0 0x7f835c0b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34968/0 (socket says 172.21.15.196:34968) 2024-04-28T13:33:18.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.510+0000 7f8361200640 1 -- 172.21.15.196:0/3365482537 learned_addr learned my addr 172.21.15.196:0/3365482537 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:18.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.510+0000 7f8353e00640 1 --1- 172.21.15.196:0/3365482537 >> v1:172.21.15.196:6789/0 conn(0x7f835c09ec80 0x7f835c0d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:57016/0 (socket says 172.21.15.196:57016) 2024-04-28T13:33:18.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.510+0000 7f8361200640 1 -- 172.21.15.196:0/3365482537 >> v1:172.21.15.203:6789/0 conn(0x7f835c0d1760 legacy=0x7f835c06df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:18.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.510+0000 7f8361200640 1 -- 172.21.15.196:0/3365482537 >> v1:172.21.15.196:6789/0 conn(0x7f835c09ec80 legacy=0x7f835c0d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.510+0000 7f8361200640 1 -- 172.21.15.196:0/3365482537 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c0b41f0 con 0x7f835c0b5ee0 2024-04-28T13:33:18.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.510+0000 7f835be00640 1 -- 172.21.15.196:0/3365482537 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3282746197 0 0) 0x7f835c15e380 con 0x7f835c09ec80 2024-04-28T13:33:18.513 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.511+0000 7f8361200640 1 --2- 172.21.15.196:0/3365482537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c0b5ee0 0x7f835c0b4f40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8344009870 tx=0x7f834402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e50428515704fd server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:18.513 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.512+0000 7f835be00640 1 -- 172.21.15.196:0/3365482537 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344037070 con 0x7f835c0b5ee0 2024-04-28T13:33:18.513 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.512+0000 7f835be00640 1 -- 172.21.15.196:0/3365482537 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8344032440 con 0x7f835c0b5ee0 2024-04-28T13:33:18.513 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.512+0000 7f835be00640 1 -- 172.21.15.196:0/3365482537 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344032810 con 0x7f835c0b5ee0 2024-04-28T13:33:18.513 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.512+0000 7f8362600640 1 -- 172.21.15.196:0/3365482537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c0b5ee0 msgr2=0x7f835c0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.513 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.512+0000 7f8362600640 1 --2- 172.21.15.196:0/3365482537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c0b5ee0 0x7f835c0b4f40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8344009870 tx=0x7f834402a2a0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.513 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.512+0000 7f8362600640 1 -- 172.21.15.196:0/3365482537 shutdown_connections 2024-04-28T13:33:18.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.512+0000 7f8362600640 1 --2- 172.21.15.196:0/3365482537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c0b5ee0 0x7f835c0b4f40 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.512+0000 7f8362600640 1 -- 172.21.15.196:0/3365482537 >> 172.21.15.196:0/3365482537 conn(0x7f835c06c6c0 msgr2=0x7f835c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:18.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.512+0000 7f8362600640 1 -- 172.21.15.196:0/3365482537 shutdown_connections 2024-04-28T13:33:18.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.512+0000 7f8362600640 1 -- 172.21.15.196:0/3365482537 wait complete. 2024-04-28T13:33:18.515 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.514+0000 7f8362600640 1 Processor -- start 2024-04-28T13:33:18.515 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.514+0000 7f8362600640 1 -- start start 2024-04-28T13:33:18.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.515+0000 7f8362600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f835c09ec80 0x7f835c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.515+0000 7f8362600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c0b5ee0 0x7f835c0b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.515+0000 7f8362600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f835c0d1760 0x7f835c072200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.515+0000 7f8362600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f835c0b3c20 con 0x7f835c0b5ee0 2024-04-28T13:33:18.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.515+0000 7f8362600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f835c0b47b0 con 0x7f835c09ec80 2024-04-28T13:33:18.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.515+0000 7f8362600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f835c0add70 con 0x7f835c0d1760 2024-04-28T13:33:18.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.515+0000 7f8353e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c0b5ee0 0x7f835c0b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.516+0000 7f8353e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c0b5ee0 0x7f835c0b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34980/0 (socket says 172.21.15.196:34980) 2024-04-28T13:33:18.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.516+0000 7f8353e00640 1 -- 172.21.15.196:0/2433460459 learned_addr learned my addr 172.21.15.196:0/2433460459 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:18.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.516+0000 7f8361200640 1 --2- 172.21.15.196:0/2433460459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f835c09ec80 0x7f835c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.516+0000 7f8361c00640 1 --2- 172.21.15.196:0/2433460459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f835c0d1760 0x7f835c072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.516+0000 7f8353e00640 1 -- 172.21.15.196:0/2433460459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f835c0d1760 msgr2=0x7f835c072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.516+0000 7f8353e00640 1 --2- 172.21.15.196:0/2433460459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f835c0d1760 0x7f835c072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.516+0000 7f8353e00640 1 -- 172.21.15.196:0/2433460459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f835c09ec80 msgr2=0x7f835c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.516+0000 7f8353e00640 1 --2- 172.21.15.196:0/2433460459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f835c09ec80 0x7f835c0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.516+0000 7f8353e00640 1 -- 172.21.15.196:0/2433460459 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c154820 con 0x7f835c0b5ee0 2024-04-28T13:33:18.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.516+0000 7f8361200640 1 --2- 172.21.15.196:0/2433460459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f835c09ec80 0x7f835c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:18.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.516+0000 7f8353e00640 1 --2- 172.21.15.196:0/2433460459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c0b5ee0 0x7f835c0b1e00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f834c00c4b0 tx=0x7f834c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:18.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.517+0000 7f8359600640 1 -- 172.21.15.196:0/2433460459 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c018070 con 0x7f835c0b5ee0 2024-04-28T13:33:18.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.517+0000 7f8359600640 1 -- 172.21.15.196:0/2433460459 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f834c007500 con 0x7f835c0b5ee0 2024-04-28T13:33:18.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.517+0000 7f8359600640 1 -- 172.21.15.196:0/2433460459 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c004a20 con 0x7f835c0b5ee0 2024-04-28T13:33:18.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.517+0000 7f8362600640 1 -- 172.21.15.196:0/2433460459 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835c154b00 con 0x7f835c0b5ee0 2024-04-28T13:33:18.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.517+0000 7f8362600640 1 -- 172.21.15.196:0/2433460459 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835c1550c0 con 0x7f835c0b5ee0 2024-04-28T13:33:18.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.518+0000 7f8362600640 1 -- 172.21.15.196:0/2433460459 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f835c06a290 con 0x7f835c0b5ee0 2024-04-28T13:33:18.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.519+0000 7f8359600640 1 -- 172.21.15.196:0/2433460459 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f834c01b070 con 0x7f835c0b5ee0 2024-04-28T13:33:18.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.520+0000 7f8359600640 1 --2- 172.21.15.196:0/2433460459 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f833806c980 0x7f833806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.520+0000 7f8359600640 1 -- 172.21.15.196:0/2433460459 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f834c08aff0 con 0x7f835c0b5ee0 2024-04-28T13:33:18.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.521+0000 7f8361200640 1 --2- 172.21.15.196:0/2433460459 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f833806c980 0x7f833806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.522+0000 7f8361200640 1 --2- 172.21.15.196:0/2433460459 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f833806c980 0x7f833806ee40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8344034000 tx=0x7f834402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:18.525 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.524+0000 7f8359600640 1 -- 172.21.15.196:0/2433460459 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f834c0593d0 con 0x7f835c0b5ee0 2024-04-28T13:33:18.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.766+0000 7f8362600640 1 -- 172.21.15.196:0/2433460459 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f835c06ee60 con 0x7f835c0b5ee0 2024-04-28T13:33:18.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.767+0000 7f8359600640 1 -- 172.21.15.196:0/2433460459 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5191 (secure 0 0 0) 0x7f834c058cd0 con 0x7f835c0b5ee0 2024-04-28T13:33:18.767 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:18.767 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/931076983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":931076983},{"type":"v1","addr":"172.21.15.196:6834","nonce":931076983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1114474485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1114474485},{"type":"v1","addr":"172.21.15.196:6835","nonce":1114474485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2070123371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2070123371},{"type":"v1","addr":"172.21.15.203:6837","nonce":2070123371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:32:31.669472+0000","modified":"2024-04-28T13:33:02.550473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4694},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4694":{"gid":4694,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6836/1998460400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1998460400},{"type":"v1","addr":"172.21.15.203:6836","nonce":1998460400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4694,"qdb_cluster":[4694]},"id":2}]} 2024-04-28T13:33:18.768 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 22 2024-04-28T13:33:18.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.770+0000 7f8362600640 1 -- 172.21.15.196:0/2433460459 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f833806c980 msgr2=0x7f833806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.770+0000 7f8362600640 1 --2- 172.21.15.196:0/2433460459 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f833806c980 0x7f833806ee40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8344034000 tx=0x7f834402e000 comp rx=0 tx=0).stop 2024-04-28T13:33:18.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.770+0000 7f8362600640 1 -- 172.21.15.196:0/2433460459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c0b5ee0 msgr2=0x7f835c0b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.770+0000 7f8362600640 1 --2- 172.21.15.196:0/2433460459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c0b5ee0 0x7f835c0b1e00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f834c00c4b0 tx=0x7f834c00c970 comp rx=0 tx=0).stop 2024-04-28T13:33:18.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.770+0000 7f8362600640 1 -- 172.21.15.196:0/2433460459 shutdown_connections 2024-04-28T13:33:18.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.770+0000 7f8362600640 1 --2- 172.21.15.196:0/2433460459 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f833806c980 0x7f833806ee40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.770 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.770+0000 7f8362600640 1 --2- 172.21.15.196:0/2433460459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f835c0d1760 0x7f835c072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.770+0000 7f8362600640 1 --2- 172.21.15.196:0/2433460459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f835c0b5ee0 0x7f835c0b1e00 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.770+0000 7f8362600640 1 --2- 172.21.15.196:0/2433460459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f835c09ec80 0x7f835c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.770+0000 7f8362600640 1 -- 172.21.15.196:0/2433460459 >> 172.21.15.196:0/2433460459 conn(0x7f835c06c6c0 msgr2=0x7f835c09ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:18.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.770+0000 7f8362600640 1 -- 172.21.15.196:0/2433460459 shutdown_connections 2024-04-28T13:33:18.771 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.770+0000 7f8362600640 1 -- 172.21.15.196:0/2433460459 wait complete. 2024-04-28T13:33:18.788 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:33:18.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.977+0000 7fae3e200640 1 Processor -- start 2024-04-28T13:33:18.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.977+0000 7fae3e200640 1 -- start start 2024-04-28T13:33:18.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.979+0000 7fae3e200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae380d12a0 0x7fae381632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.979+0000 7fae3e200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae38163910 con 0x7fae38162e50 2024-04-28T13:33:18.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.979+0000 7fae3e200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae38163bf0 con 0x7fae38073fc0 2024-04-28T13:33:18.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.979+0000 7fae3e200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fae3806ea50 con 0x7fae380d12a0 2024-04-28T13:33:18.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.979+0000 7fae36a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fae38162e50 0x7fae380d0b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:57028/0 (socket says 172.21.15.196:57028) 2024-04-28T13:33:18.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.979+0000 7fae36a00640 1 -- 172.21.15.196:0/165777886 learned_addr learned my addr 172.21.15.196:0/165777886 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:18.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae36000640 1 -- 172.21.15.196:0/165777886 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3251355815 0 0) 0x7fae38163910 con 0x7fae38162e50 2024-04-28T13:33:18.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae37e00640 1 --2- 172.21.15.196:0/165777886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae380d12a0 0x7fae381632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae36000640 1 -- 172.21.15.196:0/165777886 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae1c003520 con 0x7fae38162e50 2024-04-28T13:33:18.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae36000640 1 -- 172.21.15.196:0/165777886 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2773980273 0 0) 0x7fae1c003520 con 0x7fae38162e50 2024-04-28T13:33:18.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae36000640 1 -- 172.21.15.196:0/165777886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae380d12a0 msgr2=0x7fae381632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:18.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae36000640 1 --2- 172.21.15.196:0/165777886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae380d12a0 0x7fae381632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae36000640 1 -- 172.21.15.196:0/165777886 >> v1:172.21.15.203:6789/0 conn(0x7fae38073fc0 legacy=0x7fae38072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae36000640 1 -- 172.21.15.196:0/165777886 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3806ed30 con 0x7fae38162e50 2024-04-28T13:33:18.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae36000640 1 -- 172.21.15.196:0/165777886 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fae24001670 con 0x7fae38162e50 2024-04-28T13:33:18.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae36000640 1 -- 172.21.15.196:0/165777886 >> v1:172.21.15.196:6789/0 conn(0x7fae38162e50 legacy=0x7fae380d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae36000640 1 --2- 172.21.15.196:0/165777886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae380d12a0 0x7fae1c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae36000640 1 --2- 172.21.15.196:0/165777886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae1c004010 0x7fae1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae36000640 1 --2- 172.21.15.196:0/165777886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae1c004d40 0x7fae1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae36000640 1 -- 172.21.15.196:0/165777886 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae3806ea50 con 0x7fae1c004010 2024-04-28T13:33:18.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae36000640 1 -- 172.21.15.196:0/165777886 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c00bc10 con 0x7fae1c004d40 2024-04-28T13:33:18.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.980+0000 7fae36000640 1 -- 172.21.15.196:0/165777886 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fae1c00be10 con 0x7fae380d12a0 2024-04-28T13:33:18.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.981+0000 7fae36a00640 1 --2- 172.21.15.196:0/165777886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae1c004010 0x7fae1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.982 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.981+0000 7fae37400640 1 --2- 172.21.15.196:0/165777886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae380d12a0 0x7fae1c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.981+0000 7fae37e00640 1 --2- 172.21.15.196:0/165777886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae1c004d40 0x7fae1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.981+0000 7fae36a00640 1 -- 172.21.15.196:0/165777886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae380d12a0 msgr2=0x7fae1c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:18.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.981+0000 7fae36a00640 1 --2- 172.21.15.196:0/165777886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae380d12a0 0x7fae1c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.981+0000 7fae36a00640 1 -- 172.21.15.196:0/165777886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae1c004d40 msgr2=0x7fae1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:18.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.981+0000 7fae36a00640 1 --2- 172.21.15.196:0/165777886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae1c004d40 0x7fae1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.981+0000 7fae36a00640 1 -- 172.21.15.196:0/165777886 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1c003520 con 0x7fae1c004010 2024-04-28T13:33:18.983 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.981+0000 7fae37e00640 1 --2- 172.21.15.196:0/165777886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae1c004d40 0x7fae1c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:18.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.982+0000 7fae36a00640 1 --2- 172.21.15.196:0/165777886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae1c004010 0x7fae1c004800 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fae2400c210 tx=0x7fae1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3621ee5d02c938b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:18.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.982+0000 7fae36000640 1 -- 172.21.15.196:0/165777886 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24007d30 con 0x7fae1c004010 2024-04-28T13:33:18.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.982+0000 7fae36000640 1 -- 172.21.15.196:0/165777886 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae24005af0 con 0x7fae1c004010 2024-04-28T13:33:18.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.982+0000 7fae36000640 1 -- 172.21.15.196:0/165777886 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24007070 con 0x7fae1c004010 2024-04-28T13:33:18.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.983+0000 7fae3e200640 1 -- 172.21.15.196:0/165777886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae1c004010 msgr2=0x7fae1c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.983+0000 7fae3e200640 1 --2- 172.21.15.196:0/165777886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae1c004010 0x7fae1c004800 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fae2400c210 tx=0x7fae1c005830 comp rx=0 tx=0).stop 2024-04-28T13:33:18.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.983+0000 7fae37e00640 1 -- 172.21.15.196:0/165777886 reap_dead start 2024-04-28T13:33:18.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.983+0000 7fae3e200640 1 -- 172.21.15.196:0/165777886 shutdown_connections 2024-04-28T13:33:18.985 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.983+0000 7fae3e200640 1 -- 172.21.15.196:0/165777886 >> 172.21.15.196:0/165777886 conn(0x7fae380cd830 msgr2=0x7fae380cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:18.985 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.983+0000 7fae3e200640 1 -- 172.21.15.196:0/165777886 shutdown_connections 2024-04-28T13:33:18.985 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.983+0000 7fae3e200640 1 -- 172.21.15.196:0/165777886 wait complete. 2024-04-28T13:33:18.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.985+0000 7fae3e200640 1 Processor -- start 2024-04-28T13:33:18.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.985+0000 7fae3e200640 1 -- start start 2024-04-28T13:33:18.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.986+0000 7fae3e200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae1c004010 0x7fae3815ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.986+0000 7fae3e200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae3813db10 0x7fae38161380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.986+0000 7fae3e200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae381618c0 0x7fae381e8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.986+0000 7fae3e200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae380743a0 con 0x7fae3813db10 2024-04-28T13:33:18.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.986+0000 7fae3e200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fae381632c0 con 0x7fae1c004010 2024-04-28T13:33:18.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.986+0000 7fae3e200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fae380b5150 con 0x7fae381618c0 2024-04-28T13:33:18.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.986+0000 7fae36a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae3813db10 0x7fae38161380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.986+0000 7fae36a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae3813db10 0x7fae38161380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34984/0 (socket says 172.21.15.196:34984) 2024-04-28T13:33:18.986 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.986+0000 7fae36a00640 1 -- 172.21.15.196:0/447183775 learned_addr learned my addr 172.21.15.196:0/447183775 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:18.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.986+0000 7fae36a00640 1 -- 172.21.15.196:0/447183775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae381618c0 msgr2=0x7fae381e8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.986+0000 7fae36a00640 1 --2- 172.21.15.196:0/447183775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae381618c0 0x7fae381e8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.987 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.986+0000 7fae36a00640 1 -- 172.21.15.196:0/447183775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae1c004010 msgr2=0x7fae3815ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:18.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.986+0000 7fae36a00640 1 --2- 172.21.15.196:0/447183775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae1c004010 0x7fae3815ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:18.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.986+0000 7fae36a00640 1 -- 172.21.15.196:0/447183775 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3813df70 con 0x7fae3813db10 2024-04-28T13:33:18.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.987+0000 7fae36a00640 1 --2- 172.21.15.196:0/447183775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae3813db10 0x7fae38161380 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fae24034040 tx=0x7fae24001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:18.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.987+0000 7fae2f400640 1 -- 172.21.15.196:0/447183775 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24042020 con 0x7fae3813db10 2024-04-28T13:33:18.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.987+0000 7fae2f400640 1 -- 172.21.15.196:0/447183775 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae24004bf0 con 0x7fae3813db10 2024-04-28T13:33:18.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.987+0000 7fae2f400640 1 -- 172.21.15.196:0/447183775 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24007d10 con 0x7fae3813db10 2024-04-28T13:33:18.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.987+0000 7fae3e200640 1 -- 172.21.15.196:0/447183775 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae380b4540 con 0x7fae3813db10 2024-04-28T13:33:18.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.987+0000 7fae3e200640 1 -- 172.21.15.196:0/447183775 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae380b4b00 con 0x7fae3813db10 2024-04-28T13:33:18.988 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.988+0000 7fae3e200640 1 -- 172.21.15.196:0/447183775 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae38161ce0 con 0x7fae3813db10 2024-04-28T13:33:18.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.989+0000 7fae2f400640 1 -- 172.21.15.196:0/447183775 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fae24045020 con 0x7fae3813db10 2024-04-28T13:33:18.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.990+0000 7fae2f400640 1 --2- 172.21.15.196:0/447183775 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fae1806c9b0 0x7fae1806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:18.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.990+0000 7fae2f400640 1 -- 172.21.15.196:0/447183775 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fae240abf80 con 0x7fae3813db10 2024-04-28T13:33:18.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.991+0000 7fae37400640 1 --2- 172.21.15.196:0/447183775 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fae1806c9b0 0x7fae1806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:18.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.992+0000 7fae37400640 1 --2- 172.21.15.196:0/447183775 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fae1806c9b0 0x7fae1806ee70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fae28007fd0 tx=0x7fae28009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:18.994 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:18.995+0000 7fae2f400640 1 -- 172.21.15.196:0/447183775 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae2407a2f0 con 0x7fae3813db10 2024-04-28T13:33:19.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.194+0000 7fae3e200640 1 -- 172.21.15.196:0/447183775 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae3806a290 con 0x7fae3813db10 2024-04-28T13:33:19.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.197+0000 7fae2f400640 1 -- 172.21.15.196:0/447183775 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16834 (secure 0 0 0) 0x7fae24079bf0 con 0x7fae3813db10 2024-04-28T13:33:19.198 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:19.198 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":34,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:32:58.527505+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:32:29.352348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-28T13:32:30.637498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6836/459640695":"2024-04-29T13:32:58.527451+0000","172.21.15.203:6835/459640695":"2024-04-29T13:32:58.527451+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:33:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.200+0000 7fae3e200640 1 -- 172.21.15.196:0/447183775 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fae1806c9b0 msgr2=0x7fae1806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.200+0000 7fae3e200640 1 --2- 172.21.15.196:0/447183775 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fae1806c9b0 0x7fae1806ee70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fae28007fd0 tx=0x7fae28009040 comp rx=0 tx=0).stop 2024-04-28T13:33:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.200+0000 7fae3e200640 1 -- 172.21.15.196:0/447183775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae3813db10 msgr2=0x7fae38161380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.200+0000 7fae3e200640 1 --2- 172.21.15.196:0/447183775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae3813db10 0x7fae38161380 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fae24034040 tx=0x7fae24001a40 comp rx=0 tx=0).stop 2024-04-28T13:33:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.200+0000 7fae3e200640 1 -- 172.21.15.196:0/447183775 shutdown_connections 2024-04-28T13:33:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.200+0000 7fae3e200640 1 --2- 172.21.15.196:0/447183775 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fae1806c9b0 0x7fae1806ee70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.200+0000 7fae3e200640 1 --2- 172.21.15.196:0/447183775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae381618c0 0x7fae381e8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.200+0000 7fae3e200640 1 --2- 172.21.15.196:0/447183775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae3813db10 0x7fae38161380 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.200+0000 7fae3e200640 1 --2- 172.21.15.196:0/447183775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae1c004010 0x7fae3815ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.200+0000 7fae3e200640 1 -- 172.21.15.196:0/447183775 >> 172.21.15.196:0/447183775 conn(0x7fae380cd830 msgr2=0x7fae380b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.200+0000 7fae3e200640 1 -- 172.21.15.196:0/447183775 shutdown_connections 2024-04-28T13:33:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.200+0000 7fae3e200640 1 -- 172.21.15.196:0/447183775 wait complete. 2024-04-28T13:33:19.219 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-28T13:33:19.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.421+0000 7f180d600640 1 Processor -- start 2024-04-28T13:33:19.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.422+0000 7f180d600640 1 -- start start 2024-04-28T13:33:19.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.423+0000 7f180d600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f18081629c0 0x7f18080d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:19.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.423+0000 7f180d600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1808163740 0x7f18080b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:19.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.423+0000 7f180d600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1808162da0 con 0x7f1808163bf0 2024-04-28T13:33:19.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.423+0000 7f180d600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f18080b6190 con 0x7f18081629c0 2024-04-28T13:33:19.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.423+0000 7f180d600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f18080b6470 con 0x7f1808163740 2024-04-28T13:33:19.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.423+0000 7f1807e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f18081629c0 0x7f18080d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:19.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.423+0000 7f17ff400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f1808163bf0 0x7f1808163030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:57044/0 (socket says 172.21.15.196:57044) 2024-04-28T13:33:19.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.424+0000 7f1807e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f18081629c0 0x7f18080d3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34996/0 (socket says 172.21.15.196:34996) 2024-04-28T13:33:19.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.424+0000 7f17ff400640 1 -- 172.21.15.196:0/2499906538 learned_addr learned my addr 172.21.15.196:0/2499906538 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:19.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.424+0000 7f180cc00640 1 --2- 172.21.15.196:0/2499906538 >> v2:172.21.15.203:3300/0 conn(0x7f1808163740 0x7f18080b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:19.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.424+0000 7f1807e00640 1 -- 172.21.15.196:0/2499906538 >> v2:172.21.15.203:3300/0 conn(0x7f1808163740 msgr2=0x7f18080b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:19.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.424+0000 7f1807e00640 1 --2- 172.21.15.196:0/2499906538 >> v2:172.21.15.203:3300/0 conn(0x7f1808163740 0x7f18080b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.424+0000 7f1807e00640 1 -- 172.21.15.196:0/2499906538 >> v1:172.21.15.196:6789/0 conn(0x7f1808163bf0 legacy=0x7f1808163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:19.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.424+0000 7f1807e00640 1 -- 172.21.15.196:0/2499906538 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18080b6750 con 0x7f18081629c0 2024-04-28T13:33:19.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.424+0000 7f180cc00640 1 --2- 172.21.15.196:0/2499906538 >> v2:172.21.15.203:3300/0 conn(0x7f1808163740 0x7f18080b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:19.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.425+0000 7f1807e00640 1 --2- 172.21.15.196:0/2499906538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18081629c0 0x7f18080d3710 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f180808f170 tx=0x7f17f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10cfbe3f2a67cfd server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:19.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.425+0000 7f1807400640 1 -- 172.21.15.196:0/2499906538 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8037070 con 0x7f18081629c0 2024-04-28T13:33:19.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.425+0000 7f1807400640 1 -- 172.21.15.196:0/2499906538 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f8032440 con 0x7f18081629c0 2024-04-28T13:33:19.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.425+0000 7f1807400640 1 -- 172.21.15.196:0/2499906538 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8032810 con 0x7f18081629c0 2024-04-28T13:33:19.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.426+0000 7f180d600640 1 -- 172.21.15.196:0/2499906538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18081629c0 msgr2=0x7f18080d3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:19.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.426+0000 7f180d600640 1 --2- 172.21.15.196:0/2499906538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18081629c0 0x7f18080d3710 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f180808f170 tx=0x7f17f802a1a0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.426+0000 7f180d600640 1 -- 172.21.15.196:0/2499906538 shutdown_connections 2024-04-28T13:33:19.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.426+0000 7f180d600640 1 --2- 172.21.15.196:0/2499906538 >> v2:172.21.15.203:3300/0 conn(0x7f1808163740 0x7f18080b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.426+0000 7f180d600640 1 --2- 172.21.15.196:0/2499906538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18081629c0 0x7f18080d3710 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.426+0000 7f180d600640 1 -- 172.21.15.196:0/2499906538 >> 172.21.15.196:0/2499906538 conn(0x7f180809e570 msgr2=0x7f18080d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:19.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.426+0000 7f180d600640 1 -- 172.21.15.196:0/2499906538 shutdown_connections 2024-04-28T13:33:19.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.426+0000 7f180d600640 1 -- 172.21.15.196:0/2499906538 wait complete. 2024-04-28T13:33:19.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.428+0000 7f180d600640 1 Processor -- start 2024-04-28T13:33:19.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.428+0000 7f180d600640 1 -- start start 2024-04-28T13:33:19.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f180d600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18081629c0 0x7f18081540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:19.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f180d600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1808163740 0x7f1808154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:19.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f180d600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1808163bf0 0x7f18080b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:19.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f180d600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18080d3c50 con 0x7f1808163740 2024-04-28T13:33:19.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f180d600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f18080b1e60 con 0x7f1808163bf0 2024-04-28T13:33:19.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f180d600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f18080d3dd0 con 0x7f18081629c0 2024-04-28T13:33:19.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f17ff400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1808163740 0x7f1808154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:19.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f17ff400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1808163740 0x7f1808154630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35010/0 (socket says 172.21.15.196:35010) 2024-04-28T13:33:19.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f17ff400640 1 -- 172.21.15.196:0/3798641411 learned_addr learned my addr 172.21.15.196:0/3798641411 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:19.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f1807e00640 1 --2- 172.21.15.196:0/3798641411 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18081629c0 0x7f18081540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:19.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f17ff400640 1 -- 172.21.15.196:0/3798641411 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18081629c0 msgr2=0x7f18081540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:19.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f180cc00640 1 --2- 172.21.15.196:0/3798641411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1808163bf0 0x7f18080b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:19.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f17ff400640 1 --2- 172.21.15.196:0/3798641411 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18081629c0 0x7f18081540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f17ff400640 1 -- 172.21.15.196:0/3798641411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1808163bf0 msgr2=0x7f18080b8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:19.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f17ff400640 1 --2- 172.21.15.196:0/3798641411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1808163bf0 0x7f18080b8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.429+0000 7f17ff400640 1 -- 172.21.15.196:0/3798641411 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18080b9830 con 0x7f1808163740 2024-04-28T13:33:19.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.430+0000 7f17ff400640 1 --2- 172.21.15.196:0/3798641411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1808163740 0x7f1808154630 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f17f000c440 tx=0x7f17f000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:19.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.430+0000 7f1804c00640 1 -- 172.21.15.196:0/3798641411 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0017070 con 0x7f1808163740 2024-04-28T13:33:19.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.430+0000 7f1804c00640 1 -- 172.21.15.196:0/3798641411 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f0005ce0 con 0x7f1808163740 2024-04-28T13:33:19.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.430+0000 7f180d600640 1 -- 172.21.15.196:0/3798641411 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18080b9b10 con 0x7f1808163740 2024-04-28T13:33:19.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.430+0000 7f1804c00640 1 -- 172.21.15.196:0/3798641411 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f00072f0 con 0x7f1808163740 2024-04-28T13:33:19.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.430+0000 7f180d600640 1 -- 172.21.15.196:0/3798641411 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18080b80e0 con 0x7f1808163740 2024-04-28T13:33:19.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.431+0000 7f180d600640 1 -- 172.21.15.196:0/3798641411 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f180806a290 con 0x7f1808163740 2024-04-28T13:33:19.432 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.432+0000 7f1804c00640 1 -- 172.21.15.196:0/3798641411 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f17f001a070 con 0x7f1808163740 2024-04-28T13:33:19.433 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.433+0000 7f1804c00640 1 --2- 172.21.15.196:0/3798641411 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f17e406c980 0x7f17e406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:19.433 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.433+0000 7f1804c00640 1 -- 172.21.15.196:0/3798641411 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f17f008afd0 con 0x7f1808163740 2024-04-28T13:33:19.433 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.434+0000 7f1807e00640 1 --2- 172.21.15.196:0/3798641411 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f17e406c980 0x7f17e406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:19.434 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.435+0000 7f1807e00640 1 --2- 172.21.15.196:0/3798641411 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f17e406c980 0x7f17e406ee40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f17f8034000 tx=0x7f17f802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:19.437 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.438+0000 7f1804c00640 1 -- 172.21.15.196:0/3798641411 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f17f0059340 con 0x7f1808163740 2024-04-28T13:33:19.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.681+0000 7f180d600640 1 -- 172.21.15.196:0/3798641411 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f18080b59a0 con 0x7f1808163740 2024-04-28T13:33:19.681 INFO:tasks.ceph.mon.a.smithi196.stderr:2024-04-28T13:33:19.682+0000 7f340ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-28T13:33:19.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.685+0000 7f1804c00640 1 -- 172.21.15.196:0/3798641411 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) ==== 161+0+0 (secure 0 0 0) 0x7f17f0058c40 con 0x7f1808163740 2024-04-28T13:33:19.685 INFO:teuthology.orchestra.run.smithi196.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-28T13:33:19.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.688+0000 7f180d600640 1 -- 172.21.15.196:0/3798641411 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f17e406c980 msgr2=0x7f17e406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:19.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.688+0000 7f180d600640 1 --2- 172.21.15.196:0/3798641411 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f17e406c980 0x7f17e406ee40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f17f8034000 tx=0x7f17f802e000 comp rx=0 tx=0).stop 2024-04-28T13:33:19.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.688+0000 7f180d600640 1 -- 172.21.15.196:0/3798641411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1808163740 msgr2=0x7f1808154630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:19.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.688+0000 7f180d600640 1 --2- 172.21.15.196:0/3798641411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1808163740 0x7f1808154630 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f17f000c440 tx=0x7f17f000c900 comp rx=0 tx=0).stop 2024-04-28T13:33:19.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.688+0000 7f180d600640 1 -- 172.21.15.196:0/3798641411 shutdown_connections 2024-04-28T13:33:19.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.688+0000 7f180d600640 1 --2- 172.21.15.196:0/3798641411 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f17e406c980 0x7f17e406ee40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.688+0000 7f180d600640 1 --2- 172.21.15.196:0/3798641411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1808163bf0 0x7f18080b8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.688+0000 7f180d600640 1 --2- 172.21.15.196:0/3798641411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1808163740 0x7f1808154630 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.688+0000 7f180d600640 1 --2- 172.21.15.196:0/3798641411 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18081629c0 0x7f18081540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.688+0000 7f180d600640 1 -- 172.21.15.196:0/3798641411 >> 172.21.15.196:0/3798641411 conn(0x7f180809e570 msgr2=0x7f18080b3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:19.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.689+0000 7f180d600640 1 -- 172.21.15.196:0/3798641411 shutdown_connections 2024-04-28T13:33:19.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.689+0000 7f180d600640 1 -- 172.21.15.196:0/3798641411 wait complete. 2024-04-28T13:33:19.708 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-28T13:33:19.805 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-28T13:33:19.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.899+0000 7f94c3200640 1 Processor -- start 2024-04-28T13:33:19.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.900+0000 7f94c3200640 1 -- start start 2024-04-28T13:33:19.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.901+0000 7f94c3200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f94c40c8b60 0x7f94c40d5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:19.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.901+0000 7f94c3200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c40b62f0 con 0x7f94c4162810 2024-04-28T13:33:19.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.901+0000 7f94c3200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c40d0740 con 0x7f94c4162bf0 2024-04-28T13:33:19.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.901+0000 7f94c3200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f94c40d1620 con 0x7f94c40c8b60 2024-04-28T13:33:19.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.901+0000 7f94c2800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f94c40c8b60 0x7f94c40d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:19.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.902+0000 7f94c2800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f94c40c8b60 0x7f94c40d5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35022/0 (socket says 172.21.15.196:35022) 2024-04-28T13:33:19.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.902+0000 7f94c2800640 1 -- 172.21.15.196:0/1137460164 learned_addr learned my addr 172.21.15.196:0/1137460164 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:19.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.902+0000 7f94c1e00640 1 --1- 172.21.15.196:0/1137460164 >> v1:172.21.15.196:6789/0 conn(0x7f94c4162810 0x7f94c40d35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:57054/0 (socket says 172.21.15.196:57054) 2024-04-28T13:33:19.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.902+0000 7f94c2800640 1 -- 172.21.15.196:0/1137460164 >> v1:172.21.15.203:6789/0 conn(0x7f94c4162bf0 legacy=0x7f94c40d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:19.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.902+0000 7f94c2800640 1 -- 172.21.15.196:0/1137460164 >> v1:172.21.15.196:6789/0 conn(0x7f94c4162810 legacy=0x7f94c40d35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:19.902 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.902+0000 7f94c2800640 1 -- 172.21.15.196:0/1137460164 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c40d0a20 con 0x7f94c40c8b60 2024-04-28T13:33:19.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.903+0000 7f94c2800640 1 --2- 172.21.15.196:0/1137460164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c40c8b60 0x7f94c40d5b30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009e50 tx=0x7f94bc02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847c408160eefbba server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:19.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.903+0000 7f94c0a00640 1 -- 172.21.15.196:0/1137460164 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc036070 con 0x7f94c40c8b60 2024-04-28T13:33:19.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.903+0000 7f94c0a00640 1 -- 172.21.15.196:0/1137460164 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94bc031600 con 0x7f94c40c8b60 2024-04-28T13:33:19.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.903+0000 7f94c0a00640 1 -- 172.21.15.196:0/1137460164 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0319d0 con 0x7f94c40c8b60 2024-04-28T13:33:19.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.904+0000 7f94c3200640 1 -- 172.21.15.196:0/1137460164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c40c8b60 msgr2=0x7f94c40d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:19.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.904+0000 7f94c3200640 1 --2- 172.21.15.196:0/1137460164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c40c8b60 0x7f94c40d5b30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009e50 tx=0x7f94bc02a640 comp rx=0 tx=0).stop 2024-04-28T13:33:19.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.904+0000 7f94c3200640 1 -- 172.21.15.196:0/1137460164 shutdown_connections 2024-04-28T13:33:19.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.904+0000 7f94c3200640 1 --2- 172.21.15.196:0/1137460164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c40c8b60 0x7f94c40d5b30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.904 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.904+0000 7f94c3200640 1 -- 172.21.15.196:0/1137460164 >> 172.21.15.196:0/1137460164 conn(0x7f94c40d5010 msgr2=0x7f94c40d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:19.905 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.904+0000 7f94c3200640 1 -- 172.21.15.196:0/1137460164 shutdown_connections 2024-04-28T13:33:19.905 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.904+0000 7f94c3200640 1 -- 172.21.15.196:0/1137460164 wait complete. 2024-04-28T13:33:19.905 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.906+0000 7f94c3200640 1 Processor -- start 2024-04-28T13:33:19.906 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.906+0000 7f94c3200640 1 -- start start 2024-04-28T13:33:19.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.907+0000 7f94c3200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94c40c8b60 0x7f94c4153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:19.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.907+0000 7f94c3200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94c4162810 0x7f94c4155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:19.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.907+0000 7f94c3200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c4162bf0 0x7f94c40cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:19.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.907+0000 7f94c3200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f94c40d3d00 con 0x7f94c4162bf0 2024-04-28T13:33:19.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.907+0000 7f94c3200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f94c40d33b0 con 0x7f94c40c8b60 2024-04-28T13:33:19.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.907+0000 7f94c3200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f94c40d3e80 con 0x7f94c4162810 2024-04-28T13:33:19.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.907+0000 7f94c2800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c4162bf0 0x7f94c40cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:19.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.907+0000 7f94c2800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c4162bf0 0x7f94c40cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35024/0 (socket says 172.21.15.196:35024) 2024-04-28T13:33:19.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.907+0000 7f94c2800640 1 -- 172.21.15.196:0/4086204908 learned_addr learned my addr 172.21.15.196:0/4086204908 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:19.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.907+0000 7f94c1400640 1 --2- 172.21.15.196:0/4086204908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94c4162810 0x7f94c4155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:19.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.908+0000 7f94c1e00640 1 --2- 172.21.15.196:0/4086204908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94c40c8b60 0x7f94c4153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:19.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.908+0000 7f94c2800640 1 -- 172.21.15.196:0/4086204908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94c4162810 msgr2=0x7f94c4155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:19.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.908+0000 7f94c2800640 1 --2- 172.21.15.196:0/4086204908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94c4162810 0x7f94c4155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.908+0000 7f94c2800640 1 -- 172.21.15.196:0/4086204908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94c40c8b60 msgr2=0x7f94c4153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:19.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.908+0000 7f94c2800640 1 --2- 172.21.15.196:0/4086204908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94c40c8b60 0x7f94c4153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:19.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.908+0000 7f94c2800640 1 -- 172.21.15.196:0/4086204908 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c40b9950 con 0x7f94c4162bf0 2024-04-28T13:33:19.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.908+0000 7f94c2800640 1 --2- 172.21.15.196:0/4086204908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c4162bf0 0x7f94c40cd890 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009f70 tx=0x7f94bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:19.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.908+0000 7f94ba000640 1 -- 172.21.15.196:0/4086204908 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc036050 con 0x7f94c4162bf0 2024-04-28T13:33:19.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.908+0000 7f94ba000640 1 -- 172.21.15.196:0/4086204908 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94bc042070 con 0x7f94c4162bf0 2024-04-28T13:33:19.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.908+0000 7f94c3200640 1 -- 172.21.15.196:0/4086204908 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c40b93e0 con 0x7f94c4162bf0 2024-04-28T13:33:19.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.908+0000 7f94ba000640 1 -- 172.21.15.196:0/4086204908 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc009510 con 0x7f94c4162bf0 2024-04-28T13:33:19.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.909+0000 7f94c3200640 1 -- 172.21.15.196:0/4086204908 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c4155ce0 con 0x7f94c4162bf0 2024-04-28T13:33:19.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.910+0000 7f94c3200640 1 -- 172.21.15.196:0/4086204908 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c41567a0 con 0x7f94c4162bf0 2024-04-28T13:33:19.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.911+0000 7f94ba000640 1 -- 172.21.15.196:0/4086204908 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f94bc007bd0 con 0x7f94c4162bf0 2024-04-28T13:33:19.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.912+0000 7f94ba000640 1 --2- 172.21.15.196:0/4086204908 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f94a006c9b0 0x7f94a006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:19.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.912+0000 7f94ba000640 1 -- 172.21.15.196:0/4086204908 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5881+0+0 (secure 0 0 0) 0x7f94bc0acd80 con 0x7f94c4162bf0 2024-04-28T13:33:19.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.913+0000 7f94c1e00640 1 --2- 172.21.15.196:0/4086204908 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f94a006c9b0 0x7f94a006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:19.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.914+0000 7f94c1e00640 1 --2- 172.21.15.196:0/4086204908 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f94a006c9b0 0x7f94a006ee70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f94c4162150 tx=0x7f94ac007ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:19.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:19.917+0000 7f94ba000640 1 -- 172.21.15.196:0/4086204908 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94bc02ae20 con 0x7f94c4162bf0 2024-04-28T13:33:20.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.156+0000 7f94c3200640 1 -- 172.21.15.196:0/4086204908 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f94c4050730 con 0x7f94c4162bf0 2024-04-28T13:33:20.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.689+0000 7f94ba000640 1 -- 172.21.15.196:0/4086204908 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) ==== 104+0+0 (secure 0 0 0) 0x7f94bc045360 con 0x7f94c4162bf0 2024-04-28T13:33:20.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.692+0000 7f94c3200640 1 -- 172.21.15.196:0/4086204908 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f94a006c9b0 msgr2=0x7f94a006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:20.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.692+0000 7f94c3200640 1 --2- 172.21.15.196:0/4086204908 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f94a006c9b0 0x7f94a006ee70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f94c4162150 tx=0x7f94ac007ce0 comp rx=0 tx=0).stop 2024-04-28T13:33:20.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.693+0000 7f94c3200640 1 -- 172.21.15.196:0/4086204908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c4162bf0 msgr2=0x7f94c40cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:20.692 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.693+0000 7f94c3200640 1 --2- 172.21.15.196:0/4086204908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c4162bf0 0x7f94c40cd890 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009f70 tx=0x7f94bc004a40 comp rx=0 tx=0).stop 2024-04-28T13:33:20.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.693+0000 7f94c3200640 1 -- 172.21.15.196:0/4086204908 shutdown_connections 2024-04-28T13:33:20.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.693+0000 7f94c3200640 1 --2- 172.21.15.196:0/4086204908 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f94a006c9b0 0x7f94a006ee70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:20.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.693+0000 7f94c3200640 1 --2- 172.21.15.196:0/4086204908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c4162bf0 0x7f94c40cd890 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:20.693 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.693+0000 7f94c3200640 1 --2- 172.21.15.196:0/4086204908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94c4162810 0x7f94c4155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:20.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.693+0000 7f94c3200640 1 --2- 172.21.15.196:0/4086204908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94c40c8b60 0x7f94c4153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:20.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.693+0000 7f94c3200640 1 -- 172.21.15.196:0/4086204908 >> 172.21.15.196:0/4086204908 conn(0x7f94c40d5010 msgr2=0x7f94c40c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:20.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.693+0000 7f94c3200640 1 -- 172.21.15.196:0/4086204908 shutdown_connections 2024-04-28T13:33:20.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.693+0000 7f94c3200640 1 -- 172.21.15.196:0/4086204908 wait complete. 2024-04-28T13:33:20.718 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-28T13:33:20.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.928+0000 7f4fbf000640 1 Processor -- start 2024-04-28T13:33:20.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.928+0000 7f4fbf000640 1 -- start start 2024-04-28T13:33:20.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.930+0000 7f4fbf000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4fb81454c0 0x7f4fb80d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:20.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.930+0000 7f4fbf000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4fb80d18a0 0x7f4fb80cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:20.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.930+0000 7f4fbf000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fb806e830 con 0x7f4fb806e330 2024-04-28T13:33:20.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.930+0000 7f4fbf000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4fb80721a0 con 0x7f4fb80d18a0 2024-04-28T13:33:20.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.930+0000 7f4fbf000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4fb80ceca0 con 0x7f4fb81454c0 2024-04-28T13:33:20.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.930+0000 7f4fbd000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4fb80d18a0 0x7f4fb80cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:20.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.930+0000 7f4fbd000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4fb80d18a0 0x7f4fb80cf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35040/0 (socket says 172.21.15.196:35040) 2024-04-28T13:33:20.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.930+0000 7f4fbd000640 1 -- 172.21.15.196:0/2225131642 learned_addr learned my addr 172.21.15.196:0/2225131642 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:20.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.931+0000 7f4fb7e00640 1 --2- 172.21.15.196:0/2225131642 >> v2:172.21.15.203:3300/0 conn(0x7f4fb81454c0 0x7f4fb80d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:20.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.931+0000 7f4fbd000640 1 -- 172.21.15.196:0/2225131642 >> v2:172.21.15.203:3300/0 conn(0x7f4fb81454c0 msgr2=0x7f4fb80d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:20.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.931+0000 7f4fbd000640 1 --2- 172.21.15.196:0/2225131642 >> v2:172.21.15.203:3300/0 conn(0x7f4fb81454c0 0x7f4fb80d05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:20.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.931+0000 7f4fbd000640 1 -- 172.21.15.196:0/2225131642 >> v1:172.21.15.203:6789/0 conn(0x7f4fb806e330 legacy=0x7f4fb80d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:20.934 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.931+0000 7f4fbd000640 1 -- 172.21.15.196:0/2225131642 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb806eb10 con 0x7f4fb80d18a0 2024-04-28T13:33:20.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.932+0000 7f4fbd000640 1 --2- 172.21.15.196:0/2225131642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fb80d18a0 0x7f4fb80cf130 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4fb806fd80 tx=0x7f4fb002c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b987a25a0ab7ed server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:20.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.932+0000 7f4fb6a00640 1 -- 172.21.15.196:0/2225131642 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0009600 con 0x7f4fb80d18a0 2024-04-28T13:33:20.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.932+0000 7f4fb6a00640 1 -- 172.21.15.196:0/2225131642 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb00097a0 con 0x7f4fb80d18a0 2024-04-28T13:33:20.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.932+0000 7f4fb6a00640 1 -- 172.21.15.196:0/2225131642 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0009b70 con 0x7f4fb80d18a0 2024-04-28T13:33:20.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.933+0000 7f4fbf000640 1 -- 172.21.15.196:0/2225131642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fb80d18a0 msgr2=0x7f4fb80cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:20.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.933+0000 7f4fbf000640 1 --2- 172.21.15.196:0/2225131642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fb80d18a0 0x7f4fb80cf130 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4fb806fd80 tx=0x7f4fb002c900 comp rx=0 tx=0).stop 2024-04-28T13:33:20.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.933+0000 7f4fbf000640 1 -- 172.21.15.196:0/2225131642 shutdown_connections 2024-04-28T13:33:20.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.933+0000 7f4fbf000640 1 --2- 172.21.15.196:0/2225131642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fb80d18a0 0x7f4fb80cf130 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:20.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.933+0000 7f4fbf000640 1 --2- 172.21.15.196:0/2225131642 >> v2:172.21.15.203:3300/0 conn(0x7f4fb81454c0 0x7f4fb80d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:20.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.933+0000 7f4fbf000640 1 -- 172.21.15.196:0/2225131642 >> 172.21.15.196:0/2225131642 conn(0x7f4fb806c690 msgr2=0x7f4fb806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:20.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.933+0000 7f4fbf000640 1 -- 172.21.15.196:0/2225131642 shutdown_connections 2024-04-28T13:33:20.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.933+0000 7f4fbf000640 1 -- 172.21.15.196:0/2225131642 wait complete. 2024-04-28T13:33:20.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.935+0000 7f4fbf000640 1 Processor -- start 2024-04-28T13:33:20.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.935+0000 7f4fbf000640 1 -- start start 2024-04-28T13:33:20.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.936+0000 7f4fbf000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fb806e330 0x7f4fb8159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:20.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.936+0000 7f4fbf000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb80d18a0 0x7f4fb813ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:20.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.936+0000 7f4fbf000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fb81454c0 0x7f4fb8073ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:20.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.936+0000 7f4fbf000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb80d0ae0 con 0x7f4fb81454c0 2024-04-28T13:33:20.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.936+0000 7f4fbf000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb806f0d0 con 0x7f4fb80d18a0 2024-04-28T13:33:20.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.936+0000 7f4fbf000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb815d8a0 con 0x7f4fb806e330 2024-04-28T13:33:20.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.936+0000 7f4fbd000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fb81454c0 0x7f4fb8073ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:20.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.936+0000 7f4fbd000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fb81454c0 0x7f4fb8073ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35042/0 (socket says 172.21.15.196:35042) 2024-04-28T13:33:20.938 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.936+0000 7f4fbd000640 1 -- 172.21.15.196:0/4274643671 learned_addr learned my addr 172.21.15.196:0/4274643671 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:20.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.937+0000 7f4fb7e00640 1 --2- 172.21.15.196:0/4274643671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fb806e330 0x7f4fb8159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:20.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.937+0000 7f4fb7400640 1 --2- 172.21.15.196:0/4274643671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb80d18a0 0x7f4fb813ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:20.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.937+0000 7f4fbd000640 1 -- 172.21.15.196:0/4274643671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fb806e330 msgr2=0x7f4fb8159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:20.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.937+0000 7f4fbd000640 1 --2- 172.21.15.196:0/4274643671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fb806e330 0x7f4fb8159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:20.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.937+0000 7f4fbd000640 1 -- 172.21.15.196:0/4274643671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb80d18a0 msgr2=0x7f4fb813ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:20.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.937+0000 7f4fbd000640 1 --2- 172.21.15.196:0/4274643671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb80d18a0 0x7f4fb813ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:20.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.937+0000 7f4fbd000640 1 -- 172.21.15.196:0/4274643671 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb813df50 con 0x7f4fb81454c0 2024-04-28T13:33:20.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.937+0000 7f4fbd000640 1 --2- 172.21.15.196:0/4274643671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fb81454c0 0x7f4fb8073ac0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4fb002cef0 tx=0x7f4fb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:20.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.938+0000 7f4fafe00640 1 -- 172.21.15.196:0/4274643671 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0042070 con 0x7f4fb81454c0 2024-04-28T13:33:20.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.938+0000 7f4fafe00640 1 -- 172.21.15.196:0/4274643671 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb0034be0 con 0x7f4fb81454c0 2024-04-28T13:33:20.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.938+0000 7f4fafe00640 1 -- 172.21.15.196:0/4274643671 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb000b720 con 0x7f4fb81454c0 2024-04-28T13:33:20.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.938+0000 7f4fbf000640 1 -- 172.21.15.196:0/4274643671 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb813e230 con 0x7f4fb81454c0 2024-04-28T13:33:20.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.938+0000 7f4fbf000640 1 -- 172.21.15.196:0/4274643671 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb813e7f0 con 0x7f4fb81454c0 2024-04-28T13:33:20.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.939+0000 7f4fbf000640 1 -- 172.21.15.196:0/4274643671 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb806a290 con 0x7f4fb81454c0 2024-04-28T13:33:20.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.940+0000 7f4fafe00640 1 -- 172.21.15.196:0/4274643671 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4fb003c430 con 0x7f4fb81454c0 2024-04-28T13:33:20.940 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.941+0000 7f4fafe00640 1 --2- 172.21.15.196:0/4274643671 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4f9806c980 0x7f4f9806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:20.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.941+0000 7f4fafe00640 1 -- 172.21.15.196:0/4274643671 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5881+0+0 (secure 0 0 0) 0x7f4fb00acc90 con 0x7f4fb81454c0 2024-04-28T13:33:20.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.942+0000 7f4fb7e00640 1 --2- 172.21.15.196:0/4274643671 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4f9806c980 0x7f4f9806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:20.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.943+0000 7f4fb7e00640 1 --2- 172.21.15.196:0/4274643671 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4f9806c980 0x7f4f9806ee40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800a970 tx=0x7f4fa8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:20.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:20.945+0000 7f4fafe00640 1 -- 172.21.15.196:0/4274643671 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4fb007af20 con 0x7f4fb81454c0 2024-04-28T13:33:21.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:21.145+0000 7f4fbf000640 1 -- 172.21.15.196:0/4274643671 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4fb813e9c0 con 0x7f4fb81454c0 2024-04-28T13:33:21.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:21.854+0000 7f4fafe00640 1 -- 172.21.15.196:0/4274643671 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) ==== 181+0+0 (secure 0 0 0) 0x7f4fb007a820 con 0x7f4fb81454c0 2024-04-28T13:33:21.855 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'cephfs_metadata' removed 2024-04-28T13:33:21.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:21.858+0000 7f4fbf000640 1 -- 172.21.15.196:0/4274643671 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4f9806c980 msgr2=0x7f4f9806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:21.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:21.858+0000 7f4fbf000640 1 --2- 172.21.15.196:0/4274643671 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4f9806c980 0x7f4f9806ee40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800a970 tx=0x7f4fa8008040 comp rx=0 tx=0).stop 2024-04-28T13:33:21.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:21.858+0000 7f4fbf000640 1 -- 172.21.15.196:0/4274643671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fb81454c0 msgr2=0x7f4fb8073ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:21.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:21.858+0000 7f4fbf000640 1 --2- 172.21.15.196:0/4274643671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fb81454c0 0x7f4fb8073ac0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4fb002cef0 tx=0x7f4fb0004a40 comp rx=0 tx=0).stop 2024-04-28T13:33:21.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:21.858+0000 7f4fbf000640 1 -- 172.21.15.196:0/4274643671 shutdown_connections 2024-04-28T13:33:21.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:21.859+0000 7f4fbf000640 1 --2- 172.21.15.196:0/4274643671 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4f9806c980 0x7f4f9806ee40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:21.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:21.859+0000 7f4fbf000640 1 --2- 172.21.15.196:0/4274643671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fb81454c0 0x7f4fb8073ac0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:21.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:21.859+0000 7f4fbf000640 1 --2- 172.21.15.196:0/4274643671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fb80d18a0 0x7f4fb813ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:21.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:21.859+0000 7f4fbf000640 1 --2- 172.21.15.196:0/4274643671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fb806e330 0x7f4fb8159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:21.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:21.859+0000 7f4fbf000640 1 -- 172.21.15.196:0/4274643671 >> 172.21.15.196:0/4274643671 conn(0x7f4fb806c690 msgr2=0x7f4fb8144520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:21.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:21.859+0000 7f4fbf000640 1 -- 172.21.15.196:0/4274643671 shutdown_connections 2024-04-28T13:33:21.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:21.859+0000 7f4fbf000640 1 -- 172.21.15.196:0/4274643671 wait complete. 2024-04-28T13:33:21.886 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-28T13:33:22.081 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.081+0000 7fe4a2000640 1 Processor -- start 2024-04-28T13:33:22.082 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.082+0000 7fe4a2000640 1 -- start start 2024-04-28T13:33:22.083 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.083+0000 7fe4a2000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe49c06e330 0x7fe49c0d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:22.083 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.083+0000 7fe4a2000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe49c0d1ae0 con 0x7fe49c0d16d0 2024-04-28T13:33:22.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.083+0000 7fe4a2000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe49c06e830 con 0x7fe49c1454c0 2024-04-28T13:33:22.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.083+0000 7fe4a2000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe49c0721a0 con 0x7fe49c06e330 2024-04-28T13:33:22.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.083+0000 7fe493e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe49c06e330 0x7fe49c0d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:22.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.084+0000 7fe493e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe49c06e330 0x7fe49c0d1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37278/0 (socket says 172.21.15.196:37278) 2024-04-28T13:33:22.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.084+0000 7fe49be00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fe49c0d16d0 0x7fe49c0cf130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:54064/0 (socket says 172.21.15.196:54064) 2024-04-28T13:33:22.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.084+0000 7fe493e00640 1 -- 172.21.15.196:0/4257454027 learned_addr learned my addr 172.21.15.196:0/4257454027 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:22.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.084+0000 7fe493e00640 1 -- 172.21.15.196:0/4257454027 >> v1:172.21.15.203:6789/0 conn(0x7fe49c1454c0 legacy=0x7fe49c0d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:22.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.084+0000 7fe493e00640 1 -- 172.21.15.196:0/4257454027 >> v1:172.21.15.196:6789/0 conn(0x7fe49c0d16d0 legacy=0x7fe49c0cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:22.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.084+0000 7fe493e00640 1 -- 172.21.15.196:0/4257454027 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c06eb10 con 0x7fe49c06e330 2024-04-28T13:33:22.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.085+0000 7fe493e00640 1 --2- 172.21.15.196:0/4257454027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe49c06e330 0x7fe49c0d1190 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe484009870 tx=0x7fe48402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc685fc91f333f52 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:22.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.085+0000 7fe49aa00640 1 -- 172.21.15.196:0/4257454027 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484037070 con 0x7fe49c06e330 2024-04-28T13:33:22.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.085+0000 7fe49aa00640 1 -- 172.21.15.196:0/4257454027 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe484032440 con 0x7fe49c06e330 2024-04-28T13:33:22.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.086+0000 7fe49aa00640 1 -- 172.21.15.196:0/4257454027 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484032810 con 0x7fe49c06e330 2024-04-28T13:33:22.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.086+0000 7fe4a2000640 1 -- 172.21.15.196:0/4257454027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe49c06e330 msgr2=0x7fe49c0d1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:22.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.086+0000 7fe4a2000640 1 --2- 172.21.15.196:0/4257454027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe49c06e330 0x7fe49c0d1190 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe484009870 tx=0x7fe48402a2a0 comp rx=0 tx=0).stop 2024-04-28T13:33:22.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.086+0000 7fe4a2000640 1 -- 172.21.15.196:0/4257454027 shutdown_connections 2024-04-28T13:33:22.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.086+0000 7fe4a2000640 1 --2- 172.21.15.196:0/4257454027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe49c06e330 0x7fe49c0d1190 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:22.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.086+0000 7fe4a2000640 1 -- 172.21.15.196:0/4257454027 >> 172.21.15.196:0/4257454027 conn(0x7fe49c06c690 msgr2=0x7fe49c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:22.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.086+0000 7fe4a2000640 1 -- 172.21.15.196:0/4257454027 shutdown_connections 2024-04-28T13:33:22.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.086+0000 7fe4a2000640 1 -- 172.21.15.196:0/4257454027 wait complete. 2024-04-28T13:33:22.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.088+0000 7fe4a2000640 1 Processor -- start 2024-04-28T13:33:22.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.088+0000 7fe4a2000640 1 -- start start 2024-04-28T13:33:22.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.089+0000 7fe4a2000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe49c06e330 0x7fe49c150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:22.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.089+0000 7fe4a2000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe49c0d16d0 0x7fe49c0c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:22.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.089+0000 7fe4a2000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe49c1454c0 0x7fe49c1e8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:22.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.089+0000 7fe4a2000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c06f0d0 con 0x7fe49c06e330 2024-04-28T13:33:22.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.089+0000 7fe4a2000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c0ceca0 con 0x7fe49c1454c0 2024-04-28T13:33:22.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.089+0000 7fe4a2000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe49c15d8a0 con 0x7fe49c0d16d0 2024-04-28T13:33:22.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.089+0000 7fe49b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe49c06e330 0x7fe49c150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:22.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.089+0000 7fe49b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe49c06e330 0x7fe49c150810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37282/0 (socket says 172.21.15.196:37282) 2024-04-28T13:33:22.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.089+0000 7fe49b400640 1 -- 172.21.15.196:0/3676903338 learned_addr learned my addr 172.21.15.196:0/3676903338 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:22.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.090+0000 7fe49be00640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe49c1454c0 0x7fe49c1e8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:22.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.090+0000 7fe493e00640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe49c0d16d0 0x7fe49c0c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:22.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.090+0000 7fe49b400640 1 -- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe49c0d16d0 msgr2=0x7fe49c0c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:22.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.090+0000 7fe49b400640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe49c0d16d0 0x7fe49c0c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:22.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.090+0000 7fe49b400640 1 -- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe49c1454c0 msgr2=0x7fe49c1e8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:22.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.090+0000 7fe49b400640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe49c1454c0 0x7fe49c1e8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:22.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.090+0000 7fe49b400640 1 -- 172.21.15.196:0/3676903338 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c13ebc0 con 0x7fe49c06e330 2024-04-28T13:33:22.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.090+0000 7fe493e00640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe49c0d16d0 0x7fe49c0c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:22.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.090+0000 7fe49be00640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe49c1454c0 0x7fe49c1e8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:22.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.090+0000 7fe49b400640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe49c06e330 0x7fe49c150810 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c430 tx=0x7fe48c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:22.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.090+0000 7fe493400640 1 -- 172.21.15.196:0/3676903338 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c011790 con 0x7fe49c06e330 2024-04-28T13:33:22.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.090+0000 7fe493400640 1 -- 172.21.15.196:0/3676903338 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe48c017070 con 0x7fe49c06e330 2024-04-28T13:33:22.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.091+0000 7fe493400640 1 -- 172.21.15.196:0/3676903338 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c011bb0 con 0x7fe49c06e330 2024-04-28T13:33:22.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.091+0000 7fe4a2000640 1 -- 172.21.15.196:0/3676903338 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49c142010 con 0x7fe49c06e330 2024-04-28T13:33:22.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.091+0000 7fe4a2000640 1 -- 172.21.15.196:0/3676903338 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe49c150d50 con 0x7fe49c06e330 2024-04-28T13:33:22.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.092+0000 7fe4a2000640 1 -- 172.21.15.196:0/3676903338 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49c06a290 con 0x7fe49c06e330 2024-04-28T13:33:22.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.093+0000 7fe493400640 1 -- 172.21.15.196:0/3676903338 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe48c01c460 con 0x7fe49c06e330 2024-04-28T13:33:22.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.094+0000 7fe493400640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe47806c980 0x7fe47806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:22.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.094+0000 7fe493400640 1 -- 172.21.15.196:0/3676903338 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5443+0+0 (secure 0 0 0) 0x7fe48c08b9a0 con 0x7fe49c06e330 2024-04-28T13:33:22.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.094+0000 7fe493e00640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe47806c980 0x7fe47806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:22.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.096+0000 7fe493e00640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe47806c980 0x7fe47806ee40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe484034000 tx=0x7fe48402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:22.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.098+0000 7fe493400640 1 -- 172.21.15.196:0/3676903338 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe48c059e70 con 0x7fe49c06e330 2024-04-28T13:33:22.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.299+0000 7fe4a2000640 1 -- 172.21.15.196:0/3676903338 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe49c0cdf00 con 0x7fe49c06e330 2024-04-28T13:33:22.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.858+0000 7fe493400640 1 -- 172.21.15.196:0/3676903338 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) ==== 169+0+0 (secure 0 0 0) 0x7fe48c059770 con 0x7fe49c06e330 2024-04-28T13:33:22.859 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'cephfs_data' removed 2024-04-28T13:33:22.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.862+0000 7fe4a2000640 1 -- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe47806c980 msgr2=0x7fe47806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:22.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.862+0000 7fe4a2000640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe47806c980 0x7fe47806ee40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe484034000 tx=0x7fe48402e000 comp rx=0 tx=0).stop 2024-04-28T13:33:22.862 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.862+0000 7fe4a2000640 1 -- 172.21.15.196:0/3676903338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe49c06e330 msgr2=0x7fe49c150810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:22.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.862+0000 7fe4a2000640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe49c06e330 0x7fe49c150810 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c430 tx=0x7fe48c00c8f0 comp rx=0 tx=0).stop 2024-04-28T13:33:22.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.862+0000 7fe4a2000640 1 -- 172.21.15.196:0/3676903338 shutdown_connections 2024-04-28T13:33:22.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.862+0000 7fe4a2000640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe47806c980 0x7fe47806ee40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:22.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.862+0000 7fe4a2000640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe49c1454c0 0x7fe49c1e8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:22.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.863+0000 7fe4a2000640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe49c0d16d0 0x7fe49c0c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:22.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.863+0000 7fe4a2000640 1 --2- 172.21.15.196:0/3676903338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe49c06e330 0x7fe49c150810 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:22.863 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.863+0000 7fe4a2000640 1 -- 172.21.15.196:0/3676903338 >> 172.21.15.196:0/3676903338 conn(0x7fe49c06c690 msgr2=0x7fe49c0b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:22.864 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.864+0000 7fe4a2000640 1 -- 172.21.15.196:0/3676903338 shutdown_connections 2024-04-28T13:33:22.865 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:22.865+0000 7fe4a2000640 1 -- 172.21.15.196:0/3676903338 wait complete. 2024-04-28T13:33:22.892 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds debug_mds 2024-04-28T13:33:23.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.087+0000 7fec48e00640 1 Processor -- start 2024-04-28T13:33:23.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.087+0000 7fec48e00640 1 -- start start 2024-04-28T13:33:23.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.089+0000 7fec48e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44162970 0x7fec440d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.089+0000 7fec48e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fec440d5ad0 0x7fec440d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.089+0000 7fec48e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec440b1c50 con 0x7fec440d3b00 2024-04-28T13:33:23.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.089+0000 7fec48e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fec440b2230 con 0x7fec440d5ad0 2024-04-28T13:33:23.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.089+0000 7fec48e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fec440d30c0 con 0x7fec44162970 2024-04-28T13:33:23.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.089+0000 7fec43e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fec440d3b00 0x7fec440cfb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:54074/0 (socket says 172.21.15.196:54074) 2024-04-28T13:33:23.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.089+0000 7fec43e00640 1 -- 172.21.15.196:0/226313857 learned_addr learned my addr 172.21.15.196:0/226313857 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:23.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.089+0000 7fec3be00640 1 --2- 172.21.15.196:0/226313857 >> v2:172.21.15.203:3300/0 conn(0x7fec440d5ad0 0x7fec440d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec43400640 1 --2- 172.21.15.196:0/226313857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44162970 0x7fec440d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 -- 172.21.15.196:0/226313857 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4164482824 0 0) 0x7fec440b1c50 con 0x7fec440d3b00 2024-04-28T13:33:23.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 -- 172.21.15.196:0/226313857 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec28003520 con 0x7fec440d3b00 2024-04-28T13:33:23.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 -- 172.21.15.196:0/226313857 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2478188075 0 0) 0x7fec28003520 con 0x7fec440d3b00 2024-04-28T13:33:23.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 -- 172.21.15.196:0/226313857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44162970 msgr2=0x7fec440d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:23.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 --2- 172.21.15.196:0/226313857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44162970 0x7fec440d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 -- 172.21.15.196:0/226313857 >> v2:172.21.15.203:3300/0 conn(0x7fec440d5ad0 msgr2=0x7fec440d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:23.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 --2- 172.21.15.196:0/226313857 >> v2:172.21.15.203:3300/0 conn(0x7fec440d5ad0 0x7fec440d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 -- 172.21.15.196:0/226313857 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec440d2600 con 0x7fec440d3b00 2024-04-28T13:33:23.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 -- 172.21.15.196:0/226313857 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fec3c0019f0 con 0x7fec440d3b00 2024-04-28T13:33:23.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 -- 172.21.15.196:0/226313857 >> v1:172.21.15.196:6789/0 conn(0x7fec440d3b00 legacy=0x7fec440cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:23.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 --2- 172.21.15.196:0/226313857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44162970 0x7fec280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 --2- 172.21.15.196:0/226313857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec28003f80 0x7fec28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 --2- 172.21.15.196:0/226313857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec28004d40 0x7fec280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 -- 172.21.15.196:0/226313857 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec440b2230 con 0x7fec28003f80 2024-04-28T13:33:23.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 -- 172.21.15.196:0/226313857 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fec440d30c0 con 0x7fec28004d40 2024-04-28T13:33:23.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec42a00640 1 -- 172.21.15.196:0/226313857 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fec2800ba70 con 0x7fec44162970 2024-04-28T13:33:23.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.090+0000 7fec3be00640 1 --2- 172.21.15.196:0/226313857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec28003f80 0x7fec28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.091+0000 7fec43400640 1 --2- 172.21.15.196:0/226313857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44162970 0x7fec280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.091+0000 7fec43e00640 1 --2- 172.21.15.196:0/226313857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec28004d40 0x7fec280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.091+0000 7fec3be00640 1 -- 172.21.15.196:0/226313857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44162970 msgr2=0x7fec280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:23.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.091+0000 7fec3be00640 1 --2- 172.21.15.196:0/226313857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44162970 0x7fec280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.091+0000 7fec3be00640 1 -- 172.21.15.196:0/226313857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec28004d40 msgr2=0x7fec280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:23.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.091+0000 7fec3be00640 1 --2- 172.21.15.196:0/226313857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec28004d40 0x7fec280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.091+0000 7fec3be00640 1 -- 172.21.15.196:0/226313857 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec28003520 con 0x7fec28003f80 2024-04-28T13:33:23.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.091+0000 7fec43400640 1 --2- 172.21.15.196:0/226313857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44162970 0x7fec280039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:23.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.091+0000 7fec43e00640 1 --2- 172.21.15.196:0/226313857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec28004d40 0x7fec280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:23.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.092+0000 7fec3be00640 1 --2- 172.21.15.196:0/226313857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec28003f80 0x7fec28004800 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009ac0 tx=0x7fec28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef921807028d211 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:23.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.092+0000 7fec42a00640 1 -- 172.21.15.196:0/226313857 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c036070 con 0x7fec28003f80 2024-04-28T13:33:23.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.092+0000 7fec42a00640 1 -- 172.21.15.196:0/226313857 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec2c031400 con 0x7fec28003f80 2024-04-28T13:33:23.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.092+0000 7fec42a00640 1 -- 172.21.15.196:0/226313857 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c009690 con 0x7fec28003f80 2024-04-28T13:33:23.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.093+0000 7fec48e00640 1 -- 172.21.15.196:0/226313857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec28003f80 msgr2=0x7fec28004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:23.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.093+0000 7fec48e00640 1 --2- 172.21.15.196:0/226313857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec28003f80 0x7fec28004800 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009ac0 tx=0x7fec28005830 comp rx=0 tx=0).stop 2024-04-28T13:33:23.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.093+0000 7fec43e00640 1 -- 172.21.15.196:0/226313857 reap_dead start 2024-04-28T13:33:23.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.093+0000 7fec48e00640 1 -- 172.21.15.196:0/226313857 shutdown_connections 2024-04-28T13:33:23.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.093+0000 7fec48e00640 1 -- 172.21.15.196:0/226313857 >> 172.21.15.196:0/226313857 conn(0x7fec440d4fe0 msgr2=0x7fec440d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:23.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.093+0000 7fec48e00640 1 -- 172.21.15.196:0/226313857 shutdown_connections 2024-04-28T13:33:23.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.093+0000 7fec48e00640 1 -- 172.21.15.196:0/226313857 wait complete. 2024-04-28T13:33:23.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.095+0000 7fec48e00640 1 Processor -- start 2024-04-28T13:33:23.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.095+0000 7fec48e00640 1 -- start start 2024-04-28T13:33:23.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.096+0000 7fec48e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec28003f80 0x7fec440cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.096+0000 7fec48e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec440b90e0 0x7fec440cead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.096+0000 7fec48e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec440b9a10 0x7fec440b9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.096+0000 7fec48e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec44162d50 con 0x7fec28003f80 2024-04-28T13:33:23.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.096+0000 7fec48e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fec440d28e0 con 0x7fec440b90e0 2024-04-28T13:33:23.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.096+0000 7fec48e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fec440cd9e0 con 0x7fec440b9a10 2024-04-28T13:33:23.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.096+0000 7fec43400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec28003f80 0x7fec440cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.096+0000 7fec43400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec28003f80 0x7fec440cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37298/0 (socket says 172.21.15.196:37298) 2024-04-28T13:33:23.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.096+0000 7fec43400640 1 -- 172.21.15.196:0/2271305549 learned_addr learned my addr 172.21.15.196:0/2271305549 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:23.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.096+0000 7fec43400640 1 -- 172.21.15.196:0/2271305549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec440b9a10 msgr2=0x7fec440b9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:23.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.096+0000 7fec43400640 1 --2- 172.21.15.196:0/2271305549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec440b9a10 0x7fec440b9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.096+0000 7fec3be00640 1 --2- 172.21.15.196:0/2271305549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec440b90e0 0x7fec440cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.096+0000 7fec43400640 1 -- 172.21.15.196:0/2271305549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec440b90e0 msgr2=0x7fec440cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:23.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.096+0000 7fec43400640 1 --2- 172.21.15.196:0/2271305549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec440b90e0 0x7fec440cead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.096+0000 7fec43400640 1 -- 172.21.15.196:0/2271305549 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec440cc400 con 0x7fec28003f80 2024-04-28T13:33:23.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.097+0000 7fec43400640 1 --2- 172.21.15.196:0/2271305549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec28003f80 0x7fec440cd4a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fec3400e810 tx=0x7fec3400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:23.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.097+0000 7fec3b400640 1 -- 172.21.15.196:0/2271305549 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34017070 con 0x7fec28003f80 2024-04-28T13:33:23.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.097+0000 7fec3b400640 1 -- 172.21.15.196:0/2271305549 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3400c6d0 con 0x7fec28003f80 2024-04-28T13:33:23.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.097+0000 7fec48e00640 1 -- 172.21.15.196:0/2271305549 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec440cc6e0 con 0x7fec28003f80 2024-04-28T13:33:23.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.097+0000 7fec3b400640 1 -- 172.21.15.196:0/2271305549 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3400f040 con 0x7fec28003f80 2024-04-28T13:33:23.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.097+0000 7fec48e00640 1 -- 172.21.15.196:0/2271305549 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec440ccca0 con 0x7fec28003f80 2024-04-28T13:33:23.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.099+0000 7fec48e00640 1 -- 172.21.15.196:0/2271305549 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec44050730 con 0x7fec28003f80 2024-04-28T13:33:23.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.099+0000 7fec3b400640 1 -- 172.21.15.196:0/2271305549 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fec3401a070 con 0x7fec28003f80 2024-04-28T13:33:23.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.101+0000 7fec3b400640 1 --2- 172.21.15.196:0/2271305549 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fec2006c980 0x7fec2006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.101+0000 7fec3b400640 1 -- 172.21.15.196:0/2271305549 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5029+0+0 (secure 0 0 0) 0x7fec3408b480 con 0x7fec28003f80 2024-04-28T13:33:23.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.101+0000 7fec3be00640 1 --2- 172.21.15.196:0/2271305549 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fec2006c980 0x7fec2006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.103+0000 7fec3be00640 1 --2- 172.21.15.196:0/2271305549 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fec2006c980 0x7fec2006ee40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fec2c033000 tx=0x7fec2c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:23.105 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.105+0000 7fec3b400640 1 -- 172.21.15.196:0/2271305549 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec34059ae0 con 0x7fec28003f80 2024-04-28T13:33:23.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.311+0000 7fec48e00640 1 -- 172.21.15.196:0/2271305549 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "debug_mds"} v 0) -- 0x7fec440b5f80 con 0x7fec28003f80 2024-04-28T13:33:23.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.317+0000 7fec3b400640 1 -- 172.21.15.196:0/2271305549 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "debug_mds"}]=0 v13) ==== 92+0+0 (secure 0 0 0) 0x7fec340593e0 con 0x7fec28003f80 2024-04-28T13:33:23.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.320+0000 7fec48e00640 1 -- 172.21.15.196:0/2271305549 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fec2006c980 msgr2=0x7fec2006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:23.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.320+0000 7fec48e00640 1 --2- 172.21.15.196:0/2271305549 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fec2006c980 0x7fec2006ee40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fec2c033000 tx=0x7fec2c008030 comp rx=0 tx=0).stop 2024-04-28T13:33:23.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.320+0000 7fec48e00640 1 -- 172.21.15.196:0/2271305549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec28003f80 msgr2=0x7fec440cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:23.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.320+0000 7fec48e00640 1 --2- 172.21.15.196:0/2271305549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec28003f80 0x7fec440cd4a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fec3400e810 tx=0x7fec3400ecd0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.320+0000 7fec48e00640 1 -- 172.21.15.196:0/2271305549 shutdown_connections 2024-04-28T13:33:23.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.320+0000 7fec48e00640 1 --2- 172.21.15.196:0/2271305549 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fec2006c980 0x7fec2006ee40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.320+0000 7fec48e00640 1 --2- 172.21.15.196:0/2271305549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec440b9a10 0x7fec440b9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.320+0000 7fec48e00640 1 --2- 172.21.15.196:0/2271305549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec440b90e0 0x7fec440cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.320+0000 7fec48e00640 1 --2- 172.21.15.196:0/2271305549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec28003f80 0x7fec440cd4a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.320+0000 7fec48e00640 1 -- 172.21.15.196:0/2271305549 >> 172.21.15.196:0/2271305549 conn(0x7fec440d4fe0 msgr2=0x7fec44161d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:23.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.320+0000 7fec48e00640 1 -- 172.21.15.196:0/2271305549 shutdown_connections 2024-04-28T13:33:23.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.320+0000 7fec48e00640 1 -- 172.21.15.196:0/2271305549 wait complete. 2024-04-28T13:33:23.339 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_quiesce_splitauth 2024-04-28T13:33:23.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.543+0000 7fac48800640 1 Processor -- start 2024-04-28T13:33:23.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.544+0000 7fac48800640 1 -- start start 2024-04-28T13:33:23.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.545+0000 7fac48800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fac400b5330 0x7fac400cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.545+0000 7fac48800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac400b57c0 con 0x7fac400b4eb0 2024-04-28T13:33:23.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.545+0000 7fac48800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac4015dca0 con 0x7fac40143840 2024-04-28T13:33:23.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.545+0000 7fac48800640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fac40143c20 con 0x7fac400b5330 2024-04-28T13:33:23.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.546+0000 7fac45400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fac400b4eb0 0x7fac400ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:54080/0 (socket says 172.21.15.196:54080) 2024-04-28T13:33:23.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.546+0000 7fac45400640 1 -- 172.21.15.196:0/415479386 learned_addr learned my addr 172.21.15.196:0/415479386 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:23.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.546+0000 7fac46800640 1 --2- 172.21.15.196:0/415479386 >> v2:172.21.15.203:3300/0 conn(0x7fac400b5330 0x7fac400cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.546+0000 7fac44a00640 1 -- 172.21.15.196:0/415479386 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1832947613 0 0) 0x7fac400b57c0 con 0x7fac400b4eb0 2024-04-28T13:33:23.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.546+0000 7fac44a00640 1 -- 172.21.15.196:0/415479386 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac28003520 con 0x7fac400b4eb0 2024-04-28T13:33:23.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.547+0000 7fac44a00640 1 -- 172.21.15.196:0/415479386 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2579212241 0 0) 0x7fac28003520 con 0x7fac400b4eb0 2024-04-28T13:33:23.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.547+0000 7fac44a00640 1 -- 172.21.15.196:0/415479386 >> v2:172.21.15.203:3300/0 conn(0x7fac400b5330 msgr2=0x7fac400cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:23.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.547+0000 7fac44a00640 1 --2- 172.21.15.196:0/415479386 >> v2:172.21.15.203:3300/0 conn(0x7fac400b5330 0x7fac400cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.547+0000 7fac44a00640 1 -- 172.21.15.196:0/415479386 >> v1:172.21.15.203:6789/0 conn(0x7fac40143840 legacy=0x7fac400b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:23.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.547+0000 7fac44a00640 1 -- 172.21.15.196:0/415479386 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4015df80 con 0x7fac400b4eb0 2024-04-28T13:33:23.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.547+0000 7fac44a00640 1 -- 172.21.15.196:0/415479386 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fac30001670 con 0x7fac400b4eb0 2024-04-28T13:33:23.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.547+0000 7fac44a00640 1 -- 172.21.15.196:0/415479386 >> v1:172.21.15.196:6789/0 conn(0x7fac400b4eb0 legacy=0x7fac400ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:23.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.547+0000 7fac44a00640 1 --2- 172.21.15.196:0/415479386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac280039b0 0x7fac28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.547+0000 7fac44a00640 1 --2- 172.21.15.196:0/415479386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac28004800 0x7fac28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.547+0000 7fac44a00640 1 --2- 172.21.15.196:0/415479386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac280051c0 0x7fac2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.547+0000 7fac44a00640 1 -- 172.21.15.196:0/415479386 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac40143c20 con 0x7fac280051c0 2024-04-28T13:33:23.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.547+0000 7fac44a00640 1 -- 172.21.15.196:0/415479386 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fac2800c060 con 0x7fac28004800 2024-04-28T13:33:23.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.547+0000 7fac44a00640 1 -- 172.21.15.196:0/415479386 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fac2800c260 con 0x7fac280039b0 2024-04-28T13:33:23.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.547+0000 7fac46800640 1 --2- 172.21.15.196:0/415479386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac280051c0 0x7fac2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.548+0000 7fac45400640 1 --2- 172.21.15.196:0/415479386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac28004800 0x7fac28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.548+0000 7fac45e00640 1 --2- 172.21.15.196:0/415479386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac280039b0 0x7fac28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.548+0000 7fac46800640 1 -- 172.21.15.196:0/415479386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac280039b0 msgr2=0x7fac28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:23.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.548+0000 7fac46800640 1 --2- 172.21.15.196:0/415479386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac280039b0 0x7fac28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.548+0000 7fac46800640 1 -- 172.21.15.196:0/415479386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac28004800 msgr2=0x7fac28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:23.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.548+0000 7fac46800640 1 --2- 172.21.15.196:0/415479386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac28004800 0x7fac28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.548+0000 7fac46800640 1 -- 172.21.15.196:0/415479386 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac28003520 con 0x7fac280051c0 2024-04-28T13:33:23.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.549+0000 7fac46800640 1 -- 172.21.15.196:0/415479386 reap_dead start 2024-04-28T13:33:23.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.549+0000 7fac46800640 1 --2- 172.21.15.196:0/415479386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac280051c0 0x7fac2800b9c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009fc0 tx=0x7fac28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485139e4f81c3bf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:23.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.549+0000 7fac44a00640 1 -- 172.21.15.196:0/415479386 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c035020 con 0x7fac280051c0 2024-04-28T13:33:23.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.549+0000 7fac44a00640 1 -- 172.21.15.196:0/415479386 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac3c003bd0 con 0x7fac280051c0 2024-04-28T13:33:23.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.549+0000 7fac44a00640 1 -- 172.21.15.196:0/415479386 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c009c10 con 0x7fac280051c0 2024-04-28T13:33:23.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.550+0000 7fac48800640 1 -- 172.21.15.196:0/415479386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac280051c0 msgr2=0x7fac2800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:23.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.550+0000 7fac48800640 1 --2- 172.21.15.196:0/415479386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac280051c0 0x7fac2800b9c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009fc0 tx=0x7fac28005830 comp rx=0 tx=0).stop 2024-04-28T13:33:23.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.550+0000 7fac48800640 1 -- 172.21.15.196:0/415479386 shutdown_connections 2024-04-28T13:33:23.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.550+0000 7fac48800640 1 --2- 172.21.15.196:0/415479386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac280051c0 0x7fac2800b9c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.550+0000 7fac48800640 1 -- 172.21.15.196:0/415479386 >> 172.21.15.196:0/415479386 conn(0x7fac400cd8a0 msgr2=0x7fac400cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:23.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.550+0000 7fac48800640 1 -- 172.21.15.196:0/415479386 shutdown_connections 2024-04-28T13:33:23.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.550+0000 7fac48800640 1 -- 172.21.15.196:0/415479386 wait complete. 2024-04-28T13:33:23.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.552+0000 7fac48800640 1 Processor -- start 2024-04-28T13:33:23.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.552+0000 7fac48800640 1 -- start start 2024-04-28T13:33:23.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.553+0000 7fac48800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac280051c0 0x7fac4013edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.553+0000 7fac48800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac40143fd0 0x7fac40144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.553+0000 7fac48800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac40146180 0x7fac40146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.553+0000 7fac48800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac4015d8c0 con 0x7fac40146180 2024-04-28T13:33:23.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.553+0000 7fac48800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fac4015e540 con 0x7fac40143fd0 2024-04-28T13:33:23.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.553+0000 7fac48800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fac4015da40 con 0x7fac280051c0 2024-04-28T13:33:23.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.553+0000 7fac46800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac40146180 0x7fac40146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.553+0000 7fac46800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac40146180 0x7fac40146610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37322/0 (socket says 172.21.15.196:37322) 2024-04-28T13:33:23.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.553+0000 7fac46800640 1 -- 172.21.15.196:0/1697140544 learned_addr learned my addr 172.21.15.196:0/1697140544 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:23.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.553+0000 7fac45e00640 1 --2- 172.21.15.196:0/1697140544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac280051c0 0x7fac4013edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.553+0000 7fac45400640 1 --2- 172.21.15.196:0/1697140544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac40143fd0 0x7fac40144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.553+0000 7fac46800640 1 -- 172.21.15.196:0/1697140544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac280051c0 msgr2=0x7fac4013edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:23.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.553+0000 7fac46800640 1 --2- 172.21.15.196:0/1697140544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac280051c0 0x7fac4013edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.553+0000 7fac46800640 1 -- 172.21.15.196:0/1697140544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac40143fd0 msgr2=0x7fac40144430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:23.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.553+0000 7fac46800640 1 --2- 172.21.15.196:0/1697140544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac40143fd0 0x7fac40144430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.554+0000 7fac46800640 1 -- 172.21.15.196:0/1697140544 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac400ca9d0 con 0x7fac40146180 2024-04-28T13:33:23.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.554+0000 7fac45e00640 1 --2- 172.21.15.196:0/1697140544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac280051c0 0x7fac4013edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:23.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.554+0000 7fac46800640 1 --2- 172.21.15.196:0/1697140544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac40146180 0x7fac40146610 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fac3c02af10 tx=0x7fac3c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:23.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.554+0000 7fac3a000640 1 -- 172.21.15.196:0/1697140544 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c035030 con 0x7fac40146180 2024-04-28T13:33:23.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.554+0000 7fac3a000640 1 -- 172.21.15.196:0/1697140544 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac3c043070 con 0x7fac40146180 2024-04-28T13:33:23.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.554+0000 7fac3a000640 1 -- 172.21.15.196:0/1697140544 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c02d070 con 0x7fac40146180 2024-04-28T13:33:23.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.554+0000 7fac48800640 1 -- 172.21.15.196:0/1697140544 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac400cacb0 con 0x7fac40146180 2024-04-28T13:33:23.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.554+0000 7fac48800640 1 -- 172.21.15.196:0/1697140544 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac400cb270 con 0x7fac40146180 2024-04-28T13:33:23.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.556+0000 7fac48800640 1 -- 172.21.15.196:0/1697140544 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac4006a290 con 0x7fac40146180 2024-04-28T13:33:23.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.557+0000 7fac3a000640 1 -- 172.21.15.196:0/1697140544 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fac3c005d10 con 0x7fac40146180 2024-04-28T13:33:23.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.557+0000 7fac3a000640 1 --2- 172.21.15.196:0/1697140544 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fac2406c980 0x7fac2406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:23.557 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.558+0000 7fac3a000640 1 -- 172.21.15.196:0/1697140544 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5029+0+0 (secure 0 0 0) 0x7fac3c0ac8e0 con 0x7fac40146180 2024-04-28T13:33:23.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.558+0000 7fac45e00640 1 --2- 172.21.15.196:0/1697140544 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fac2406c980 0x7fac2406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:23.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.559+0000 7fac45e00640 1 --2- 172.21.15.196:0/1697140544 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fac2406c980 0x7fac2406ee40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fac34000c00 tx=0x7fac34009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:23.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.562+0000 7fac3a000640 1 -- 172.21.15.196:0/1697140544 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fac3c07ad20 con 0x7fac40146180 2024-04-28T13:33:23.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.769+0000 7fac48800640 1 -- 172.21.15.196:0/1697140544 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_quiesce_splitauth"} v 0) -- 0x7fac400cb440 con 0x7fac40146180 2024-04-28T13:33:23.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.775+0000 7fac3a000640 1 -- 172.21.15.196:0/1697140544 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_quiesce_splitauth"}]=0 v14) ==== 110+0+0 (secure 0 0 0) 0x7fac3c07a620 con 0x7fac40146180 2024-04-28T13:33:23.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.778+0000 7fac48800640 1 -- 172.21.15.196:0/1697140544 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fac2406c980 msgr2=0x7fac2406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:23.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.778+0000 7fac48800640 1 --2- 172.21.15.196:0/1697140544 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fac2406c980 0x7fac2406ee40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fac34000c00 tx=0x7fac34009040 comp rx=0 tx=0).stop 2024-04-28T13:33:23.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.778+0000 7fac48800640 1 -- 172.21.15.196:0/1697140544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac40146180 msgr2=0x7fac40146610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:23.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.778+0000 7fac48800640 1 --2- 172.21.15.196:0/1697140544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac40146180 0x7fac40146610 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fac3c02af10 tx=0x7fac3c009750 comp rx=0 tx=0).stop 2024-04-28T13:33:23.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.778+0000 7fac48800640 1 -- 172.21.15.196:0/1697140544 shutdown_connections 2024-04-28T13:33:23.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.778+0000 7fac48800640 1 --2- 172.21.15.196:0/1697140544 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fac2406c980 0x7fac2406ee40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.778+0000 7fac48800640 1 --2- 172.21.15.196:0/1697140544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac40146180 0x7fac40146610 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.778+0000 7fac48800640 1 --2- 172.21.15.196:0/1697140544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac40143fd0 0x7fac40144430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.778+0000 7fac48800640 1 --2- 172.21.15.196:0/1697140544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac280051c0 0x7fac4013edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:23.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.778+0000 7fac48800640 1 -- 172.21.15.196:0/1697140544 >> 172.21.15.196:0/1697140544 conn(0x7fac400cd8a0 msgr2=0x7fac400ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:23.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.778+0000 7fac48800640 1 -- 172.21.15.196:0/1697140544 shutdown_connections 2024-04-28T13:33:23.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:23.779+0000 7fac48800640 1 -- 172.21.15.196:0/1697140544 wait complete. 2024-04-28T13:33:23.798 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-28T13:33:24.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.005+0000 7fd75f000640 1 Processor -- start 2024-04-28T13:33:24.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.005+0000 7fd75f000640 1 -- start start 2024-04-28T13:33:24.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.007+0000 7fd75f000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd7580b93e0 0x7fd7580d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:24.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.007+0000 7fd75f000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd7580d2600 0x7fd7580d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:24.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.007+0000 7fd75f000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7580d0710 con 0x7fd7581628c0 2024-04-28T13:33:24.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.007+0000 7fd75f000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd7580b7f40 con 0x7fd7580d2600 2024-04-28T13:33:24.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.007+0000 7fd75f000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd7580d09f0 con 0x7fd7580b93e0 2024-04-28T13:33:24.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.007+0000 7fd75dc00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fd7581628c0 0x7fd7580d19b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:54088/0 (socket says 172.21.15.196:54088) 2024-04-28T13:33:24.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.007+0000 7fd75dc00640 1 -- 172.21.15.196:0/326946917 learned_addr learned my addr 172.21.15.196:0/326946917 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:24.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.007+0000 7fd75e600640 1 --2- 172.21.15.196:0/326946917 >> v2:172.21.15.196:3300/0 conn(0x7fd7580d2600 0x7fd7580d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:24.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.008+0000 7fd75d200640 1 --2- 172.21.15.196:0/326946917 >> v2:172.21.15.203:3300/0 conn(0x7fd7580b93e0 0x7fd7580d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:24.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.008+0000 7fd757e00640 1 -- 172.21.15.196:0/326946917 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1251149134 0 0) 0x7fd7580d0710 con 0x7fd7581628c0 2024-04-28T13:33:24.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.008+0000 7fd757e00640 1 -- 172.21.15.196:0/326946917 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd740003520 con 0x7fd7581628c0 2024-04-28T13:33:24.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.008+0000 7fd75e600640 1 -- 172.21.15.196:0/326946917 >> v2:172.21.15.203:3300/0 conn(0x7fd7580b93e0 msgr2=0x7fd7580d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:24.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.008+0000 7fd75e600640 1 --2- 172.21.15.196:0/326946917 >> v2:172.21.15.203:3300/0 conn(0x7fd7580b93e0 0x7fd7580d20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.008+0000 7fd75e600640 1 -- 172.21.15.196:0/326946917 >> v1:172.21.15.196:6789/0 conn(0x7fd7581628c0 legacy=0x7fd7580d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:24.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.008+0000 7fd75e600640 1 -- 172.21.15.196:0/326946917 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7580d0cd0 con 0x7fd7580d2600 2024-04-28T13:33:24.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.009+0000 7fd75e600640 1 --2- 172.21.15.196:0/326946917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7580d2600 0x7fd7580d2b90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd7580cf5c0 tx=0x7fd75002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c043951aa8d8fedd server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:24.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.009+0000 7fd757e00640 1 -- 172.21.15.196:0/326946917 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750007740 con 0x7fd7580d2600 2024-04-28T13:33:24.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.009+0000 7fd757e00640 1 -- 172.21.15.196:0/326946917 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7500078e0 con 0x7fd7580d2600 2024-04-28T13:33:24.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.009+0000 7fd757e00640 1 -- 172.21.15.196:0/326946917 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750007cb0 con 0x7fd7580d2600 2024-04-28T13:33:24.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.010+0000 7fd75f000640 1 -- 172.21.15.196:0/326946917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7580d2600 msgr2=0x7fd7580d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:24.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.010+0000 7fd75f000640 1 --2- 172.21.15.196:0/326946917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7580d2600 0x7fd7580d2b90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd7580cf5c0 tx=0x7fd75002a680 comp rx=0 tx=0).stop 2024-04-28T13:33:24.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.010+0000 7fd75f000640 1 -- 172.21.15.196:0/326946917 shutdown_connections 2024-04-28T13:33:24.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.010+0000 7fd75f000640 1 --2- 172.21.15.196:0/326946917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7580d2600 0x7fd7580d2b90 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.010+0000 7fd75f000640 1 --2- 172.21.15.196:0/326946917 >> v2:172.21.15.203:3300/0 conn(0x7fd7580b93e0 0x7fd7580d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.010+0000 7fd75f000640 1 -- 172.21.15.196:0/326946917 >> 172.21.15.196:0/326946917 conn(0x7fd7580d4fc0 msgr2=0x7fd7580d53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:24.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.010+0000 7fd75f000640 1 -- 172.21.15.196:0/326946917 shutdown_connections 2024-04-28T13:33:24.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.010+0000 7fd75f000640 1 -- 172.21.15.196:0/326946917 wait complete. 2024-04-28T13:33:24.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.012+0000 7fd75f000640 1 Processor -- start 2024-04-28T13:33:24.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.012+0000 7fd75f000640 1 -- start start 2024-04-28T13:33:24.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.013+0000 7fd75f000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7580b93e0 0x7fd7581552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:24.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.013+0000 7fd75f000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7580d2600 0x7fd7580ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:24.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.013+0000 7fd75f000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7581628c0 0x7fd758160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:24.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.013+0000 7fd75f000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd758162e20 con 0x7fd7581628c0 2024-04-28T13:33:24.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.013+0000 7fd75f000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd758162ca0 con 0x7fd7580d2600 2024-04-28T13:33:24.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.013+0000 7fd75f000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd7580d30d0 con 0x7fd7580b93e0 2024-04-28T13:33:24.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.013+0000 7fd75e600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7581628c0 0x7fd758160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:24.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.013+0000 7fd75e600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7581628c0 0x7fd758160bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37342/0 (socket says 172.21.15.196:37342) 2024-04-28T13:33:24.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.013+0000 7fd75e600640 1 -- 172.21.15.196:0/57608090 learned_addr learned my addr 172.21.15.196:0/57608090 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:24.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.013+0000 7fd75e600640 1 -- 172.21.15.196:0/57608090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7580b93e0 msgr2=0x7fd7581552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:24.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.013+0000 7fd75e600640 1 --2- 172.21.15.196:0/57608090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7580b93e0 0x7fd7581552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.013+0000 7fd75e600640 1 -- 172.21.15.196:0/57608090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7580d2600 msgr2=0x7fd7580ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:24.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.013+0000 7fd75e600640 1 --2- 172.21.15.196:0/57608090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7580d2600 0x7fd7580ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.013+0000 7fd75e600640 1 -- 172.21.15.196:0/57608090 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd758156cb0 con 0x7fd7581628c0 2024-04-28T13:33:24.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.014+0000 7fd75e600640 1 --2- 172.21.15.196:0/57608090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7581628c0 0x7fd758160bf0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd75002ac70 tx=0x7fd750004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:24.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.014+0000 7fd755600640 1 -- 172.21.15.196:0/57608090 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750040070 con 0x7fd7581628c0 2024-04-28T13:33:24.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.014+0000 7fd755600640 1 -- 172.21.15.196:0/57608090 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd750031bc0 con 0x7fd7581628c0 2024-04-28T13:33:24.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.014+0000 7fd755600640 1 -- 172.21.15.196:0/57608090 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750009450 con 0x7fd7581628c0 2024-04-28T13:33:24.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.014+0000 7fd75f000640 1 -- 172.21.15.196:0/57608090 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd758156f90 con 0x7fd7581628c0 2024-04-28T13:33:24.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.014+0000 7fd75f000640 1 -- 172.21.15.196:0/57608090 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd758157550 con 0x7fd7581628c0 2024-04-28T13:33:24.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.015+0000 7fd75f000640 1 -- 172.21.15.196:0/57608090 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7581579a0 con 0x7fd7581628c0 2024-04-28T13:33:24.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.016+0000 7fd755600640 1 -- 172.21.15.196:0/57608090 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd750039040 con 0x7fd7581628c0 2024-04-28T13:33:24.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.017+0000 7fd755600640 1 --2- 172.21.15.196:0/57608090 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd73806c980 0x7fd73806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:24.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.017+0000 7fd755600640 1 -- 172.21.15.196:0/57608090 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5029+0+0 (secure 0 0 0) 0x7fd7500ad3a0 con 0x7fd7581628c0 2024-04-28T13:33:24.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.017+0000 7fd75dc00640 1 --2- 172.21.15.196:0/57608090 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd73806c980 0x7fd73806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:24.018 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.019+0000 7fd75dc00640 1 --2- 172.21.15.196:0/57608090 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd73806c980 0x7fd73806ee40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd74c000f60 tx=0x7fd74c0078f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:24.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.021+0000 7fd755600640 1 -- 172.21.15.196:0/57608090 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd7581579a0 con 0x7fd7581628c0 2024-04-28T13:33:24.222 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.222+0000 7fd75f000640 1 -- 172.21.15.196:0/57608090 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd758050730 con 0x7fd7581628c0 2024-04-28T13:33:24.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.227+0000 7fd755600640 1 -- 172.21.15.196:0/57608090 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7fd75003ae60 con 0x7fd7581628c0 2024-04-28T13:33:24.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.230+0000 7fd75f000640 1 -- 172.21.15.196:0/57608090 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd73806c980 msgr2=0x7fd73806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:24.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.230+0000 7fd75f000640 1 --2- 172.21.15.196:0/57608090 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd73806c980 0x7fd73806ee40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd74c000f60 tx=0x7fd74c0078f0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.230 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.230+0000 7fd75f000640 1 -- 172.21.15.196:0/57608090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7581628c0 msgr2=0x7fd758160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:24.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.230+0000 7fd75f000640 1 --2- 172.21.15.196:0/57608090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7581628c0 0x7fd758160bf0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd75002ac70 tx=0x7fd750004a40 comp rx=0 tx=0).stop 2024-04-28T13:33:24.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.230+0000 7fd75f000640 1 -- 172.21.15.196:0/57608090 shutdown_connections 2024-04-28T13:33:24.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.230+0000 7fd75f000640 1 --2- 172.21.15.196:0/57608090 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd73806c980 0x7fd73806ee40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.230+0000 7fd75f000640 1 --2- 172.21.15.196:0/57608090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7581628c0 0x7fd758160bf0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.230+0000 7fd75f000640 1 --2- 172.21.15.196:0/57608090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7580d2600 0x7fd7580ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.230+0000 7fd75f000640 1 --2- 172.21.15.196:0/57608090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7580b93e0 0x7fd7581552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.230+0000 7fd75f000640 1 -- 172.21.15.196:0/57608090 >> 172.21.15.196:0/57608090 conn(0x7fd7580d4fc0 msgr2=0x7fd7580d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:24.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.230+0000 7fd75f000640 1 -- 172.21.15.196:0/57608090 shutdown_connections 2024-04-28T13:33:24.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.230+0000 7fd75f000640 1 -- 172.21.15.196:0/57608090 wait complete. 2024-04-28T13:33:24.250 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked' 2024-04-28T13:33:24.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.451+0000 7fc361800640 1 Processor -- start 2024-04-28T13:33:24.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.451+0000 7fc361800640 1 -- start start 2024-04-28T13:33:24.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.452+0000 7fc361800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc35c0b5c80 0x7fc35c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:24.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.452+0000 7fc361800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc35c06dce0 con 0x7fc35c0d1790 2024-04-28T13:33:24.452 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.452+0000 7fc361800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc35c06dfc0 con 0x7fc35c15f050 2024-04-28T13:33:24.453 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.453+0000 7fc361800640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc35c0b60b0 con 0x7fc35c0b5c80 2024-04-28T13:33:24.453 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.453+0000 7fc353400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc35c0d1790 0x7fc35c072170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:54090/0 (socket says 172.21.15.196:54090) 2024-04-28T13:33:24.453 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.453+0000 7fc353400640 1 -- 172.21.15.196:0/3268531605 learned_addr learned my addr 172.21.15.196:0/3268531605 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:24.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.453+0000 7fc35b400640 1 -- 172.21.15.196:0/3268531605 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3709796815 0 0) 0x7fc35c06dce0 con 0x7fc35c0d1790 2024-04-28T13:33:24.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.453+0000 7fc35b400640 1 -- 172.21.15.196:0/3268531605 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc340003520 con 0x7fc35c0d1790 2024-04-28T13:33:24.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.453+0000 7fc360e00640 1 --2- 172.21.15.196:0/3268531605 >> v2:172.21.15.203:3300/0 conn(0x7fc35c0b5c80 0x7fc35c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:24.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc35b400640 1 -- 172.21.15.196:0/3268531605 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3710984068 0 0) 0x7fc340003520 con 0x7fc35c0d1790 2024-04-28T13:33:24.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc35b400640 1 -- 172.21.15.196:0/3268531605 >> v2:172.21.15.203:3300/0 conn(0x7fc35c0b5c80 msgr2=0x7fc35c06f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:24.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc35b400640 1 --2- 172.21.15.196:0/3268531605 >> v2:172.21.15.203:3300/0 conn(0x7fc35c0b5c80 0x7fc35c06f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc35b400640 1 -- 172.21.15.196:0/3268531605 >> v1:172.21.15.203:6789/0 conn(0x7fc35c15f050 legacy=0x7fc35c0d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:24.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc35b400640 1 -- 172.21.15.196:0/3268531605 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35c06e2a0 con 0x7fc35c0d1790 2024-04-28T13:33:24.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc35b400640 1 -- 172.21.15.196:0/3268531605 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fc344001670 con 0x7fc35c0d1790 2024-04-28T13:33:24.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc35b400640 1 -- 172.21.15.196:0/3268531605 >> v1:172.21.15.196:6789/0 conn(0x7fc35c0d1790 legacy=0x7fc35c072170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:24.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc35b400640 1 --2- 172.21.15.196:0/3268531605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc340003a00 0x7fc340003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:24.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc35b400640 1 --2- 172.21.15.196:0/3268531605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc340004800 0x7fc340004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:24.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc35b400640 1 --2- 172.21.15.196:0/3268531605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc340005210 0x7fc34000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:24.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc35b400640 1 -- 172.21.15.196:0/3268531605 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c0b60b0 con 0x7fc340004800 2024-04-28T13:33:24.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc35b400640 1 -- 172.21.15.196:0/3268531605 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc34000c160 con 0x7fc340005210 2024-04-28T13:33:24.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc35b400640 1 -- 172.21.15.196:0/3268531605 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc34000c360 con 0x7fc340003a00 2024-04-28T13:33:24.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc353400640 1 --2- 172.21.15.196:0/3268531605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc340004800 0x7fc340004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:24.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc35be00640 1 --2- 172.21.15.196:0/3268531605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc340003a00 0x7fc340003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:24.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc353400640 1 -- 172.21.15.196:0/3268531605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc340003a00 msgr2=0x7fc340003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:24.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc360e00640 1 --2- 172.21.15.196:0/3268531605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc340005210 0x7fc34000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:24.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc353400640 1 --2- 172.21.15.196:0/3268531605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc340003a00 0x7fc340003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc353400640 1 -- 172.21.15.196:0/3268531605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc340005210 msgr2=0x7fc34000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:24.456 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc353400640 1 --2- 172.21.15.196:0/3268531605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc340005210 0x7fc34000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.454+0000 7fc353400640 1 -- 172.21.15.196:0/3268531605 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc340003520 con 0x7fc340004800 2024-04-28T13:33:24.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.455+0000 7fc360e00640 1 -- 172.21.15.196:0/3268531605 reap_dead start 2024-04-28T13:33:24.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.455+0000 7fc353400640 1 --2- 172.21.15.196:0/3268531605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc340004800 0x7fc340004c80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc34400c0e0 tx=0x7fc340005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2695214df727356c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:24.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.456+0000 7fc35b400640 1 -- 172.21.15.196:0/3268531605 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344007c20 con 0x7fc340004800 2024-04-28T13:33:24.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.456+0000 7fc35b400640 1 -- 172.21.15.196:0/3268531605 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc344005af0 con 0x7fc340004800 2024-04-28T13:33:24.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.456+0000 7fc35b400640 1 -- 172.21.15.196:0/3268531605 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344007070 con 0x7fc340004800 2024-04-28T13:33:24.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.456+0000 7fc361800640 1 -- 172.21.15.196:0/3268531605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc340004800 msgr2=0x7fc340004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:24.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.456+0000 7fc361800640 1 --2- 172.21.15.196:0/3268531605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc340004800 0x7fc340004c80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc34400c0e0 tx=0x7fc340005830 comp rx=0 tx=0).stop 2024-04-28T13:33:24.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.456+0000 7fc361800640 1 -- 172.21.15.196:0/3268531605 shutdown_connections 2024-04-28T13:33:24.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.456+0000 7fc361800640 1 --2- 172.21.15.196:0/3268531605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc340004800 0x7fc340004c80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.456+0000 7fc361800640 1 -- 172.21.15.196:0/3268531605 >> 172.21.15.196:0/3268531605 conn(0x7fc35c06c6c0 msgr2=0x7fc35c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:24.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.456+0000 7fc361800640 1 -- 172.21.15.196:0/3268531605 shutdown_connections 2024-04-28T13:33:24.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.456+0000 7fc361800640 1 -- 172.21.15.196:0/3268531605 wait complete. 2024-04-28T13:33:24.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.458+0000 7fc361800640 1 Processor -- start 2024-04-28T13:33:24.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.458+0000 7fc361800640 1 -- start start 2024-04-28T13:33:24.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.459+0000 7fc361800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc340004800 0x7fc35c147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:24.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.459+0000 7fc361800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc35c1e9b90 0x7fc35c1e9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:24.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.459+0000 7fc361800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc35c1ea530 0x7fc35c160970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:24.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.459+0000 7fc361800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c15f430 con 0x7fc340004800 2024-04-28T13:33:24.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.459+0000 7fc361800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c06e860 con 0x7fc35c1e9b90 2024-04-28T13:33:24.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.459+0000 7fc361800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc35c15f5b0 con 0x7fc35c1ea530 2024-04-28T13:33:24.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.459+0000 7fc35be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc340004800 0x7fc35c147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:24.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.459+0000 7fc35be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc340004800 0x7fc35c147570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37360/0 (socket says 172.21.15.196:37360) 2024-04-28T13:33:24.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.459+0000 7fc35be00640 1 -- 172.21.15.196:0/502206478 learned_addr learned my addr 172.21.15.196:0/502206478 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:24.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.460+0000 7fc35be00640 1 -- 172.21.15.196:0/502206478 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc35c1ea530 msgr2=0x7fc35c160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:24.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.460+0000 7fc35be00640 1 --2- 172.21.15.196:0/502206478 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc35c1ea530 0x7fc35c160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.460+0000 7fc35be00640 1 -- 172.21.15.196:0/502206478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc35c1e9b90 msgr2=0x7fc35c1e9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:24.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.460+0000 7fc35be00640 1 --2- 172.21.15.196:0/502206478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc35c1e9b90 0x7fc35c1e9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.460+0000 7fc35be00640 1 -- 172.21.15.196:0/502206478 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35c1ea9b0 con 0x7fc340004800 2024-04-28T13:33:24.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.460+0000 7fc35be00640 1 --2- 172.21.15.196:0/502206478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc340004800 0x7fc35c147570 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00d460 tx=0x7fc34c00d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:24.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.460+0000 7fc358c00640 1 -- 172.21.15.196:0/502206478 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c018070 con 0x7fc340004800 2024-04-28T13:33:24.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.460+0000 7fc358c00640 1 -- 172.21.15.196:0/502206478 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc34c003610 con 0x7fc340004800 2024-04-28T13:33:24.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.460+0000 7fc358c00640 1 -- 172.21.15.196:0/502206478 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c003a00 con 0x7fc340004800 2024-04-28T13:33:24.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.460+0000 7fc361800640 1 -- 172.21.15.196:0/502206478 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35c15fbb0 con 0x7fc340004800 2024-04-28T13:33:24.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.461+0000 7fc361800640 1 -- 172.21.15.196:0/502206478 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc35c160170 con 0x7fc340004800 2024-04-28T13:33:24.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.462+0000 7fc361800640 1 -- 172.21.15.196:0/502206478 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc35c09f0f0 con 0x7fc340004800 2024-04-28T13:33:24.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.463+0000 7fc358c00640 1 -- 172.21.15.196:0/502206478 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc34c01b070 con 0x7fc340004800 2024-04-28T13:33:24.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.464+0000 7fc358c00640 1 --2- 172.21.15.196:0/502206478 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc33806c9d0 0x7fc33806ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:24.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.464+0000 7fc358c00640 1 -- 172.21.15.196:0/502206478 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5029+0+0 (secure 0 0 0) 0x7fc34c08b260 con 0x7fc340004800 2024-04-28T13:33:24.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.464+0000 7fc353400640 1 --2- 172.21.15.196:0/502206478 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc33806c9d0 0x7fc33806ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:24.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.466+0000 7fc353400640 1 --2- 172.21.15.196:0/502206478 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc33806c9d0 0x7fc33806ee90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc344032000 tx=0x7fc344007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:24.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.468+0000 7fc358c00640 1 -- 172.21.15.196:0/502206478 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc34c0590d0 con 0x7fc340004800 2024-04-28T13:33:24.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.738+0000 7fc361800640 1 -- 172.21.15.196:0/502206478 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]} v 0) -- 0x7fc35c0507a0 con 0x7fc340004800 2024-04-28T13:33:24.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.878+0000 7fc358c00640 1 -- 172.21.15.196:0/502206478 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_blocked"]}]=0 v138) ==== 137+0+0 (secure 0 0 0) 0x7fc34c05d3d0 con 0x7fc340004800 2024-04-28T13:33:24.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.881+0000 7fc361800640 1 -- 172.21.15.196:0/502206478 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc33806c9d0 msgr2=0x7fc33806ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:24.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.881+0000 7fc361800640 1 --2- 172.21.15.196:0/502206478 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc33806c9d0 0x7fc33806ee90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc344032000 tx=0x7fc344007000 comp rx=0 tx=0).stop 2024-04-28T13:33:24.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.881+0000 7fc361800640 1 -- 172.21.15.196:0/502206478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc340004800 msgr2=0x7fc35c147570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:24.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.881+0000 7fc361800640 1 --2- 172.21.15.196:0/502206478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc340004800 0x7fc35c147570 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00d460 tx=0x7fc34c00d920 comp rx=0 tx=0).stop 2024-04-28T13:33:24.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.881+0000 7fc361800640 1 -- 172.21.15.196:0/502206478 shutdown_connections 2024-04-28T13:33:24.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.881+0000 7fc361800640 1 --2- 172.21.15.196:0/502206478 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc33806c9d0 0x7fc33806ee90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.881+0000 7fc361800640 1 --2- 172.21.15.196:0/502206478 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc35c1ea530 0x7fc35c160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.881+0000 7fc361800640 1 --2- 172.21.15.196:0/502206478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc35c1e9b90 0x7fc35c1e9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.881+0000 7fc361800640 1 --2- 172.21.15.196:0/502206478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc340004800 0x7fc35c147570 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:24.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.881+0000 7fc361800640 1 -- 172.21.15.196:0/502206478 >> 172.21.15.196:0/502206478 conn(0x7fc35c06c6c0 msgr2=0x7fc35c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:24.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.882+0000 7fc361800640 1 -- 172.21.15.196:0/502206478 shutdown_connections 2024-04-28T13:33:24.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:24.882+0000 7fc361800640 1 -- 172.21.15.196:0/502206478 wait complete. 2024-04-28T13:33:24.901 INFO:tasks.cephfs_test_runner:That a file with ceph.quiesce.block is not quiesced. ... ok 2024-04-28T13:33:24.901 INFO:tasks.cephfs_test_runner:Starting test: test_quiesce_find (tasks.cephfs.test_quiesce.TestQuiesce) That a `find` can be executed on a quiesced path. 2024-04-28T13:33:24.901 INFO:tasks.cephfs_test_runner:test_quiesce_find (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-28T13:33:24.902 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:33:25.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.092+0000 7fada6800640 1 Processor -- start 2024-04-28T13:33:25.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.092+0000 7fada6800640 1 -- start start 2024-04-28T13:33:25.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.094+0000 7fada6800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fada01469f0 0x7fada00ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.094+0000 7fada6800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fada00d1410 0x7fada00b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.094+0000 7fada6800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada0160f40 con 0x7fada00cd420 2024-04-28T13:33:25.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.094+0000 7fada6800640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fada00b5960 con 0x7fada00d1410 2024-04-28T13:33:25.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.094+0000 7fada6800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fada0161ac0 con 0x7fada01469f0 2024-04-28T13:33:25.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.094+0000 7fad9ea00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fada00cd420 0x7fada00d0d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:54100/0 (socket says 172.21.15.196:54100) 2024-04-28T13:33:25.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.094+0000 7fad9ea00640 1 -- 172.21.15.196:0/3237594112 learned_addr learned my addr 172.21.15.196:0/3237594112 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:25.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.094+0000 7fad9f400640 1 --2- 172.21.15.196:0/3237594112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fada01469f0 0x7fada00ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:25.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.094+0000 7fad9fe00640 1 --2- 172.21.15.196:0/3237594112 >> v2:172.21.15.203:3300/0 conn(0x7fada00d1410 0x7fada00b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:25.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.094+0000 7fad9e000640 1 -- 172.21.15.196:0/3237594112 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1386161957 0 0) 0x7fada0160f40 con 0x7fada00cd420 2024-04-28T13:33:25.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9e000640 1 -- 172.21.15.196:0/3237594112 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad84003520 con 0x7fada00cd420 2024-04-28T13:33:25.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9e000640 1 -- 172.21.15.196:0/3237594112 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3802370532 0 0) 0x7fad84003520 con 0x7fada00cd420 2024-04-28T13:33:25.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9e000640 1 -- 172.21.15.196:0/3237594112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fada01469f0 msgr2=0x7fada00ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:25.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9e000640 1 --2- 172.21.15.196:0/3237594112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fada01469f0 0x7fada00ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9e000640 1 -- 172.21.15.196:0/3237594112 >> v2:172.21.15.203:3300/0 conn(0x7fada00d1410 msgr2=0x7fada00b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:25.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9e000640 1 --2- 172.21.15.196:0/3237594112 >> v2:172.21.15.203:3300/0 conn(0x7fada00d1410 0x7fada00b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9e000640 1 -- 172.21.15.196:0/3237594112 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada0161220 con 0x7fada00cd420 2024-04-28T13:33:25.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9e000640 1 -- 172.21.15.196:0/3237594112 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fad8c001690 con 0x7fada00cd420 2024-04-28T13:33:25.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9e000640 1 -- 172.21.15.196:0/3237594112 >> v1:172.21.15.196:6789/0 conn(0x7fada00cd420 legacy=0x7fada00d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:25.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9e000640 1 --2- 172.21.15.196:0/3237594112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad840039b0 0x7fad84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9e000640 1 --2- 172.21.15.196:0/3237594112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fada01469f0 0x7fad84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9e000640 1 --2- 172.21.15.196:0/3237594112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad84004d40 0x7fad840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9e000640 1 -- 172.21.15.196:0/3237594112 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fada0161ac0 con 0x7fad840039b0 2024-04-28T13:33:25.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9e000640 1 -- 172.21.15.196:0/3237594112 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fad8400b9c0 con 0x7fad84004d40 2024-04-28T13:33:25.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9e000640 1 -- 172.21.15.196:0/3237594112 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fad8400bb40 con 0x7fada01469f0 2024-04-28T13:33:25.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.095+0000 7fad9f400640 1 --2- 172.21.15.196:0/3237594112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad840039b0 0x7fad84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:25.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.096+0000 7fad9f400640 1 -- 172.21.15.196:0/3237594112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fada01469f0 msgr2=0x7fad84004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:25.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.096+0000 7fad9f400640 1 --2- 172.21.15.196:0/3237594112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fada01469f0 0x7fad84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.096+0000 7fad9f400640 1 -- 172.21.15.196:0/3237594112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad84004d40 msgr2=0x7fad840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:25.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.096+0000 7fad9f400640 1 --2- 172.21.15.196:0/3237594112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad84004d40 0x7fad840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.098 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.096+0000 7fad9f400640 1 -- 172.21.15.196:0/3237594112 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad84003520 con 0x7fad840039b0 2024-04-28T13:33:25.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.097+0000 7fad9f400640 1 --2- 172.21.15.196:0/3237594112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad840039b0 0x7fad84003dd0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fad90009ac0 tx=0x7fad84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf6d080d60eb2b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:25.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.097+0000 7fad9e000640 1 -- 172.21.15.196:0/3237594112 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90036070 con 0x7fad840039b0 2024-04-28T13:33:25.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.097+0000 7fad9e000640 1 -- 172.21.15.196:0/3237594112 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad90031400 con 0x7fad840039b0 2024-04-28T13:33:25.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.097+0000 7fad9e000640 1 -- 172.21.15.196:0/3237594112 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90009690 con 0x7fad840039b0 2024-04-28T13:33:25.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.097+0000 7fada6800640 1 -- 172.21.15.196:0/3237594112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad840039b0 msgr2=0x7fad84003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:25.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.097+0000 7fada6800640 1 --2- 172.21.15.196:0/3237594112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad840039b0 0x7fad84003dd0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fad90009ac0 tx=0x7fad84005830 comp rx=0 tx=0).stop 2024-04-28T13:33:25.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.097+0000 7fad9fe00640 1 -- 172.21.15.196:0/3237594112 reap_dead start 2024-04-28T13:33:25.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.098+0000 7fada6800640 1 -- 172.21.15.196:0/3237594112 shutdown_connections 2024-04-28T13:33:25.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.098+0000 7fada6800640 1 -- 172.21.15.196:0/3237594112 >> 172.21.15.196:0/3237594112 conn(0x7fada00cd830 msgr2=0x7fada00cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:25.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.098+0000 7fada6800640 1 -- 172.21.15.196:0/3237594112 shutdown_connections 2024-04-28T13:33:25.099 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.098+0000 7fada6800640 1 -- 172.21.15.196:0/3237594112 wait complete. 2024-04-28T13:33:25.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.099+0000 7fada6800640 1 Processor -- start 2024-04-28T13:33:25.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.100+0000 7fada6800640 1 -- start start 2024-04-28T13:33:25.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.100+0000 7fada6800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad840039b0 0x7fada015ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fada6800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fada00af990 0x7fada015e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fada6800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada015e9c0 0x7fada014df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fada6800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fada0072020 con 0x7fada015e9c0 2024-04-28T13:33:25.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fada6800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fada01617e0 con 0x7fad840039b0 2024-04-28T13:33:25.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fada6800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fada0160b80 con 0x7fada00af990 2024-04-28T13:33:25.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fad9fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada015e9c0 0x7fada014df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:25.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fad9fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada015e9c0 0x7fada014df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37386/0 (socket says 172.21.15.196:37386) 2024-04-28T13:33:25.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fad9fe00640 1 -- 172.21.15.196:0/1815193278 learned_addr learned my addr 172.21.15.196:0/1815193278 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:25.102 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fad9ea00640 1 --2- 172.21.15.196:0/1815193278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fada00af990 0x7fada015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:25.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fad9fe00640 1 -- 172.21.15.196:0/1815193278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fada00af990 msgr2=0x7fada015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:25.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fad9f400640 1 --2- 172.21.15.196:0/1815193278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad840039b0 0x7fada015ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:25.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fad9fe00640 1 --2- 172.21.15.196:0/1815193278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fada00af990 0x7fada015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fad9fe00640 1 -- 172.21.15.196:0/1815193278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad840039b0 msgr2=0x7fada015ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:25.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fad9fe00640 1 --2- 172.21.15.196:0/1815193278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad840039b0 0x7fada015ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fad9fe00640 1 -- 172.21.15.196:0/1815193278 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada01604e0 con 0x7fada015e9c0 2024-04-28T13:33:25.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.101+0000 7fad9ea00640 1 --2- 172.21.15.196:0/1815193278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fada00af990 0x7fada015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:25.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.102+0000 7fad9fe00640 1 --2- 172.21.15.196:0/1815193278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada015e9c0 0x7fada014df40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fada0073630 tx=0x7fad9800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:25.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.102+0000 7fad97400640 1 -- 172.21.15.196:0/1815193278 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9800bb40 con 0x7fada015e9c0 2024-04-28T13:33:25.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.102+0000 7fad97400640 1 -- 172.21.15.196:0/1815193278 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad9801a070 con 0x7fada015e9c0 2024-04-28T13:33:25.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.102+0000 7fad97400640 1 -- 172.21.15.196:0/1815193278 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98014400 con 0x7fada015e9c0 2024-04-28T13:33:25.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.102+0000 7fada6800640 1 -- 172.21.15.196:0/1815193278 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada015ef00 con 0x7fada015e9c0 2024-04-28T13:33:25.103 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.102+0000 7fada6800640 1 -- 172.21.15.196:0/1815193278 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada015f4c0 con 0x7fada015e9c0 2024-04-28T13:33:25.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.103+0000 7fada6800640 1 -- 172.21.15.196:0/1815193278 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada006a290 con 0x7fada015e9c0 2024-04-28T13:33:25.104 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.104+0000 7fad97400640 1 -- 172.21.15.196:0/1815193278 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fad98005d10 con 0x7fada015e9c0 2024-04-28T13:33:25.105 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.105+0000 7fad97400640 1 --2- 172.21.15.196:0/1815193278 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad8006c9b0 0x7fad8006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.105 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.106+0000 7fad97400640 1 -- 172.21.15.196:0/1815193278 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5029+0+0 (secure 0 0 0) 0x7fad9808ba80 con 0x7fada015e9c0 2024-04-28T13:33:25.106 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.106+0000 7fad9f400640 1 --2- 172.21.15.196:0/1815193278 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad8006c9b0 0x7fad8006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:25.107 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.107+0000 7fad9f400640 1 --2- 172.21.15.196:0/1815193278 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad8006c9b0 0x7fad8006ee70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fad90033000 tx=0x7fad90008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:25.110 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.110+0000 7fad97400640 1 -- 172.21.15.196:0/1815193278 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad9808de20 con 0x7fada015e9c0 2024-04-28T13:33:25.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.310+0000 7fada6800640 1 -- 172.21.15.196:0/1815193278 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fada006fd90 con 0x7fada015e9c0 2024-04-28T13:33:25.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.312+0000 7fad97400640 1 -- 172.21.15.196:0/1815193278 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+13188 (secure 0 0 0) 0x7fad98059f50 con 0x7fada015e9c0 2024-04-28T13:33:25.313 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:25.313 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":37,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:33:22.854974+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6836/1998460400":"2024-04-29T13:33:19.682526+0000","172.21.15.203:6835/1998460400":"2024-04-29T13:33:19.682526+0000","172.21.15.203:6836/459640695":"2024-04-29T13:32:58.527451+0000","172.21.15.203:6835/459640695":"2024-04-29T13:32:58.527451+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:33:25.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.315+0000 7fada6800640 1 -- 172.21.15.196:0/1815193278 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad8006c9b0 msgr2=0x7fad8006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:25.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.315+0000 7fada6800640 1 --2- 172.21.15.196:0/1815193278 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad8006c9b0 0x7fad8006ee70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fad90033000 tx=0x7fad90008030 comp rx=0 tx=0).stop 2024-04-28T13:33:25.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.315+0000 7fada6800640 1 -- 172.21.15.196:0/1815193278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada015e9c0 msgr2=0x7fada014df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:25.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.315+0000 7fada6800640 1 --2- 172.21.15.196:0/1815193278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada015e9c0 0x7fada014df40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fada0073630 tx=0x7fad9800eec0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.315+0000 7fada6800640 1 -- 172.21.15.196:0/1815193278 shutdown_connections 2024-04-28T13:33:25.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.315+0000 7fada6800640 1 --2- 172.21.15.196:0/1815193278 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fad8006c9b0 0x7fad8006ee70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.315+0000 7fada6800640 1 --2- 172.21.15.196:0/1815193278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada015e9c0 0x7fada014df40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.315+0000 7fada6800640 1 --2- 172.21.15.196:0/1815193278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fada00af990 0x7fada015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.315+0000 7fada6800640 1 --2- 172.21.15.196:0/1815193278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad840039b0 0x7fada015ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.315+0000 7fada6800640 1 -- 172.21.15.196:0/1815193278 >> 172.21.15.196:0/1815193278 conn(0x7fada00cd830 msgr2=0x7fada01440d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:25.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.315+0000 7fada6800640 1 -- 172.21.15.196:0/1815193278 shutdown_connections 2024-04-28T13:33:25.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.315+0000 7fada6800640 1 -- 172.21.15.196:0/1815193278 wait complete. 2024-04-28T13:33:25.334 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:33:25.334 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:33:25.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.532+0000 7f3c43e00640 1 Processor -- start 2024-04-28T13:33:25.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.533+0000 7f3c43e00640 1 -- start start 2024-04-28T13:33:25.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.534+0000 7f3c43e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3c440d3830 0x7f3c440d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.534+0000 7f3c43e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c440d0600 0x7f3c440d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.534+0000 7f3c43e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c440cfb80 con 0x7f3c440d3c10 2024-04-28T13:33:25.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.534+0000 7f3c43e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3c440d0f90 con 0x7f3c440d3830 2024-04-28T13:33:25.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.534+0000 7f3c43e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3c440d2ee0 con 0x7f3c440d0600 2024-04-28T13:33:25.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.534+0000 7f3c42a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3c440d3830 0x7f3c440d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:25.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.534+0000 7f3c42a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3c440d3830 0x7f3c440d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37390/0 (socket says 172.21.15.196:37390) 2024-04-28T13:33:25.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.534+0000 7f3c42a00640 1 -- 172.21.15.196:0/1168469334 learned_addr learned my addr 172.21.15.196:0/1168469334 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:25.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.535+0000 7f3c42a00640 1 -- 172.21.15.196:0/1168469334 >> v2:172.21.15.203:3300/0 conn(0x7f3c440d0600 msgr2=0x7f3c440d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:25.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.535+0000 7f3c42a00640 1 --2- 172.21.15.196:0/1168469334 >> v2:172.21.15.203:3300/0 conn(0x7f3c440d0600 0x7f3c440d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.535+0000 7f3c42a00640 1 -- 172.21.15.196:0/1168469334 >> v1:172.21.15.203:6789/0 conn(0x7f3c440d3c10 legacy=0x7f3c440b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:25.536 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.535+0000 7f3c42a00640 1 -- 172.21.15.196:0/1168469334 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c440d31c0 con 0x7f3c440d3830 2024-04-28T13:33:25.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.536+0000 7f3c42a00640 1 --2- 172.21.15.196:0/1168469334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c440d3830 0x7f3c440d2400 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3c4408e7e0 tx=0x7f3c3402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0376774c9f88f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:25.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.536+0000 7f3c41600640 1 -- 172.21.15.196:0/1168469334 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34037070 con 0x7f3c440d3830 2024-04-28T13:33:25.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.536+0000 7f3c41600640 1 -- 172.21.15.196:0/1168469334 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c34032440 con 0x7f3c440d3830 2024-04-28T13:33:25.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.536+0000 7f3c41600640 1 -- 172.21.15.196:0/1168469334 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34032810 con 0x7f3c440d3830 2024-04-28T13:33:25.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.537+0000 7f3c43e00640 1 -- 172.21.15.196:0/1168469334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c440d3830 msgr2=0x7f3c440d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:25.537 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.537+0000 7f3c43e00640 1 --2- 172.21.15.196:0/1168469334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c440d3830 0x7f3c440d2400 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3c4408e7e0 tx=0x7f3c3402a230 comp rx=0 tx=0).stop 2024-04-28T13:33:25.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.537+0000 7f3c43e00640 1 -- 172.21.15.196:0/1168469334 shutdown_connections 2024-04-28T13:33:25.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.537+0000 7f3c43e00640 1 --2- 172.21.15.196:0/1168469334 >> v2:172.21.15.203:3300/0 conn(0x7f3c440d0600 0x7f3c440d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.537+0000 7f3c43e00640 1 --2- 172.21.15.196:0/1168469334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c440d3830 0x7f3c440d2400 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.537+0000 7f3c43e00640 1 -- 172.21.15.196:0/1168469334 >> 172.21.15.196:0/1168469334 conn(0x7f3c440d4fe0 msgr2=0x7f3c440d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:25.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.537+0000 7f3c43e00640 1 -- 172.21.15.196:0/1168469334 shutdown_connections 2024-04-28T13:33:25.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.537+0000 7f3c43e00640 1 -- 172.21.15.196:0/1168469334 wait complete. 2024-04-28T13:33:25.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.539+0000 7f3c43e00640 1 Processor -- start 2024-04-28T13:33:25.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.539+0000 7f3c43e00640 1 -- start start 2024-04-28T13:33:25.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.539+0000 7f3c43e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c440d0600 0x7f3c441e8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.539+0000 7f3c43e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c440d3830 0x7f3c440c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.539+0000 7f3c43e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c440d3c10 0x7f3c44156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.540+0000 7f3c43e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c440b8540 con 0x7f3c440d3830 2024-04-28T13:33:25.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.540+0000 7f3c43e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3c440d00e0 con 0x7f3c440d0600 2024-04-28T13:33:25.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.540+0000 7f3c43e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3c440d2940 con 0x7f3c440d3c10 2024-04-28T13:33:25.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.540+0000 7f3c42000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c440d3830 0x7f3c440c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:25.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.540+0000 7f3c42000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c440d3830 0x7f3c440c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37402/0 (socket says 172.21.15.196:37402) 2024-04-28T13:33:25.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.540+0000 7f3c42000640 1 -- 172.21.15.196:0/1700029648 learned_addr learned my addr 172.21.15.196:0/1700029648 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:25.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.540+0000 7f3c42000640 1 -- 172.21.15.196:0/1700029648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c440d3c10 msgr2=0x7f3c44156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:25.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.540+0000 7f3c42000640 1 --2- 172.21.15.196:0/1700029648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c440d3c10 0x7f3c44156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.540+0000 7f3c42000640 1 -- 172.21.15.196:0/1700029648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c440d0600 msgr2=0x7f3c441e8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:25.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.540+0000 7f3c42000640 1 --2- 172.21.15.196:0/1700029648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c440d0600 0x7f3c441e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.540+0000 7f3c42000640 1 -- 172.21.15.196:0/1700029648 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c441e7570 con 0x7f3c440d3830 2024-04-28T13:33:25.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.540+0000 7f3c42000640 1 --2- 172.21.15.196:0/1700029648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c440d3830 0x7f3c440c94f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00ba50 tx=0x7f3c2c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:25.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.541+0000 7f3c3aa00640 1 -- 172.21.15.196:0/1700029648 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c019070 con 0x7f3c440d3830 2024-04-28T13:33:25.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.541+0000 7f3c3aa00640 1 -- 172.21.15.196:0/1700029648 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c2c00cc10 con 0x7f3c440d3830 2024-04-28T13:33:25.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.541+0000 7f3c3aa00640 1 -- 172.21.15.196:0/1700029648 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c014440 con 0x7f3c440d3830 2024-04-28T13:33:25.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.541+0000 7f3c43e00640 1 -- 172.21.15.196:0/1700029648 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c441e7850 con 0x7f3c440d3830 2024-04-28T13:33:25.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.541+0000 7f3c43e00640 1 -- 172.21.15.196:0/1700029648 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c441e7e10 con 0x7f3c440d3830 2024-04-28T13:33:25.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.542+0000 7f3c43e00640 1 -- 172.21.15.196:0/1700029648 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c44050730 con 0x7f3c440d3830 2024-04-28T13:33:25.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.543+0000 7f3c3aa00640 1 -- 172.21.15.196:0/1700029648 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f3c2c01c070 con 0x7f3c440d3830 2024-04-28T13:33:25.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.544+0000 7f3c3aa00640 1 --2- 172.21.15.196:0/1700029648 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3c2006c980 0x7f3c2006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.544+0000 7f3c3aa00640 1 -- 172.21.15.196:0/1700029648 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5029+0+0 (secure 0 0 0) 0x7f3c2c08b770 con 0x7f3c440d3830 2024-04-28T13:33:25.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.545+0000 7f3c42a00640 1 --2- 172.21.15.196:0/1700029648 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3c2006c980 0x7f3c2006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:25.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.546+0000 7f3c42a00640 1 --2- 172.21.15.196:0/1700029648 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3c2006c980 0x7f3c2006ee40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3c34034000 tx=0x7f3c3402c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:25.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.548+0000 7f3c3aa00640 1 -- 172.21.15.196:0/1700029648 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3c2c059de0 con 0x7f3c440d3830 2024-04-28T13:33:25.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.736+0000 7f3c43e00640 1 -- 172.21.15.196:0/1700029648 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3c440d17d0 con 0x7f3c440d3830 2024-04-28T13:33:25.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.736+0000 7f3c3aa00640 1 -- 172.21.15.196:0/1700029648 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) ==== 93+0+10 (secure 0 0 0) 0x7f3c2c0596e0 con 0x7f3c440d3830 2024-04-28T13:33:25.737 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:33:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.739+0000 7f3c43e00640 1 -- 172.21.15.196:0/1700029648 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3c2006c980 msgr2=0x7f3c2006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.739+0000 7f3c43e00640 1 --2- 172.21.15.196:0/1700029648 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3c2006c980 0x7f3c2006ee40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3c34034000 tx=0x7f3c3402c000 comp rx=0 tx=0).stop 2024-04-28T13:33:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.739+0000 7f3c43e00640 1 -- 172.21.15.196:0/1700029648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c440d3830 msgr2=0x7f3c440c94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.739+0000 7f3c43e00640 1 --2- 172.21.15.196:0/1700029648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c440d3830 0x7f3c440c94f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00ba50 tx=0x7f3c2c00bf10 comp rx=0 tx=0).stop 2024-04-28T13:33:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.739+0000 7f3c43e00640 1 -- 172.21.15.196:0/1700029648 shutdown_connections 2024-04-28T13:33:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.739+0000 7f3c43e00640 1 --2- 172.21.15.196:0/1700029648 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3c2006c980 0x7f3c2006ee40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.739+0000 7f3c43e00640 1 --2- 172.21.15.196:0/1700029648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c440d3c10 0x7f3c44156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.739+0000 7f3c43e00640 1 --2- 172.21.15.196:0/1700029648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c440d3830 0x7f3c440c94f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.739+0000 7f3c43e00640 1 --2- 172.21.15.196:0/1700029648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c440d0600 0x7f3c441e8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.739+0000 7f3c43e00640 1 -- 172.21.15.196:0/1700029648 >> 172.21.15.196:0/1700029648 conn(0x7f3c440d4fe0 msgr2=0x7f3c441554c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.740+0000 7f3c43e00640 1 -- 172.21.15.196:0/1700029648 shutdown_connections 2024-04-28T13:33:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.740+0000 7f3c43e00640 1 -- 172.21.15.196:0/1700029648 wait complete. 2024-04-28T13:33:25.757 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_find' 2024-04-28T13:33:25.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.952+0000 7f243f200640 1 Processor -- start 2024-04-28T13:33:25.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.952+0000 7f243f200640 1 -- start start 2024-04-28T13:33:25.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.954+0000 7f243f200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f244015a9a0 0x7f24400cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.954+0000 7f243f200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f24400add70 0x7f24400d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.954+0000 7f243f200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24400d15c0 0x7f24400af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.954+0000 7f243f200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f24400d1a20 con 0x7f24400add70 2024-04-28T13:33:25.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.954+0000 7f243f200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f2440160b00 con 0x7f244015a9a0 2024-04-28T13:33:25.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.954+0000 7f243f200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2440160de0 con 0x7f24400d15c0 2024-04-28T13:33:25.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.954+0000 7f243d400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f24400add70 0x7f24400d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:25.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.954+0000 7f243d400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f24400add70 0x7f24400d1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37414/0 (socket says 172.21.15.196:37414) 2024-04-28T13:33:25.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.955+0000 7f243d400640 1 -- 172.21.15.196:0/3303872298 learned_addr learned my addr 172.21.15.196:0/3303872298 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:25.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.955+0000 7f243e800640 1 --2- 172.21.15.196:0/3303872298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24400d15c0 0x7f24400af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:25.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.955+0000 7f243d400640 1 -- 172.21.15.196:0/3303872298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24400d15c0 msgr2=0x7f24400af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:25.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.955+0000 7f243de00640 1 --2- 172.21.15.196:0/3303872298 >> v2:172.21.15.203:3300/0 conn(0x7f244015a9a0 0x7f24400cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:25.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.955+0000 7f243d400640 1 --2- 172.21.15.196:0/3303872298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24400d15c0 0x7f24400af7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.955+0000 7f243d400640 1 -- 172.21.15.196:0/3303872298 >> v2:172.21.15.203:3300/0 conn(0x7f244015a9a0 msgr2=0x7f24400cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:25.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.955+0000 7f243d400640 1 --2- 172.21.15.196:0/3303872298 >> v2:172.21.15.203:3300/0 conn(0x7f244015a9a0 0x7f24400cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.955+0000 7f243d400640 1 -- 172.21.15.196:0/3303872298 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24401610c0 con 0x7f24400add70 2024-04-28T13:33:25.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.955+0000 7f243de00640 1 --2- 172.21.15.196:0/3303872298 >> v2:172.21.15.203:3300/0 conn(0x7f244015a9a0 0x7f24400cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:25.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.956+0000 7f243d400640 1 --2- 172.21.15.196:0/3303872298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24400add70 0x7f24400d1080 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2440160d10 tx=0x7f242c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6d17eff6cd329 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:25.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.956+0000 7f243ca00640 1 -- 172.21.15.196:0/3303872298 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c037070 con 0x7f24400add70 2024-04-28T13:33:25.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.956+0000 7f243ca00640 1 -- 172.21.15.196:0/3303872298 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f242c032440 con 0x7f24400add70 2024-04-28T13:33:25.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.956+0000 7f243ca00640 1 -- 172.21.15.196:0/3303872298 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c032810 con 0x7f24400add70 2024-04-28T13:33:25.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.957+0000 7f243f200640 1 -- 172.21.15.196:0/3303872298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24400add70 msgr2=0x7f24400d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:25.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.957+0000 7f243f200640 1 --2- 172.21.15.196:0/3303872298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24400add70 0x7f24400d1080 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2440160d10 tx=0x7f242c02a1e0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.957+0000 7f243f200640 1 -- 172.21.15.196:0/3303872298 shutdown_connections 2024-04-28T13:33:25.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.957+0000 7f243f200640 1 --2- 172.21.15.196:0/3303872298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24400add70 0x7f24400d1080 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.957+0000 7f243f200640 1 --2- 172.21.15.196:0/3303872298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24400d15c0 0x7f24400af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.957+0000 7f243f200640 1 --2- 172.21.15.196:0/3303872298 >> v2:172.21.15.203:3300/0 conn(0x7f244015a9a0 0x7f24400cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.957+0000 7f243f200640 1 -- 172.21.15.196:0/3303872298 >> 172.21.15.196:0/3303872298 conn(0x7f244006c6c0 msgr2=0x7f244006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:25.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.957+0000 7f243f200640 1 -- 172.21.15.196:0/3303872298 shutdown_connections 2024-04-28T13:33:25.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.957+0000 7f243f200640 1 -- 172.21.15.196:0/3303872298 wait complete. 2024-04-28T13:33:25.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.959+0000 7f243f200640 1 Processor -- start 2024-04-28T13:33:25.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.959+0000 7f243f200640 1 -- start start 2024-04-28T13:33:25.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.960+0000 7f243f200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24400add70 0x7f244009def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.960+0000 7f243f200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24400d15c0 0x7f24400cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.960+0000 7f243f200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244015a9a0 0x7f24400b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.960+0000 7f243f200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24401618c0 con 0x7f24400d15c0 2024-04-28T13:33:25.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.960+0000 7f243f200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f24400d03d0 con 0x7f24400add70 2024-04-28T13:33:25.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.960+0000 7f243f200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f24400d06e0 con 0x7f244015a9a0 2024-04-28T13:33:25.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.960+0000 7f243d400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24400d15c0 0x7f24400cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:25.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.960+0000 7f243d400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24400d15c0 0x7f24400cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37422/0 (socket says 172.21.15.196:37422) 2024-04-28T13:33:25.960 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.960+0000 7f243d400640 1 -- 172.21.15.196:0/1905795102 learned_addr learned my addr 172.21.15.196:0/1905795102 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:25.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.960+0000 7f243d400640 1 -- 172.21.15.196:0/1905795102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244015a9a0 msgr2=0x7f24400b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:25.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.960+0000 7f243d400640 1 --2- 172.21.15.196:0/1905795102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244015a9a0 0x7f24400b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.960+0000 7f243d400640 1 -- 172.21.15.196:0/1905795102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24400add70 msgr2=0x7f244009def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:25.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.960+0000 7f243d400640 1 --2- 172.21.15.196:0/1905795102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24400add70 0x7f244009def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:25.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.960+0000 7f243d400640 1 -- 172.21.15.196:0/1905795102 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24400b1030 con 0x7f24400d15c0 2024-04-28T13:33:25.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.961+0000 7f243d400640 1 --2- 172.21.15.196:0/1905795102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24400d15c0 0x7f24400cbeb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f242c000c00 tx=0x7f242c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:25.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.961+0000 7f2436000640 1 -- 172.21.15.196:0/1905795102 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c037040 con 0x7f24400d15c0 2024-04-28T13:33:25.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.961+0000 7f2436000640 1 -- 172.21.15.196:0/1905795102 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f242c042070 con 0x7f24400d15c0 2024-04-28T13:33:25.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.961+0000 7f243f200640 1 -- 172.21.15.196:0/1905795102 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244009bde0 con 0x7f24400d15c0 2024-04-28T13:33:25.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.961+0000 7f2436000640 1 -- 172.21.15.196:0/1905795102 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c030bb0 con 0x7f24400d15c0 2024-04-28T13:33:25.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.962+0000 7f243f200640 1 -- 172.21.15.196:0/1905795102 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244009c3a0 con 0x7f24400d15c0 2024-04-28T13:33:25.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.963+0000 7f243f200640 1 -- 172.21.15.196:0/1905795102 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24400b1200 con 0x7f24400d15c0 2024-04-28T13:33:25.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.964+0000 7f2436000640 1 -- 172.21.15.196:0/1905795102 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f242c047460 con 0x7f24400d15c0 2024-04-28T13:33:25.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.964+0000 7f2436000640 1 --2- 172.21.15.196:0/1905795102 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f242006c9b0 0x7f242006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:25.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.965+0000 7f2436000640 1 -- 172.21.15.196:0/1905795102 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5029+0+0 (secure 0 0 0) 0x7f242c0ac6c0 con 0x7f24400d15c0 2024-04-28T13:33:25.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.965+0000 7f243de00640 1 --2- 172.21.15.196:0/1905795102 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f242006c9b0 0x7f242006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:25.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.966+0000 7f243de00640 1 --2- 172.21.15.196:0/1905795102 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f242006c9b0 0x7f242006ee70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2430006fd0 tx=0x7f2430008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:25.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:25.969+0000 7f2436000640 1 -- 172.21.15.196:0/1905795102 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f242c045360 con 0x7f24400d15c0 2024-04-28T13:33:26.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:26.244+0000 7f243f200640 1 -- 172.21.15.196:0/1905795102 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_find"]} v 0) -- 0x7f244009f0f0 con 0x7f24400d15c0 2024-04-28T13:33:26.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:26.893+0000 7f2436000640 1 -- 172.21.15.196:0/1905795102 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quiesce.TestQuiesce.test_quiesce_find"]}]=0 v140) ==== 137+0+0 (secure 0 0 0) 0x7f242c07ac10 con 0x7f24400d15c0 2024-04-28T13:33:26.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:26.897+0000 7f243f200640 1 -- 172.21.15.196:0/1905795102 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f242006c9b0 msgr2=0x7f242006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:26.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:26.897+0000 7f243f200640 1 --2- 172.21.15.196:0/1905795102 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f242006c9b0 0x7f242006ee70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2430006fd0 tx=0x7f2430008040 comp rx=0 tx=0).stop 2024-04-28T13:33:26.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:26.897+0000 7f243f200640 1 -- 172.21.15.196:0/1905795102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24400d15c0 msgr2=0x7f24400cbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:26.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:26.897+0000 7f243f200640 1 --2- 172.21.15.196:0/1905795102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24400d15c0 0x7f24400cbeb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f242c000c00 tx=0x7f242c0046c0 comp rx=0 tx=0).stop 2024-04-28T13:33:26.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:26.897+0000 7f243f200640 1 -- 172.21.15.196:0/1905795102 shutdown_connections 2024-04-28T13:33:26.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:26.897+0000 7f243f200640 1 --2- 172.21.15.196:0/1905795102 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f242006c9b0 0x7f242006ee70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:26.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:26.897+0000 7f243f200640 1 --2- 172.21.15.196:0/1905795102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244015a9a0 0x7f24400b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:26.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:26.897+0000 7f243f200640 1 --2- 172.21.15.196:0/1905795102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24400d15c0 0x7f24400cbeb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:26.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:26.897+0000 7f243f200640 1 --2- 172.21.15.196:0/1905795102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24400add70 0x7f244009def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:26.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:26.897+0000 7f243f200640 1 -- 172.21.15.196:0/1905795102 >> 172.21.15.196:0/1905795102 conn(0x7f244006c6c0 msgr2=0x7f2440072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:26.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:26.897+0000 7f243f200640 1 -- 172.21.15.196:0/1905795102 shutdown_connections 2024-04-28T13:33:26.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:26.897+0000 7f243f200640 1 -- 172.21.15.196:0/1905795102 wait complete. 2024-04-28T13:33:26.920 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-28T13:33:27.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.125+0000 7fd143a00640 1 Processor -- start 2024-04-28T13:33:27.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.126+0000 7fd143a00640 1 -- start start 2024-04-28T13:33:27.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.127+0000 7fd143a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd13c160950 0x7fd13c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:27.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.127+0000 7fd143a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd13c0b1ef0 0x7fd13c0cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:27.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.127+0000 7fd143a00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd13c0d07e0 con 0x7fd13c0cf430 2024-04-28T13:33:27.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.127+0000 7fd143a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd13c0cf040 con 0x7fd13c160950 2024-04-28T13:33:27.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.127+0000 7fd143a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd13c0d2090 con 0x7fd13c0b1ef0 2024-04-28T13:33:27.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.128+0000 7fd141000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd13c160950 0x7fd13c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:27.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.128+0000 7fd141000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd13c160950 0x7fd13c0cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37426/0 (socket says 172.21.15.196:37426) 2024-04-28T13:33:27.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.128+0000 7fd141000640 1 -- 172.21.15.196:0/1820535439 learned_addr learned my addr 172.21.15.196:0/1820535439 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:27.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.128+0000 7fd141000640 1 -- 172.21.15.196:0/1820535439 >> v2:172.21.15.203:3300/0 conn(0x7fd13c0b1ef0 msgr2=0x7fd13c0cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:27.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.128+0000 7fd141000640 1 --2- 172.21.15.196:0/1820535439 >> v2:172.21.15.203:3300/0 conn(0x7fd13c0b1ef0 0x7fd13c0cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:27.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.128+0000 7fd141000640 1 -- 172.21.15.196:0/1820535439 >> v1:172.21.15.203:6789/0 conn(0x7fd13c0cf430 legacy=0x7fd13c161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:27.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.128+0000 7fd141000640 1 -- 172.21.15.196:0/1820535439 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c09c700 con 0x7fd13c160950 2024-04-28T13:33:27.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.129+0000 7fd141000640 1 --2- 172.21.15.196:0/1820535439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd13c160950 0x7fd13c0cf980 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd13c162140 tx=0x7fd12c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0353f3eaec40580 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:27.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.129+0000 7fd13b400640 1 -- 172.21.15.196:0/1820535439 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c036070 con 0x7fd13c160950 2024-04-28T13:33:27.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.129+0000 7fd13b400640 1 -- 172.21.15.196:0/1820535439 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd12c031600 con 0x7fd13c160950 2024-04-28T13:33:27.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.129+0000 7fd13b400640 1 -- 172.21.15.196:0/1820535439 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0319d0 con 0x7fd13c160950 2024-04-28T13:33:27.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.130+0000 7fd143a00640 1 -- 172.21.15.196:0/1820535439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd13c160950 msgr2=0x7fd13c0cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:27.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.130+0000 7fd143a00640 1 --2- 172.21.15.196:0/1820535439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd13c160950 0x7fd13c0cf980 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd13c162140 tx=0x7fd12c02a6f0 comp rx=0 tx=0).stop 2024-04-28T13:33:27.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.130+0000 7fd143a00640 1 -- 172.21.15.196:0/1820535439 shutdown_connections 2024-04-28T13:33:27.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.130+0000 7fd143a00640 1 --2- 172.21.15.196:0/1820535439 >> v2:172.21.15.203:3300/0 conn(0x7fd13c0b1ef0 0x7fd13c0cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:27.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.130+0000 7fd143a00640 1 --2- 172.21.15.196:0/1820535439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd13c160950 0x7fd13c0cf980 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:27.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.130+0000 7fd143a00640 1 -- 172.21.15.196:0/1820535439 >> 172.21.15.196:0/1820535439 conn(0x7fd13c09ba40 msgr2=0x7fd13c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:27.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.130+0000 7fd143a00640 1 -- 172.21.15.196:0/1820535439 shutdown_connections 2024-04-28T13:33:27.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.130+0000 7fd143a00640 1 -- 172.21.15.196:0/1820535439 wait complete. 2024-04-28T13:33:27.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.132+0000 7fd143a00640 1 Processor -- start 2024-04-28T13:33:27.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.132+0000 7fd143a00640 1 -- start start 2024-04-28T13:33:27.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.133+0000 7fd143a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd13c0b1ef0 0x7fd13c15dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:27.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.133+0000 7fd143a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c0cf430 0x7fd13c15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:27.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.133+0000 7fd143a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd13c160950 0x7fd13c0c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:27.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.133+0000 7fd143a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c160d30 con 0x7fd13c0b1ef0 2024-04-28T13:33:27.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.133+0000 7fd143a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c0d1d20 con 0x7fd13c0cf430 2024-04-28T13:33:27.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.133+0000 7fd143a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd13c0b28f0 con 0x7fd13c160950 2024-04-28T13:33:27.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.133+0000 7fd141000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd13c0b1ef0 0x7fd13c15dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:27.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.133+0000 7fd141000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd13c0b1ef0 0x7fd13c15dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37442/0 (socket says 172.21.15.196:37442) 2024-04-28T13:33:27.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.133+0000 7fd141000640 1 -- 172.21.15.196:0/900739109 learned_addr learned my addr 172.21.15.196:0/900739109 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:27.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.134+0000 7fd141000640 1 -- 172.21.15.196:0/900739109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd13c160950 msgr2=0x7fd13c0c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:27.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.134+0000 7fd141000640 1 --2- 172.21.15.196:0/900739109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd13c160950 0x7fd13c0c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:27.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.134+0000 7fd141000640 1 -- 172.21.15.196:0/900739109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c0cf430 msgr2=0x7fd13c15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:27.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.134+0000 7fd141000640 1 --2- 172.21.15.196:0/900739109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c0cf430 0x7fd13c15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:27.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.134+0000 7fd141000640 1 -- 172.21.15.196:0/900739109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c15cb40 con 0x7fd13c0b1ef0 2024-04-28T13:33:27.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.134+0000 7fd141000640 1 --2- 172.21.15.196:0/900739109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd13c0b1ef0 0x7fd13c15dbc0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0093e0 tx=0x7fd12c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:27.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.134+0000 7fd138c00640 1 -- 172.21.15.196:0/900739109 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c041020 con 0x7fd13c0b1ef0 2024-04-28T13:33:27.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.134+0000 7fd138c00640 1 -- 172.21.15.196:0/900739109 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd12c007820 con 0x7fd13c0b1ef0 2024-04-28T13:33:27.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.134+0000 7fd138c00640 1 -- 172.21.15.196:0/900739109 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c036040 con 0x7fd13c0b1ef0 2024-04-28T13:33:27.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.134+0000 7fd143a00640 1 -- 172.21.15.196:0/900739109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13c15ce20 con 0x7fd13c0b1ef0 2024-04-28T13:33:27.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.134+0000 7fd143a00640 1 -- 172.21.15.196:0/900739109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13c15d6c0 con 0x7fd13c0b1ef0 2024-04-28T13:33:27.136 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.135+0000 7fd143a00640 1 -- 172.21.15.196:0/900739109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd13c06a290 con 0x7fd13c0b1ef0 2024-04-28T13:33:27.136 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.136+0000 7fd138c00640 1 -- 172.21.15.196:0/900739109 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd12c044020 con 0x7fd13c0b1ef0 2024-04-28T13:33:27.137 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.137+0000 7fd138c00640 1 --2- 172.21.15.196:0/900739109 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd11806c980 0x7fd11806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:27.137 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.137+0000 7fd138c00640 1 -- 172.21.15.196:0/900739109 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5029+0+0 (secure 0 0 0) 0x7fd12c0ac920 con 0x7fd13c0b1ef0 2024-04-28T13:33:27.137 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.138+0000 7fd13be00640 1 --2- 172.21.15.196:0/900739109 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd11806c980 0x7fd11806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:27.139 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.139+0000 7fd13be00640 1 --2- 172.21.15.196:0/900739109 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd11806c980 0x7fd11806ee40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0b1130 tx=0x7fd124006bf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:27.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.142+0000 7fd138c00640 1 -- 172.21.15.196:0/900739109 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd12c04b030 con 0x7fd13c0b1ef0 2024-04-28T13:33:27.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.339+0000 7fd143a00640 1 -- 172.21.15.196:0/900739109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd13c0b8f10 con 0x7fd13c0b1ef0 2024-04-28T13:33:27.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.346+0000 7fd138c00640 1 -- 172.21.15.196:0/900739109 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fd12c07ae80 con 0x7fd13c0b1ef0 2024-04-28T13:33:27.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.348+0000 7fd143a00640 1 -- 172.21.15.196:0/900739109 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd11806c980 msgr2=0x7fd11806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:27.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.348+0000 7fd143a00640 1 --2- 172.21.15.196:0/900739109 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd11806c980 0x7fd11806ee40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0b1130 tx=0x7fd124006bf0 comp rx=0 tx=0).stop 2024-04-28T13:33:27.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.349+0000 7fd143a00640 1 -- 172.21.15.196:0/900739109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd13c0b1ef0 msgr2=0x7fd13c15dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:27.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.349+0000 7fd143a00640 1 --2- 172.21.15.196:0/900739109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd13c0b1ef0 0x7fd13c15dbc0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0093e0 tx=0x7fd12c004a40 comp rx=0 tx=0).stop 2024-04-28T13:33:27.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.349+0000 7fd143a00640 1 -- 172.21.15.196:0/900739109 shutdown_connections 2024-04-28T13:33:27.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.349+0000 7fd143a00640 1 --2- 172.21.15.196:0/900739109 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd11806c980 0x7fd11806ee40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:27.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.349+0000 7fd143a00640 1 --2- 172.21.15.196:0/900739109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd13c160950 0x7fd13c0c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:27.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.349+0000 7fd143a00640 1 --2- 172.21.15.196:0/900739109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c0cf430 0x7fd13c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:27.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.349+0000 7fd143a00640 1 --2- 172.21.15.196:0/900739109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd13c0b1ef0 0x7fd13c15dbc0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:27.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.349+0000 7fd143a00640 1 -- 172.21.15.196:0/900739109 >> 172.21.15.196:0/900739109 conn(0x7fd13c09ba40 msgr2=0x7fd13c0afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:27.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.349+0000 7fd143a00640 1 -- 172.21.15.196:0/900739109 shutdown_connections 2024-04-28T13:33:27.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:27.349+0000 7fd143a00640 1 -- 172.21.15.196:0/900739109 wait complete. 2024-04-28T13:33:27.367 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-28T13:33:27.367 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-28T13:33:27.474 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-28T13:33:27.591 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-28T13:33:27.591 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:33:27.591 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:33:27.647 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-28T13:33:27.647 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:33:27.647 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:33:27.704 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-28T13:33:27.704 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:33:27.704 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:33:27.762 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-28T13:33:27.762 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:33:27.762 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:33:27.819 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-28T13:33:27.819 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-28T13:33:27.819 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:33:28.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.032+0000 7f27cc600640 1 Processor -- start 2024-04-28T13:33:28.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.032+0000 7f27cc600640 1 -- start start 2024-04-28T13:33:28.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.034+0000 7f27cc600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27c4162e50 0x7f27c4163290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:28.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.034+0000 7f27cc600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f27c40d0b90 0x7f27c40d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:28.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.034+0000 7f27cc600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27c4163910 con 0x7f27c4073fc0 2024-04-28T13:33:28.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.034+0000 7f27cc600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f27c4163bf0 con 0x7f27c40d0b90 2024-04-28T13:33:28.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.034+0000 7f27cc600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f27c406ea50 con 0x7f27c4162e50 2024-04-28T13:33:28.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.034+0000 7f27c9c00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f27c4073fc0 0x7f27c4072020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:54104/0 (socket says 172.21.15.196:54104) 2024-04-28T13:33:28.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.034+0000 7f27c9c00640 1 -- 172.21.15.196:0/1707598391 learned_addr learned my addr 172.21.15.196:0/1707598391 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:28.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.035+0000 7f27ca600640 1 --2- 172.21.15.196:0/1707598391 >> v2:172.21.15.203:3300/0 conn(0x7f27c40d0b90 0x7f27c40d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:28.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.035+0000 7f27c3e00640 1 -- 172.21.15.196:0/1707598391 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 922272604 0 0) 0x7f27c4163910 con 0x7f27c4073fc0 2024-04-28T13:33:28.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.035+0000 7f27c9200640 1 --2- 172.21.15.196:0/1707598391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27c4162e50 0x7f27c4163290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:28.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.035+0000 7f27c3e00640 1 -- 172.21.15.196:0/1707598391 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27ac003520 con 0x7f27c4073fc0 2024-04-28T13:33:28.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.035+0000 7f27c3e00640 1 -- 172.21.15.196:0/1707598391 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1258320246 0 0) 0x7f27ac003520 con 0x7f27c4073fc0 2024-04-28T13:33:28.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.035+0000 7f27c3e00640 1 -- 172.21.15.196:0/1707598391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27c4162e50 msgr2=0x7f27c4163290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:28.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.035+0000 7f27c3e00640 1 --2- 172.21.15.196:0/1707598391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27c4162e50 0x7f27c4163290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.035+0000 7f27c3e00640 1 -- 172.21.15.196:0/1707598391 >> v2:172.21.15.203:3300/0 conn(0x7f27c40d0b90 msgr2=0x7f27c40d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:28.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.035+0000 7f27c3e00640 1 --2- 172.21.15.196:0/1707598391 >> v2:172.21.15.203:3300/0 conn(0x7f27c40d0b90 0x7f27c40d0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.035+0000 7f27c3e00640 1 -- 172.21.15.196:0/1707598391 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c406ed30 con 0x7f27c4073fc0 2024-04-28T13:33:28.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.035+0000 7f27c3e00640 1 -- 172.21.15.196:0/1707598391 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f27b8001670 con 0x7f27c4073fc0 2024-04-28T13:33:28.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.035+0000 7f27c3e00640 1 -- 172.21.15.196:0/1707598391 >> v1:172.21.15.196:6789/0 conn(0x7f27c4073fc0 legacy=0x7f27c4072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:28.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.035+0000 7f27c3e00640 1 --2- 172.21.15.196:0/1707598391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27ac0039b0 0x7f27ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:28.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.035+0000 7f27c3e00640 1 --2- 172.21.15.196:0/1707598391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27c4162e50 0x7f27ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:28.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.036+0000 7f27c3e00640 1 --2- 172.21.15.196:0/1707598391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27ac004d40 0x7f27ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:28.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.036+0000 7f27c3e00640 1 -- 172.21.15.196:0/1707598391 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4163bf0 con 0x7f27ac004d40 2024-04-28T13:33:28.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.036+0000 7f27c3e00640 1 -- 172.21.15.196:0/1707598391 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f27c406ea50 con 0x7f27ac0039b0 2024-04-28T13:33:28.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.036+0000 7f27c3e00640 1 -- 172.21.15.196:0/1707598391 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f27ac00ba80 con 0x7f27c4162e50 2024-04-28T13:33:28.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.036+0000 7f27ca600640 1 --2- 172.21.15.196:0/1707598391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27ac004d40 0x7f27ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:28.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.036+0000 7f27c9200640 1 --2- 172.21.15.196:0/1707598391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27c4162e50 0x7f27ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:28.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.036+0000 7f27c9c00640 1 --2- 172.21.15.196:0/1707598391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27ac0039b0 0x7f27ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:28.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.036+0000 7f27ca600640 1 -- 172.21.15.196:0/1707598391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27c4162e50 msgr2=0x7f27ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:28.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.036+0000 7f27ca600640 1 --2- 172.21.15.196:0/1707598391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27c4162e50 0x7f27ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.036+0000 7f27ca600640 1 -- 172.21.15.196:0/1707598391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27ac0039b0 msgr2=0x7f27ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:28.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.036+0000 7f27ca600640 1 --2- 172.21.15.196:0/1707598391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27ac0039b0 0x7f27ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.036+0000 7f27ca600640 1 -- 172.21.15.196:0/1707598391 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ac003520 con 0x7f27ac004d40 2024-04-28T13:33:28.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.036+0000 7f27c9c00640 1 --2- 172.21.15.196:0/1707598391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27ac0039b0 0x7f27ac003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:28.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.038+0000 7f27ca600640 1 --2- 172.21.15.196:0/1707598391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27ac004d40 0x7f27ac0051c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009fc0 tx=0x7f27ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a15c7aaaf8c53d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:28.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.038+0000 7f27c3e00640 1 -- 172.21.15.196:0/1707598391 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc009c10 con 0x7f27ac004d40 2024-04-28T13:33:28.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.038+0000 7f27c3e00640 1 -- 172.21.15.196:0/1707598391 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27bc036070 con 0x7f27ac004d40 2024-04-28T13:33:28.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.038+0000 7f27c3e00640 1 -- 172.21.15.196:0/1707598391 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc02d070 con 0x7f27ac004d40 2024-04-28T13:33:28.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.039+0000 7f27cc600640 1 -- 172.21.15.196:0/1707598391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27ac004d40 msgr2=0x7f27ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:28.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.039+0000 7f27cc600640 1 --2- 172.21.15.196:0/1707598391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27ac004d40 0x7f27ac0051c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009fc0 tx=0x7f27ac005830 comp rx=0 tx=0).stop 2024-04-28T13:33:28.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.039+0000 7f27ca600640 1 -- 172.21.15.196:0/1707598391 reap_dead start 2024-04-28T13:33:28.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.039+0000 7f27cc600640 1 -- 172.21.15.196:0/1707598391 shutdown_connections 2024-04-28T13:33:28.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.039+0000 7f27cc600640 1 -- 172.21.15.196:0/1707598391 >> 172.21.15.196:0/1707598391 conn(0x7f27c40cd830 msgr2=0x7f27c40cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:28.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.039+0000 7f27cc600640 1 -- 172.21.15.196:0/1707598391 shutdown_connections 2024-04-28T13:33:28.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.039+0000 7f27cc600640 1 -- 172.21.15.196:0/1707598391 wait complete. 2024-04-28T13:33:28.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.041+0000 7f27cc600640 1 Processor -- start 2024-04-28T13:33:28.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.041+0000 7f27cc600640 1 -- start start 2024-04-28T13:33:28.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.042+0000 7f27cc600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c415ee30 0x7f27c40b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:28.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.042+0000 7f27cc600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27c415f210 0x7f27c40b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:28.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.042+0000 7f27cc600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27c40b0c00 0x7f27c40b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:28.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.042+0000 7f27cc600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27c40743a0 con 0x7f27c415ee30 2024-04-28T13:33:28.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.042+0000 7f27cc600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f27c40d1520 con 0x7f27c40b0c00 2024-04-28T13:33:28.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.042+0000 7f27cc600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f27c40b1180 con 0x7f27c415f210 2024-04-28T13:33:28.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.042+0000 7f27c9c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c415ee30 0x7f27c40b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:28.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.042+0000 7f27c9c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c415ee30 0x7f27c40b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37452/0 (socket says 172.21.15.196:37452) 2024-04-28T13:33:28.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.042+0000 7f27c9c00640 1 -- 172.21.15.196:0/1836575495 learned_addr learned my addr 172.21.15.196:0/1836575495 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:28.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.042+0000 7f27c9c00640 1 -- 172.21.15.196:0/1836575495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27c415f210 msgr2=0x7f27c40b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:28.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.042+0000 7f27c9c00640 1 --2- 172.21.15.196:0/1836575495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27c415f210 0x7f27c40b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.042+0000 7f27c9c00640 1 -- 172.21.15.196:0/1836575495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27c40b0c00 msgr2=0x7f27c40b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:28.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.042+0000 7f27c9c00640 1 --2- 172.21.15.196:0/1836575495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27c40b0c00 0x7f27c40b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.042+0000 7f27c9c00640 1 -- 172.21.15.196:0/1836575495 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c40b1460 con 0x7f27c415ee30 2024-04-28T13:33:28.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.043+0000 7f27c9c00640 1 --2- 172.21.15.196:0/1836575495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c415ee30 0x7f27c40b2220 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f27b800d8e0 tx=0x7f27b800dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:28.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.043+0000 7f27c1600640 1 -- 172.21.15.196:0/1836575495 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8019070 con 0x7f27c415ee30 2024-04-28T13:33:28.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.043+0000 7f27c1600640 1 -- 172.21.15.196:0/1836575495 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27b8004cb0 con 0x7f27c415ee30 2024-04-28T13:33:28.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.043+0000 7f27c1600640 1 -- 172.21.15.196:0/1836575495 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8004e60 con 0x7f27c415ee30 2024-04-28T13:33:28.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.043+0000 7f27cc600640 1 -- 172.21.15.196:0/1836575495 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c40b1740 con 0x7f27c415ee30 2024-04-28T13:33:28.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.043+0000 7f27cc600640 1 -- 172.21.15.196:0/1836575495 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c40b1d00 con 0x7f27c415ee30 2024-04-28T13:33:28.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.044+0000 7f27cc600640 1 -- 172.21.15.196:0/1836575495 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c406a290 con 0x7f27c415ee30 2024-04-28T13:33:28.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.045+0000 7f27c1600640 1 -- 172.21.15.196:0/1836575495 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f27b801c070 con 0x7f27c415ee30 2024-04-28T13:33:28.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.047+0000 7f27c1600640 1 --2- 172.21.15.196:0/1836575495 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f27a406c980 0x7f27a406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:28.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.047+0000 7f27c1600640 1 -- 172.21.15.196:0/1836575495 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5029+0+0 (secure 0 0 0) 0x7f27b808b6c0 con 0x7f27c415ee30 2024-04-28T13:33:28.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.047+0000 7f27c9200640 1 --2- 172.21.15.196:0/1836575495 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f27a406c980 0x7f27a406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:28.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.049+0000 7f27c9200640 1 --2- 172.21.15.196:0/1836575495 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f27a406c980 0x7f27a406ee40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f27c41f5eb0 tx=0x7f27b000b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:28.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.051+0000 7f27c1600640 1 -- 172.21.15.196:0/1836575495 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f27b8059c20 con 0x7f27c415ee30 2024-04-28T13:33:28.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.294+0000 7f27cc600640 1 -- 172.21.15.196:0/1836575495 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27c4148930 con 0x7f27c415ee30 2024-04-28T13:33:28.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.296+0000 7f27c1600640 1 -- 172.21.15.196:0/1836575495 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+3875 (secure 0 0 0) 0x7f27b8059520 con 0x7f27c415ee30 2024-04-28T13:33:28.296 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:28.296 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/931076983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":931076983},{"type":"v1","addr":"172.21.15.196:6834","nonce":931076983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1114474485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1114474485},{"type":"v1","addr":"172.21.15.196:6835","nonce":1114474485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2070123371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2070123371},{"type":"v1","addr":"172.21.15.203:6837","nonce":2070123371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4999,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/561846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":561846184},{"type":"v1","addr":"172.21.15.203:6836","nonce":561846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[]} 2024-04-28T13:33:28.296 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 24 2024-04-28T13:33:28.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.298+0000 7f27cc600640 1 -- 172.21.15.196:0/1836575495 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f27a406c980 msgr2=0x7f27a406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.298+0000 7f27cc600640 1 --2- 172.21.15.196:0/1836575495 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f27a406c980 0x7f27a406ee40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f27c41f5eb0 tx=0x7f27b000b2e0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.298+0000 7f27cc600640 1 -- 172.21.15.196:0/1836575495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c415ee30 msgr2=0x7f27c40b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.298+0000 7f27cc600640 1 --2- 172.21.15.196:0/1836575495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c415ee30 0x7f27c40b2220 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f27b800d8e0 tx=0x7f27b800dda0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.298+0000 7f27cc600640 1 -- 172.21.15.196:0/1836575495 shutdown_connections 2024-04-28T13:33:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.298+0000 7f27cc600640 1 --2- 172.21.15.196:0/1836575495 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f27a406c980 0x7f27a406ee40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.298+0000 7f27cc600640 1 --2- 172.21.15.196:0/1836575495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27c40b0c00 0x7f27c40b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.298+0000 7f27cc600640 1 --2- 172.21.15.196:0/1836575495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27c415f210 0x7f27c40b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.298+0000 7f27cc600640 1 --2- 172.21.15.196:0/1836575495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c415ee30 0x7f27c40b2220 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.298+0000 7f27cc600640 1 -- 172.21.15.196:0/1836575495 >> 172.21.15.196:0/1836575495 conn(0x7f27c40cd830 msgr2=0x7f27c40ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.298+0000 7f27cc600640 1 -- 172.21.15.196:0/1836575495 shutdown_connections 2024-04-28T13:33:28.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.299+0000 7f27cc600640 1 -- 172.21.15.196:0/1836575495 wait complete. 2024-04-28T13:33:28.320 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-28T13:33:28.320 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-28T13:33:28.320 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-28T13:33:28.320 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:33:28.320 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-28T13:33:28.321 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-28T13:33:28.321 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-28T13:33:28.321 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:33:28.321 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-28T13:33:28.321 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-28T13:33:28.321 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-28T13:33:28.321 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:33:28.322 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-28T13:33:28.322 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-28T13:33:28.322 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-28T13:33:28.322 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:33:28.342 INFO:tasks.ceph.mds.c:Stopped 2024-04-28T13:33:28.342 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-28T13:33:28.346 INFO:tasks.ceph.mds.a:Stopped 2024-04-28T13:33:28.346 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-28T13:33:28.348 INFO:tasks.ceph.mds.b:Stopped 2024-04-28T13:33:28.348 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-28T13:33:28.354 INFO:tasks.ceph.mds.d:Stopped 2024-04-28T13:33:28.355 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-28T13:33:28.393 INFO:tasks.ceph.mds.c:Started 2024-04-28T13:33:28.394 DEBUG:teuthology.parallel:result is None 2024-04-28T13:33:28.399 INFO:tasks.ceph.mds.a:Started 2024-04-28T13:33:28.400 DEBUG:teuthology.parallel:result is None 2024-04-28T13:33:28.402 INFO:tasks.ceph.mds.b:Started 2024-04-28T13:33:28.402 DEBUG:teuthology.parallel:result is None 2024-04-28T13:33:28.408 INFO:tasks.ceph.mds.d:Started 2024-04-28T13:33:28.408 DEBUG:teuthology.parallel:result is None 2024-04-28T13:33:28.409 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-28T13:33:28.542 INFO:tasks.ceph.mds.a.smithi196.stdout:starting mds.a at 2024-04-28T13:33:28.543 INFO:tasks.ceph.mds.c.smithi196.stdout:starting mds.c at 2024-04-28T13:33:28.559 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-28T13:33:28.560 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-28T13:33:28.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.583+0000 7fc9f9c00640 1 Processor -- start 2024-04-28T13:33:28.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.583+0000 7fc9f9c00640 1 -- start start 2024-04-28T13:33:28.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.584+0000 7fc9f9c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc9f4073fc0 0x7fc9f4072020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:28.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.584+0000 7fc9f9c00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc9f4162e50 0x7fc9f41632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:28.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.584+0000 7fc9f9c00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9f4072560 con 0x7fc9f40d0b90 2024-04-28T13:33:28.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.584+0000 7fc9f9c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc9f4163910 con 0x7fc9f4073fc0 2024-04-28T13:33:28.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.584+0000 7fc9f9c00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc9f4163bf0 con 0x7fc9f4162e50 2024-04-28T13:33:28.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.584+0000 7fc9f2a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc9f4073fc0 0x7fc9f4072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:28.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.584+0000 7fc9f2a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc9f4073fc0 0x7fc9f4072020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37504/0 (socket says 172.21.15.196:37504) 2024-04-28T13:33:28.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.584+0000 7fc9f2a00640 1 -- 172.21.15.196:0/4015576076 learned_addr learned my addr 172.21.15.196:0/4015576076 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:28.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.584+0000 7fc9f2000640 1 --2- 172.21.15.196:0/4015576076 >> v2:172.21.15.203:3300/0 conn(0x7fc9f4162e50 0x7fc9f41632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:28.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.585+0000 7fc9f2a00640 1 -- 172.21.15.196:0/4015576076 >> v2:172.21.15.203:3300/0 conn(0x7fc9f4162e50 msgr2=0x7fc9f41632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:28.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.585+0000 7fc9f2a00640 1 --2- 172.21.15.196:0/4015576076 >> v2:172.21.15.203:3300/0 conn(0x7fc9f4162e50 0x7fc9f41632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.585+0000 7fc9f2a00640 1 -- 172.21.15.196:0/4015576076 >> v1:172.21.15.203:6789/0 conn(0x7fc9f40d0b90 legacy=0x7fc9f40d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:28.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.585+0000 7fc9f2a00640 1 -- 172.21.15.196:0/4015576076 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f406ea50 con 0x7fc9f4073fc0 2024-04-28T13:33:28.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.586+0000 7fc9f2a00640 1 --2- 172.21.15.196:0/4015576076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f4073fc0 0x7fc9f4072020 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc9f40ce810 tx=0x7fc9e402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cbe5eadb9acc8f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:28.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.586+0000 7fc9f1600640 1 -- 172.21.15.196:0/4015576076 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4037070 con 0x7fc9f4073fc0 2024-04-28T13:33:28.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.586+0000 7fc9f1600640 1 -- 172.21.15.196:0/4015576076 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e402ae50 con 0x7fc9f4073fc0 2024-04-28T13:33:28.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.586+0000 7fc9f1600640 1 -- 172.21.15.196:0/4015576076 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4032860 con 0x7fc9f4073fc0 2024-04-28T13:33:28.587 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.587+0000 7fc9f9c00640 1 -- 172.21.15.196:0/4015576076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f4073fc0 msgr2=0x7fc9f4072020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:28.587 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.587+0000 7fc9f9c00640 1 --2- 172.21.15.196:0/4015576076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f4073fc0 0x7fc9f4072020 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc9f40ce810 tx=0x7fc9e402a300 comp rx=0 tx=0).stop 2024-04-28T13:33:28.587 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.587+0000 7fc9f9c00640 1 -- 172.21.15.196:0/4015576076 shutdown_connections 2024-04-28T13:33:28.587 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.587+0000 7fc9f9c00640 1 --2- 172.21.15.196:0/4015576076 >> v2:172.21.15.203:3300/0 conn(0x7fc9f4162e50 0x7fc9f41632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.587+0000 7fc9f9c00640 1 --2- 172.21.15.196:0/4015576076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f4073fc0 0x7fc9f4072020 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.587+0000 7fc9f9c00640 1 -- 172.21.15.196:0/4015576076 >> 172.21.15.196:0/4015576076 conn(0x7fc9f40cd830 msgr2=0x7fc9f40cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:28.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.587+0000 7fc9f9c00640 1 -- 172.21.15.196:0/4015576076 shutdown_connections 2024-04-28T13:33:28.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.587+0000 7fc9f9c00640 1 -- 172.21.15.196:0/4015576076 wait complete. 2024-04-28T13:33:28.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.589+0000 7fc9f9c00640 1 Processor -- start 2024-04-28T13:33:28.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.589+0000 7fc9f9c00640 1 -- start start 2024-04-28T13:33:28.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.590+0000 7fc9f9c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f4073fc0 0x7fc9f41eb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:28.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.590+0000 7fc9f9c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9f40d0b90 0x7fc9f415e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:28.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.590+0000 7fc9f9c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9f4162e50 0x7fc9f415e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:28.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.590+0000 7fc9f9c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f40743a0 con 0x7fc9f4073fc0 2024-04-28T13:33:28.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.590+0000 7fc9f9c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4148de0 con 0x7fc9f40d0b90 2024-04-28T13:33:28.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.590+0000 7fc9f9c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f41627e0 con 0x7fc9f4162e50 2024-04-28T13:33:28.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.590+0000 7fc9f2a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f4073fc0 0x7fc9f41eb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:28.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.590+0000 7fc9f2a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f4073fc0 0x7fc9f41eb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37506/0 (socket says 172.21.15.196:37506) 2024-04-28T13:33:28.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.590+0000 7fc9f2a00640 1 -- 172.21.15.196:0/1465069074 learned_addr learned my addr 172.21.15.196:0/1465069074 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:28.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.590+0000 7fc9f2000640 1 --2- 172.21.15.196:0/1465069074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9f40d0b90 0x7fc9f415e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:28.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.590+0000 7fc9f3400640 1 --2- 172.21.15.196:0/1465069074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9f4162e50 0x7fc9f415e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:28.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.591+0000 7fc9f2a00640 1 -- 172.21.15.196:0/1465069074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9f4162e50 msgr2=0x7fc9f415e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:28.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.591+0000 7fc9f2a00640 1 --2- 172.21.15.196:0/1465069074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9f4162e50 0x7fc9f415e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.591+0000 7fc9f2a00640 1 -- 172.21.15.196:0/1465069074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9f40d0b90 msgr2=0x7fc9f415e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:28.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.591+0000 7fc9f2a00640 1 --2- 172.21.15.196:0/1465069074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9f40d0b90 0x7fc9f415e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.591+0000 7fc9f2a00640 1 -- 172.21.15.196:0/1465069074 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f41ea1b0 con 0x7fc9f4073fc0 2024-04-28T13:33:28.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.591+0000 7fc9f2a00640 1 --2- 172.21.15.196:0/1465069074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f4073fc0 0x7fc9f41eb250 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc9e402a340 tx=0x7fc9e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:28.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.591+0000 7fc9eaa00640 1 -- 172.21.15.196:0/1465069074 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4041070 con 0x7fc9f4073fc0 2024-04-28T13:33:28.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.591+0000 7fc9eaa00640 1 -- 172.21.15.196:0/1465069074 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e403c680 con 0x7fc9f4073fc0 2024-04-28T13:33:28.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.591+0000 7fc9eaa00640 1 -- 172.21.15.196:0/1465069074 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4037040 con 0x7fc9f4073fc0 2024-04-28T13:33:28.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.591+0000 7fc9f9c00640 1 -- 172.21.15.196:0/1465069074 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f41ea490 con 0x7fc9f4073fc0 2024-04-28T13:33:28.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.592+0000 7fc9f9c00640 1 -- 172.21.15.196:0/1465069074 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f41eaa50 con 0x7fc9f4073fc0 2024-04-28T13:33:28.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.593+0000 7fc9f9c00640 1 -- 172.21.15.196:0/1465069074 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f41eb790 con 0x7fc9f4073fc0 2024-04-28T13:33:28.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.594+0000 7fc9eaa00640 1 -- 172.21.15.196:0/1465069074 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc9e4044070 con 0x7fc9f4073fc0 2024-04-28T13:33:28.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.594+0000 7fc9eaa00640 1 --2- 172.21.15.196:0/1465069074 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc9d406c980 0x7fc9d406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:28.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.595+0000 7fc9eaa00640 1 -- 172.21.15.196:0/1465069074 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5029+0+0 (secure 0 0 0) 0x7fc9e40abf20 con 0x7fc9f4073fc0 2024-04-28T13:33:28.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.595+0000 7fc9f2000640 1 --2- 172.21.15.196:0/1465069074 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc9d406c980 0x7fc9d406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:28.595 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.596+0000 7fc9f2000640 1 --2- 172.21.15.196:0/1465069074 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc9d406c980 0x7fc9d406ee40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc9f41ebb30 tx=0x7fc9e0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:28.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.599+0000 7fc9eaa00640 1 -- 172.21.15.196:0/1465069074 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc9e4079eb0 con 0x7fc9f4073fc0 2024-04-28T13:33:28.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.798+0000 7fc9f9c00640 1 -- 172.21.15.196:0/1465069074 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc9f406a290 con 0x7fc9f4073fc0 2024-04-28T13:33:28.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.918+0000 7fc9eaa00640 1 -- 172.21.15.196:0/1465069074 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) ==== 97+0+0 (secure 0 0 0) 0x7fc9e407e1b0 con 0x7fc9f4073fc0 2024-04-28T13:33:28.918 INFO:teuthology.orchestra.run.smithi196.stderr: removed all blocklist entries 2024-04-28T13:33:28.922 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.922+0000 7fc9f9c00640 1 -- 172.21.15.196:0/1465069074 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc9d406c980 msgr2=0x7fc9d406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:28.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.922+0000 7fc9f9c00640 1 --2- 172.21.15.196:0/1465069074 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc9d406c980 0x7fc9d406ee40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc9f41ebb30 tx=0x7fc9e0009190 comp rx=0 tx=0).stop 2024-04-28T13:33:28.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.922+0000 7fc9f9c00640 1 -- 172.21.15.196:0/1465069074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f4073fc0 msgr2=0x7fc9f41eb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:28.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.922+0000 7fc9f9c00640 1 --2- 172.21.15.196:0/1465069074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f4073fc0 0x7fc9f41eb250 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc9e402a340 tx=0x7fc9e40046c0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.922+0000 7fc9f9c00640 1 -- 172.21.15.196:0/1465069074 shutdown_connections 2024-04-28T13:33:28.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.922+0000 7fc9f9c00640 1 --2- 172.21.15.196:0/1465069074 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc9d406c980 0x7fc9d406ee40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.922+0000 7fc9f9c00640 1 --2- 172.21.15.196:0/1465069074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9f4162e50 0x7fc9f415e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.922+0000 7fc9f9c00640 1 --2- 172.21.15.196:0/1465069074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9f40d0b90 0x7fc9f415e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.922+0000 7fc9f9c00640 1 --2- 172.21.15.196:0/1465069074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f4073fc0 0x7fc9f41eb250 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:28.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.922+0000 7fc9f9c00640 1 -- 172.21.15.196:0/1465069074 >> 172.21.15.196:0/1465069074 conn(0x7fc9f40cd830 msgr2=0x7fc9f40d1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:28.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.922+0000 7fc9f9c00640 1 -- 172.21.15.196:0/1465069074 shutdown_connections 2024-04-28T13:33:28.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:28.923+0000 7fc9f9c00640 1 -- 172.21.15.196:0/1465069074 wait complete. 2024-04-28T13:33:28.943 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-28T13:33:29.115 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.115+0000 7f4b80a00640 1 Processor -- start 2024-04-28T13:33:29.115 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.115+0000 7f4b80a00640 1 -- start start 2024-04-28T13:33:29.116 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.116+0000 7f4b80a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7c15a9a0 0x7f4b7c0cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:29.116 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.116+0000 7f4b80a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b7c160b00 con 0x7f4b7c0add70 2024-04-28T13:33:29.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.117+0000 7f4b80a00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b7c160de0 con 0x7f4b7c0d1790 2024-04-28T13:33:29.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.117+0000 7f4b80a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4b7c1610c0 con 0x7f4b7c15a9a0 2024-04-28T13:33:29.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.117+0000 7f4b79e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f4b7c0add70 0x7f4b7c0d1080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:54116/0 (socket says 172.21.15.196:54116) 2024-04-28T13:33:29.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.117+0000 7f4b79e00640 1 -- 172.21.15.196:0/3582127876 learned_addr learned my addr 172.21.15.196:0/3582127876 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:29.118 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.117+0000 7f4b79400640 1 -- 172.21.15.196:0/3582127876 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1193317743 0 0) 0x7f4b7c160b00 con 0x7f4b7c0add70 2024-04-28T13:33:29.118 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.117+0000 7f4b79400640 1 -- 172.21.15.196:0/3582127876 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b60003520 con 0x7f4b7c0add70 2024-04-28T13:33:29.118 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.117+0000 7f4b7a800640 1 --2- 172.21.15.196:0/3582127876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7c15a9a0 0x7f4b7c0cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:29.118 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.118+0000 7f4b79400640 1 -- 172.21.15.196:0/3582127876 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2677321711 0 0) 0x7f4b60003520 con 0x7f4b7c0add70 2024-04-28T13:33:29.118 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.118+0000 7f4b79400640 1 -- 172.21.15.196:0/3582127876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7c15a9a0 msgr2=0x7f4b7c0cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:29.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.118+0000 7f4b79400640 1 --2- 172.21.15.196:0/3582127876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7c15a9a0 0x7f4b7c0cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.118+0000 7f4b79400640 1 -- 172.21.15.196:0/3582127876 >> v1:172.21.15.203:6789/0 conn(0x7f4b7c0d1790 legacy=0x7f4b7c15d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:29.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.118+0000 7f4b79400640 1 -- 172.21.15.196:0/3582127876 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7c1613a0 con 0x7f4b7c0add70 2024-04-28T13:33:29.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.118+0000 7f4b79400640 1 -- 172.21.15.196:0/3582127876 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f4b68001670 con 0x7f4b7c0add70 2024-04-28T13:33:29.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.118+0000 7f4b79400640 1 -- 172.21.15.196:0/3582127876 >> v1:172.21.15.196:6789/0 conn(0x7f4b7c0add70 legacy=0x7f4b7c0d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:29.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.118+0000 7f4b79400640 1 --2- 172.21.15.196:0/3582127876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b600039e0 0x7f4b60003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:29.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.118+0000 7f4b79400640 1 --2- 172.21.15.196:0/3582127876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b60004800 0x7f4b60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:29.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.118+0000 7f4b79400640 1 --2- 172.21.15.196:0/3582127876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7c15a9a0 0x7f4b60005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:29.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.118+0000 7f4b79400640 1 -- 172.21.15.196:0/3582127876 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c1610c0 con 0x7f4b60004800 2024-04-28T13:33:29.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.118+0000 7f4b79400640 1 -- 172.21.15.196:0/3582127876 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6000bb50 con 0x7f4b600039e0 2024-04-28T13:33:29.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.118+0000 7f4b79400640 1 -- 172.21.15.196:0/3582127876 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4b6000bd50 con 0x7f4b7c15a9a0 2024-04-28T13:33:29.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.118+0000 7f4b79e00640 1 --2- 172.21.15.196:0/3582127876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b60004800 0x7f4b60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:29.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.119+0000 7f4b7a800640 1 --2- 172.21.15.196:0/3582127876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b600039e0 0x7f4b60003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:29.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.119+0000 7f4b7b200640 1 --2- 172.21.15.196:0/3582127876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7c15a9a0 0x7f4b60005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:29.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.119+0000 7f4b79e00640 1 -- 172.21.15.196:0/3582127876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7c15a9a0 msgr2=0x7f4b60005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:29.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.119+0000 7f4b79e00640 1 --2- 172.21.15.196:0/3582127876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7c15a9a0 0x7f4b60005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.119+0000 7f4b79e00640 1 -- 172.21.15.196:0/3582127876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b600039e0 msgr2=0x7f4b60003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:29.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.119+0000 7f4b79e00640 1 --2- 172.21.15.196:0/3582127876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b600039e0 0x7f4b60003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.119+0000 7f4b79e00640 1 -- 172.21.15.196:0/3582127876 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b60003520 con 0x7f4b60004800 2024-04-28T13:33:29.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.120+0000 7f4b79e00640 1 --2- 172.21.15.196:0/3582127876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b60004800 0x7f4b60004c80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c0e0 tx=0x7f4b60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2db9aa813aaac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:29.121 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.120+0000 7f4b79400640 1 -- 172.21.15.196:0/3582127876 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b68007c20 con 0x7f4b60004800 2024-04-28T13:33:29.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.120+0000 7f4b79400640 1 -- 172.21.15.196:0/3582127876 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b68005af0 con 0x7f4b60004800 2024-04-28T13:33:29.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.120+0000 7f4b79400640 1 -- 172.21.15.196:0/3582127876 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b68007070 con 0x7f4b60004800 2024-04-28T13:33:29.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.121+0000 7f4b80a00640 1 -- 172.21.15.196:0/3582127876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b60004800 msgr2=0x7f4b60004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:29.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.121+0000 7f4b80a00640 1 --2- 172.21.15.196:0/3582127876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b60004800 0x7f4b60004c80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c0e0 tx=0x7f4b60005830 comp rx=0 tx=0).stop 2024-04-28T13:33:29.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.121+0000 7f4b7b200640 1 -- 172.21.15.196:0/3582127876 reap_dead start 2024-04-28T13:33:29.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.121+0000 7f4b80a00640 1 -- 172.21.15.196:0/3582127876 shutdown_connections 2024-04-28T13:33:29.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.121+0000 7f4b80a00640 1 -- 172.21.15.196:0/3582127876 >> 172.21.15.196:0/3582127876 conn(0x7f4b7c06c6c0 msgr2=0x7f4b7c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:29.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.121+0000 7f4b80a00640 1 -- 172.21.15.196:0/3582127876 shutdown_connections 2024-04-28T13:33:29.122 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.121+0000 7f4b80a00640 1 -- 172.21.15.196:0/3582127876 wait complete. 2024-04-28T13:33:29.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.123+0000 7f4b80a00640 1 Processor -- start 2024-04-28T13:33:29.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.123+0000 7f4b80a00640 1 -- start start 2024-04-28T13:33:29.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.124+0000 7f4b80a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b60004800 0x7f4b7c155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:29.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.124+0000 7f4b80a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b7c1e82f0 0x7f4b7c1e8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:29.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.124+0000 7f4b80a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7c09bb40 0x7f4b7c09bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:29.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.124+0000 7f4b80a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c15ad80 con 0x7f4b60004800 2024-04-28T13:33:29.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.124+0000 7f4b80a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c0d06e0 con 0x7f4b7c1e82f0 2024-04-28T13:33:29.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.124+0000 7f4b80a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4b7c09d4f0 con 0x7f4b7c09bb40 2024-04-28T13:33:29.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.125+0000 7f4b7a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b60004800 0x7f4b7c155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:29.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.125+0000 7f4b7a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b60004800 0x7f4b7c155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37530/0 (socket says 172.21.15.196:37530) 2024-04-28T13:33:29.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.125+0000 7f4b7a800640 1 -- 172.21.15.196:0/3522177201 learned_addr learned my addr 172.21.15.196:0/3522177201 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:29.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.125+0000 7f4b7b200640 1 --2- 172.21.15.196:0/3522177201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7c09bb40 0x7f4b7c09bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:29.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.125+0000 7f4b79e00640 1 --2- 172.21.15.196:0/3522177201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b7c1e82f0 0x7f4b7c1e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:29.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.125+0000 7f4b7a800640 1 -- 172.21.15.196:0/3522177201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7c09bb40 msgr2=0x7f4b7c09bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:29.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.125+0000 7f4b7a800640 1 --2- 172.21.15.196:0/3522177201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7c09bb40 0x7f4b7c09bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.125+0000 7f4b7a800640 1 -- 172.21.15.196:0/3522177201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b7c1e82f0 msgr2=0x7f4b7c1e8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:29.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.125+0000 7f4b7a800640 1 --2- 172.21.15.196:0/3522177201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b7c1e82f0 0x7f4b7c1e8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.125+0000 7f4b7a800640 1 -- 172.21.15.196:0/3522177201 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7c09c8e0 con 0x7f4b60004800 2024-04-28T13:33:29.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.125+0000 7f4b7a800640 1 --2- 172.21.15.196:0/3522177201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b60004800 0x7f4b7c155a80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00b820 tx=0x7f4b6c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:29.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.126+0000 7f4b72a00640 1 -- 172.21.15.196:0/3522177201 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c018070 con 0x7f4b60004800 2024-04-28T13:33:29.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.126+0000 7f4b72a00640 1 -- 172.21.15.196:0/3522177201 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b6c00be10 con 0x7f4b60004800 2024-04-28T13:33:29.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.126+0000 7f4b72a00640 1 -- 172.21.15.196:0/3522177201 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c010c60 con 0x7f4b60004800 2024-04-28T13:33:29.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.126+0000 7f4b80a00640 1 -- 172.21.15.196:0/3522177201 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7c09cbc0 con 0x7f4b60004800 2024-04-28T13:33:29.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.126+0000 7f4b80a00640 1 -- 172.21.15.196:0/3522177201 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b7c09d180 con 0x7f4b60004800 2024-04-28T13:33:29.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.127+0000 7f4b80a00640 1 -- 172.21.15.196:0/3522177201 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b7c09f0f0 con 0x7f4b60004800 2024-04-28T13:33:29.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.128+0000 7f4b72a00640 1 -- 172.21.15.196:0/3522177201 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4b6c01b070 con 0x7f4b60004800 2024-04-28T13:33:29.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.129+0000 7f4b72a00640 1 --2- 172.21.15.196:0/3522177201 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4b5c06c980 0x7f4b5c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:29.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.129+0000 7f4b72a00640 1 -- 172.21.15.196:0/3522177201 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f4b6c08bda0 con 0x7f4b60004800 2024-04-28T13:33:29.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.130+0000 7f4b79e00640 1 --2- 172.21.15.196:0/3522177201 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4b5c06c980 0x7f4b5c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:29.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.131+0000 7f4b79e00640 1 --2- 172.21.15.196:0/3522177201 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4b5c06c980 0x7f4b5c06ee40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c0e0 tx=0x7f4b68009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:29.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.134+0000 7f4b72a00640 1 -- 172.21.15.196:0/3522177201 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4b6c05a430 con 0x7f4b60004800 2024-04-28T13:33:29.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.410+0000 7f4b80a00640 1 -- 172.21.15.196:0/3522177201 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f4b7c153c30 con 0x7f4b60004800 2024-04-28T13:33:29.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.412+0000 7f4b72a00640 1 -- 172.21.15.196:0/3522177201 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f4b6c059d30 con 0x7f4b60004800 2024-04-28T13:33:29.412 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:29.412 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:33:29.412 INFO:teuthology.orchestra.run.smithi196.stdout: "auth_dump": [ 2024-04-28T13:33:29.412 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.412 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "mds.a", 2024-04-28T13:33:29.412 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBkTy5mGDY2OhAAO3Cl3Q176Jk7wky/cM1g7A==", 2024-04-28T13:33:29.412 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.412 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:33:29.412 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow *", 2024-04-28T13:33:29.412 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow *", 2024-04-28T13:33:29.413 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:33:29.413 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.413 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.413 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.413 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "mds.b", 2024-04-28T13:33:29.413 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBlTy5m5wRGFhAAzFebm/R1yHYctJDIlWndyQ==", 2024-04-28T13:33:29.413 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.413 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:33:29.413 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow *", 2024-04-28T13:33:29.413 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow *", 2024-04-28T13:33:29.413 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:33:29.413 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.413 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.413 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "mds.c", 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBlTy5mJqNwCRAA9EAW936ooX1fIhI8D5Y71w==", 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow *", 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow *", 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "mds.d", 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBlTy5m0Bo4JBAASWXwIPgangccm1ErTZtFxg==", 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow *", 2024-04-28T13:33:29.414 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow *", 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.0", 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBpTy5mOG48JhAAMO70X2PHMWLcrTFLBuDe1Q==", 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.1", 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBtTy5mADYZBhAARTLL+BqFjPyzcEKny+CBYg==", 2024-04-28T13:33:29.415 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.2", 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBwTy5m26gJEBAAuBn3BMPn/EZzOLzMeaaX2w==", 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.3", 2024-04-28T13:33:29.416 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBzTy5mrx1KKRAAUG56DD6uIyHP7oD0wOpRkQ==", 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.4", 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQB6Ty5mPoFDAhAA/sprBELVpeGanXkX7jCEHw==", 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.417 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.418 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.5", 2024-04-28T13:33:29.418 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQB9Ty5m77ATCxAA0Fxy75nfDOZjBxd5an+1Gg==", 2024-04-28T13:33:29.418 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.418 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:33:29.418 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:33:29.418 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:33:29.418 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.418 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.418 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.418 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.6", 2024-04-28T13:33:29.418 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQCATy5mrnQ4FhAAmKIZXHzagKbIkyof69CBTA==", 2024-04-28T13:33:29.418 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.418 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:33:29.418 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:33:29.418 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "osd.7", 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQCDTy5mSQbQIRAAUHxlY004z6danZMCClG+BQ==", 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow profile osd", 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile osd", 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.0", 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBlTy5m8movMBAAnKfzHF0pAkX1F/dZiB0qBg==", 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:33:29.419 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow r", 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.1", 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBmTy5m9BmQBhAAQh8TiIkSt7WqdfovcCbPBg==", 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow r", 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow rw", 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow rwx" 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.2", 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBlTy5mpPCYORAAPt0CzLT0pHKQTOTtE3L1xw==", 2024-04-28T13:33:29.420 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow r", 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow rw", 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow rwx" 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.3", 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBmTy5m0oq7DxAAm765AI1ow5e0KlQoBRCr/Q==", 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow", 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow r", 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow rw", 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow rwx" 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.421 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "client.admin", 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBjTy5mYyTCORAAp9GesFfEOekDdOFZC8mrhw==", 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow *", 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: "mgr": "allow *", 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow *", 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "mgr.x", 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBkTy5mV7E7MhAAtess+f/dkv11zc9Ujl2jqg==", 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow *", 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile mgr", 2024-04-28T13:33:29.422 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:33:29.423 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.423 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-28T13:33:29.423 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:29.423 INFO:teuthology.orchestra.run.smithi196.stdout: "entity": "mgr.y", 2024-04-28T13:33:29.423 INFO:teuthology.orchestra.run.smithi196.stdout: "key": "AQBkTy5mh2AXJxAA+BPc+Xi43waIQJ2YjGmw9Q==", 2024-04-28T13:33:29.423 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": { 2024-04-28T13:33:29.423 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": "allow *", 2024-04-28T13:33:29.423 INFO:teuthology.orchestra.run.smithi196.stdout: "mon": "allow profile mgr", 2024-04-28T13:33:29.423 INFO:teuthology.orchestra.run.smithi196.stdout: "osd": "allow *" 2024-04-28T13:33:29.423 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.423 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:29.423 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-28T13:33:29.423 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:33:29.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.414+0000 7f4b80a00640 1 -- 172.21.15.196:0/3522177201 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4b5c06c980 msgr2=0x7f4b5c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:29.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.414+0000 7f4b80a00640 1 --2- 172.21.15.196:0/3522177201 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4b5c06c980 0x7f4b5c06ee40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c0e0 tx=0x7f4b68009030 comp rx=0 tx=0).stop 2024-04-28T13:33:29.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.414+0000 7f4b80a00640 1 -- 172.21.15.196:0/3522177201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b60004800 msgr2=0x7f4b7c155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:29.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.414+0000 7f4b80a00640 1 --2- 172.21.15.196:0/3522177201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b60004800 0x7f4b7c155a80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00b820 tx=0x7f4b6c00bce0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.414+0000 7f4b80a00640 1 -- 172.21.15.196:0/3522177201 shutdown_connections 2024-04-28T13:33:29.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.414+0000 7f4b80a00640 1 --2- 172.21.15.196:0/3522177201 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f4b5c06c980 0x7f4b5c06ee40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.414+0000 7f4b80a00640 1 --2- 172.21.15.196:0/3522177201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7c09bb40 0x7f4b7c09bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.414+0000 7f4b80a00640 1 --2- 172.21.15.196:0/3522177201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b7c1e82f0 0x7f4b7c1e8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.414+0000 7f4b80a00640 1 --2- 172.21.15.196:0/3522177201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b60004800 0x7f4b7c155a80 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.414+0000 7f4b80a00640 1 -- 172.21.15.196:0/3522177201 >> 172.21.15.196:0/3522177201 conn(0x7f4b7c06c6c0 msgr2=0x7f4b7c15c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:29.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.415+0000 7f4b80a00640 1 -- 172.21.15.196:0/3522177201 shutdown_connections 2024-04-28T13:33:29.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.415+0000 7f4b80a00640 1 -- 172.21.15.196:0/3522177201 wait complete. 2024-04-28T13:33:29.433 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:33:29.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.636+0000 7fbba7200640 1 Processor -- start 2024-04-28T13:33:29.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.636+0000 7fbba7200640 1 -- start start 2024-04-28T13:33:29.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.638+0000 7fbba7200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbba80d0540 0x7fbba80af560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:29.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.638+0000 7fbba7200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbba80b3c20 0x7fbba80b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:29.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.638+0000 7fbba7200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbba80afaa0 con 0x7fbba8073370 2024-04-28T13:33:29.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.638+0000 7fbba7200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbba80b47a0 con 0x7fbba80d0540 2024-04-28T13:33:29.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.638+0000 7fbba7200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbba80738e0 con 0x7fbba80b3c20 2024-04-28T13:33:29.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.638+0000 7fbba5e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbba80d0540 0x7fbba80af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:29.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.638+0000 7fbba5e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbba80d0540 0x7fbba80af560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37546/0 (socket says 172.21.15.196:37546) 2024-04-28T13:33:29.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.638+0000 7fbba5e00640 1 -- 172.21.15.196:0/427963575 learned_addr learned my addr 172.21.15.196:0/427963575 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:29.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.638+0000 7fbba5400640 1 --1- 172.21.15.196:0/427963575 >> v1:172.21.15.196:6789/0 conn(0x7fbba8073370 0x7fbba8072900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:54130/0 (socket says 172.21.15.196:54130) 2024-04-28T13:33:29.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.639+0000 7fbba6800640 1 --2- 172.21.15.196:0/427963575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbba80b3c20 0x7fbba80b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:29.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.639+0000 7fbba5e00640 1 -- 172.21.15.196:0/427963575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbba80b3c20 msgr2=0x7fbba80b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:29.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.639+0000 7fbba5e00640 1 --2- 172.21.15.196:0/427963575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbba80b3c20 0x7fbba80b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.639+0000 7fbba5e00640 1 -- 172.21.15.196:0/427963575 >> v1:172.21.15.196:6789/0 conn(0x7fbba8073370 legacy=0x7fbba8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:29.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.639+0000 7fbba5e00640 1 -- 172.21.15.196:0/427963575 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba80aea80 con 0x7fbba80d0540 2024-04-28T13:33:29.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.639+0000 7fbba4a00640 1 -- 172.21.15.196:0/427963575 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2823054302 0 0) 0x7fbba80afaa0 con 0x7fbba8073370 2024-04-28T13:33:29.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.639+0000 7fbba6800640 1 --2- 172.21.15.196:0/427963575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbba80b3c20 0x7fbba80b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:29.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.640+0000 7fbba5e00640 1 --2- 172.21.15.196:0/427963575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbba80d0540 0x7fbba80af560 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009870 tx=0x7fbb9802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a86b6e7dac5519 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:29.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.640+0000 7fbba4a00640 1 -- 172.21.15.196:0/427963575 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98037070 con 0x7fbba80d0540 2024-04-28T13:33:29.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.640+0000 7fbba4a00640 1 -- 172.21.15.196:0/427963575 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb98032440 con 0x7fbba80d0540 2024-04-28T13:33:29.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.640+0000 7fbba4a00640 1 -- 172.21.15.196:0/427963575 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98032810 con 0x7fbba80d0540 2024-04-28T13:33:29.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.641+0000 7fbba7200640 1 -- 172.21.15.196:0/427963575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbba80d0540 msgr2=0x7fbba80af560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:29.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.641+0000 7fbba7200640 1 --2- 172.21.15.196:0/427963575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbba80d0540 0x7fbba80af560 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009870 tx=0x7fbb9802a250 comp rx=0 tx=0).stop 2024-04-28T13:33:29.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.641+0000 7fbba7200640 1 -- 172.21.15.196:0/427963575 shutdown_connections 2024-04-28T13:33:29.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.641+0000 7fbba7200640 1 --2- 172.21.15.196:0/427963575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbba80b3c20 0x7fbba80b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.641+0000 7fbba7200640 1 --2- 172.21.15.196:0/427963575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbba80d0540 0x7fbba80af560 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.641+0000 7fbba7200640 1 -- 172.21.15.196:0/427963575 >> 172.21.15.196:0/427963575 conn(0x7fbba806c6c0 msgr2=0x7fbba806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:29.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.641+0000 7fbba7200640 1 -- 172.21.15.196:0/427963575 shutdown_connections 2024-04-28T13:33:29.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.641+0000 7fbba7200640 1 -- 172.21.15.196:0/427963575 wait complete. 2024-04-28T13:33:29.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.642+0000 7fbba7200640 1 Processor -- start 2024-04-28T13:33:29.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.643+0000 7fbba7200640 1 -- start start 2024-04-28T13:33:29.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba7200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbba8073370 0x7fbba8156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:29.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba7200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbba80b3c20 0x7fbba8149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:29.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba7200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbba80d0540 0x7fbba809d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:29.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba7200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbba80d0920 con 0x7fbba80b3c20 2024-04-28T13:33:29.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba7200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbba80af040 con 0x7fbba80d0540 2024-04-28T13:33:29.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba7200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbba815ddc0 con 0x7fbba8073370 2024-04-28T13:33:29.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba5400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbba80b3c20 0x7fbba8149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:29.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba5400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbba80b3c20 0x7fbba8149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37550/0 (socket says 172.21.15.196:37550) 2024-04-28T13:33:29.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba5400640 1 -- 172.21.15.196:0/328015413 learned_addr learned my addr 172.21.15.196:0/328015413 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:29.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba5e00640 1 --2- 172.21.15.196:0/328015413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbba8073370 0x7fbba8156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:29.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba5400640 1 -- 172.21.15.196:0/328015413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbba8073370 msgr2=0x7fbba8156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:29.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba5400640 1 --2- 172.21.15.196:0/328015413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbba8073370 0x7fbba8156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba5400640 1 -- 172.21.15.196:0/328015413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbba80d0540 msgr2=0x7fbba809d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:29.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba6800640 1 --2- 172.21.15.196:0/328015413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbba80d0540 0x7fbba809d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:29.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba5400640 1 --2- 172.21.15.196:0/328015413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbba80d0540 0x7fbba809d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba5400640 1 -- 172.21.15.196:0/328015413 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba81f60e0 con 0x7fbba80b3c20 2024-04-28T13:33:29.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.644+0000 7fbba5e00640 1 --2- 172.21.15.196:0/328015413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbba8073370 0x7fbba8156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:29.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.645+0000 7fbba5400640 1 --2- 172.21.15.196:0/328015413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbba80b3c20 0x7fbba8149640 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400c430 tx=0x7fbb9400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:29.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.645+0000 7fbb9e000640 1 -- 172.21.15.196:0/328015413 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb940074e0 con 0x7fbba80b3c20 2024-04-28T13:33:29.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.645+0000 7fbb9e000640 1 -- 172.21.15.196:0/328015413 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb940047e0 con 0x7fbba80b3c20 2024-04-28T13:33:29.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.645+0000 7fbb9e000640 1 -- 172.21.15.196:0/328015413 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94004bd0 con 0x7fbba80b3c20 2024-04-28T13:33:29.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.645+0000 7fbba7200640 1 -- 172.21.15.196:0/328015413 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba815e080 con 0x7fbba80b3c20 2024-04-28T13:33:29.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.645+0000 7fbba7200640 1 -- 172.21.15.196:0/328015413 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba809dad0 con 0x7fbba80b3c20 2024-04-28T13:33:29.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.647+0000 7fbba7200640 1 -- 172.21.15.196:0/328015413 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba806a290 con 0x7fbba80b3c20 2024-04-28T13:33:29.647 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.647+0000 7fbb9e000640 1 -- 172.21.15.196:0/328015413 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fbb94011070 con 0x7fbba80b3c20 2024-04-28T13:33:29.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.648+0000 7fbb9e000640 1 --2- 172.21.15.196:0/328015413 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbb8806c980 0x7fbb8806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:29.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.648+0000 7fbb9e000640 1 -- 172.21.15.196:0/328015413 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7fbb9408b510 con 0x7fbba80b3c20 2024-04-28T13:33:29.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.649+0000 7fbba5e00640 1 --2- 172.21.15.196:0/328015413 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbb8806c980 0x7fbb8806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:29.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.650+0000 7fbba5e00640 1 --2- 172.21.15.196:0/328015413 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbb8806c980 0x7fbb8806ee40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbb98034000 tx=0x7fbb9802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:29.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.653+0000 7fbb9e000640 1 -- 172.21.15.196:0/328015413 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb94059ca0 con 0x7fbba80b3c20 2024-04-28T13:33:29.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.852+0000 7fbba7200640 1 -- 172.21.15.196:0/328015413 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbba80cf750 con 0x7fbba80b3c20 2024-04-28T13:33:29.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.854+0000 7fbb9e000640 1 -- 172.21.15.196:0/328015413 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+12927 (secure 0 0 0) 0x7fbb940595a0 con 0x7fbba80b3c20 2024-04-28T13:33:29.854 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:29.854 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":41,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:33:28.930558+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:33:29.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.856+0000 7fbba7200640 1 -- 172.21.15.196:0/328015413 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbb8806c980 msgr2=0x7fbb8806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:29.856 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.856+0000 7fbba7200640 1 --2- 172.21.15.196:0/328015413 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbb8806c980 0x7fbb8806ee40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbb98034000 tx=0x7fbb9802e000 comp rx=0 tx=0).stop 2024-04-28T13:33:29.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.856+0000 7fbba7200640 1 -- 172.21.15.196:0/328015413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbba80b3c20 msgr2=0x7fbba8149640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:29.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.856+0000 7fbba7200640 1 --2- 172.21.15.196:0/328015413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbba80b3c20 0x7fbba8149640 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400c430 tx=0x7fbb9400c8f0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.856+0000 7fbba7200640 1 -- 172.21.15.196:0/328015413 shutdown_connections 2024-04-28T13:33:29.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.856+0000 7fbba7200640 1 --2- 172.21.15.196:0/328015413 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbb8806c980 0x7fbb8806ee40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.856+0000 7fbba7200640 1 --2- 172.21.15.196:0/328015413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbba80d0540 0x7fbba809d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.856+0000 7fbba7200640 1 --2- 172.21.15.196:0/328015413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbba80b3c20 0x7fbba8149640 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.856+0000 7fbba7200640 1 --2- 172.21.15.196:0/328015413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbba8073370 0x7fbba8156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:29.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.856+0000 7fbba7200640 1 -- 172.21.15.196:0/328015413 >> 172.21.15.196:0/328015413 conn(0x7fbba806c6c0 msgr2=0x7fbba806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:29.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.856+0000 7fbba7200640 1 -- 172.21.15.196:0/328015413 shutdown_connections 2024-04-28T13:33:29.857 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:29.857+0000 7fbba7200640 1 -- 172.21.15.196:0/328015413 wait complete. 2024-04-28T13:33:29.877 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:33:29.877 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:33:30.083 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.083+0000 7fbf62600640 1 Processor -- start 2024-04-28T13:33:30.083 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.083+0000 7fbf62600640 1 -- start start 2024-04-28T13:33:30.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.085+0000 7fbf62600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf5c0d0300 0x7fbf5c0b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.085+0000 7fbf62600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf5c0b46a0 con 0x7fbf5c06f340 2024-04-28T13:33:30.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.085+0000 7fbf62600640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf5c0b5db0 con 0x7fbf5c0b3e50 2024-04-28T13:33:30.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.085+0000 7fbf62600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbf5c0b6090 con 0x7fbf5c0d0300 2024-04-28T13:33:30.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.085+0000 7fbf5b400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fbf5c06f340 0x7fbf5c06f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:54146/0 (socket says 172.21.15.196:54146) 2024-04-28T13:33:30.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.085+0000 7fbf5b400640 1 -- 172.21.15.196:0/3394313887 learned_addr learned my addr 172.21.15.196:0/3394313887 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:30.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.085+0000 7fbf5a000640 1 -- 172.21.15.196:0/3394313887 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2994581182 0 0) 0x7fbf5c0b46a0 con 0x7fbf5c06f340 2024-04-28T13:33:30.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.085+0000 7fbf5aa00640 1 --2- 172.21.15.196:0/3394313887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf5c0d0300 0x7fbf5c0b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.086+0000 7fbf5a000640 1 -- 172.21.15.196:0/3394313887 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf40005730 con 0x7fbf5c06f340 2024-04-28T13:33:30.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.086+0000 7fbf5a000640 1 -- 172.21.15.196:0/3394313887 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3178204544 0 0) 0x7fbf40005730 con 0x7fbf5c06f340 2024-04-28T13:33:30.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.086+0000 7fbf5a000640 1 -- 172.21.15.196:0/3394313887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf5c0d0300 msgr2=0x7fbf5c0b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:30.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.086+0000 7fbf5a000640 1 --2- 172.21.15.196:0/3394313887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf5c0d0300 0x7fbf5c0b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.086+0000 7fbf5a000640 1 -- 172.21.15.196:0/3394313887 >> v1:172.21.15.203:6789/0 conn(0x7fbf5c0b3e50 legacy=0x7fbf5c0720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.086+0000 7fbf5a000640 1 -- 172.21.15.196:0/3394313887 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c0b6370 con 0x7fbf5c06f340 2024-04-28T13:33:30.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.086+0000 7fbf5a000640 1 -- 172.21.15.196:0/3394313887 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fbf4c001670 con 0x7fbf5c06f340 2024-04-28T13:33:30.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.086+0000 7fbf5a000640 1 -- 172.21.15.196:0/3394313887 >> v1:172.21.15.196:6789/0 conn(0x7fbf5c06f340 legacy=0x7fbf5c06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.086+0000 7fbf5a000640 1 --2- 172.21.15.196:0/3394313887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf5c0d0300 0x7fbf40005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.086+0000 7fbf5a000640 1 --2- 172.21.15.196:0/3394313887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf400061c0 0x7fbf40006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.086+0000 7fbf5a000640 1 --2- 172.21.15.196:0/3394313887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf40006f50 0x7fbf400073b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.086+0000 7fbf5a000640 1 -- 172.21.15.196:0/3394313887 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c0b6090 con 0x7fbf400061c0 2024-04-28T13:33:30.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.086+0000 7fbf5a000640 1 -- 172.21.15.196:0/3394313887 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4000de60 con 0x7fbf40006f50 2024-04-28T13:33:30.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.086+0000 7fbf5a000640 1 -- 172.21.15.196:0/3394313887 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4000e060 con 0x7fbf5c0d0300 2024-04-28T13:33:30.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.086+0000 7fbf5aa00640 1 --2- 172.21.15.196:0/3394313887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf400061c0 0x7fbf40006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.087+0000 7fbf5b400640 1 --2- 172.21.15.196:0/3394313887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf5c0d0300 0x7fbf40005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.087+0000 7fbf5aa00640 1 -- 172.21.15.196:0/3394313887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf5c0d0300 msgr2=0x7fbf40005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:30.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.087+0000 7fbf5aa00640 1 --2- 172.21.15.196:0/3394313887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf5c0d0300 0x7fbf40005bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.087+0000 7fbf5be00640 1 --2- 172.21.15.196:0/3394313887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf40006f50 0x7fbf400073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.087+0000 7fbf5aa00640 1 -- 172.21.15.196:0/3394313887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf40006f50 msgr2=0x7fbf400073b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:30.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.087+0000 7fbf5aa00640 1 --2- 172.21.15.196:0/3394313887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf40006f50 0x7fbf400073b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.089 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.087+0000 7fbf5aa00640 1 -- 172.21.15.196:0/3394313887 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf40005730 con 0x7fbf400061c0 2024-04-28T13:33:30.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.088+0000 7fbf5aa00640 1 --2- 172.21.15.196:0/3394313887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf400061c0 0x7fbf40006a10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009ac0 tx=0x7fbf40007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a23399894e38a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:30.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.088+0000 7fbf5a000640 1 -- 172.21.15.196:0/3394313887 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48036070 con 0x7fbf400061c0 2024-04-28T13:33:30.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.088+0000 7fbf5a000640 1 -- 172.21.15.196:0/3394313887 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf48031400 con 0x7fbf400061c0 2024-04-28T13:33:30.090 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.088+0000 7fbf5a000640 1 -- 172.21.15.196:0/3394313887 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48009690 con 0x7fbf400061c0 2024-04-28T13:33:30.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.089+0000 7fbf62600640 1 -- 172.21.15.196:0/3394313887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf400061c0 msgr2=0x7fbf40006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.089+0000 7fbf62600640 1 --2- 172.21.15.196:0/3394313887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf400061c0 0x7fbf40006a10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009ac0 tx=0x7fbf40007a40 comp rx=0 tx=0).stop 2024-04-28T13:33:30.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.089+0000 7fbf5be00640 1 -- 172.21.15.196:0/3394313887 reap_dead start 2024-04-28T13:33:30.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.089+0000 7fbf62600640 1 -- 172.21.15.196:0/3394313887 shutdown_connections 2024-04-28T13:33:30.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.089+0000 7fbf62600640 1 -- 172.21.15.196:0/3394313887 >> 172.21.15.196:0/3394313887 conn(0x7fbf5c0cd8a0 msgr2=0x7fbf5c0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:30.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.089+0000 7fbf62600640 1 -- 172.21.15.196:0/3394313887 shutdown_connections 2024-04-28T13:33:30.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.089+0000 7fbf62600640 1 -- 172.21.15.196:0/3394313887 wait complete. 2024-04-28T13:33:30.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.090+0000 7fbf62600640 1 Processor -- start 2024-04-28T13:33:30.091 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.091+0000 7fbf62600640 1 -- start start 2024-04-28T13:33:30.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.091+0000 7fbf62600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf400061c0 0x7fbf5c1edfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.091+0000 7fbf62600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf5c143f40 0x7fbf5c1443c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.092+0000 7fbf62600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf5c1f0cd0 0x7fbf5c1f1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.092+0000 7fbf62600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c0b4460 con 0x7fbf5c1f0cd0 2024-04-28T13:33:30.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.092+0000 7fbf62600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c0b42e0 con 0x7fbf5c143f40 2024-04-28T13:33:30.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.092+0000 7fbf62600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbf5c142a30 con 0x7fbf400061c0 2024-04-28T13:33:30.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.092+0000 7fbf5be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf5c1f0cd0 0x7fbf5c1f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.092+0000 7fbf5be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf5c1f0cd0 0x7fbf5c1f1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37570/0 (socket says 172.21.15.196:37570) 2024-04-28T13:33:30.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.092+0000 7fbf5be00640 1 -- 172.21.15.196:0/2042030702 learned_addr learned my addr 172.21.15.196:0/2042030702 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:30.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.092+0000 7fbf5b400640 1 --2- 172.21.15.196:0/2042030702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf400061c0 0x7fbf5c1edfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.092+0000 7fbf5aa00640 1 --2- 172.21.15.196:0/2042030702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf5c143f40 0x7fbf5c1443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.092+0000 7fbf5be00640 1 -- 172.21.15.196:0/2042030702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf400061c0 msgr2=0x7fbf5c1edfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.092+0000 7fbf5be00640 1 --2- 172.21.15.196:0/2042030702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf400061c0 0x7fbf5c1edfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.092+0000 7fbf5be00640 1 -- 172.21.15.196:0/2042030702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf5c143f40 msgr2=0x7fbf5c1443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.092+0000 7fbf5be00640 1 --2- 172.21.15.196:0/2042030702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf5c143f40 0x7fbf5c1443c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.092+0000 7fbf5be00640 1 -- 172.21.15.196:0/2042030702 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c153e30 con 0x7fbf5c1f0cd0 2024-04-28T13:33:30.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.092+0000 7fbf5b400640 1 --2- 172.21.15.196:0/2042030702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf400061c0 0x7fbf5c1edfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:30.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.093+0000 7fbf5be00640 1 --2- 172.21.15.196:0/2042030702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf5c1f0cd0 0x7fbf5c1f1160 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbf540012e0 tx=0x7fbf5400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:30.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.093+0000 7fbf53400640 1 -- 172.21.15.196:0/2042030702 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54018070 con 0x7fbf5c1f0cd0 2024-04-28T13:33:30.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.093+0000 7fbf53400640 1 -- 172.21.15.196:0/2042030702 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf54003b90 con 0x7fbf5c1f0cd0 2024-04-28T13:33:30.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.093+0000 7fbf62600640 1 -- 172.21.15.196:0/2042030702 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5c1540a0 con 0x7fbf5c1f0cd0 2024-04-28T13:33:30.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.093+0000 7fbf53400640 1 -- 172.21.15.196:0/2042030702 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54005070 con 0x7fbf5c1f0cd0 2024-04-28T13:33:30.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.093+0000 7fbf62600640 1 -- 172.21.15.196:0/2042030702 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5c154660 con 0x7fbf5c1f0cd0 2024-04-28T13:33:30.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.094+0000 7fbf62600640 1 -- 172.21.15.196:0/2042030702 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf5c06a290 con 0x7fbf5c1f0cd0 2024-04-28T13:33:30.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.095+0000 7fbf53400640 1 -- 172.21.15.196:0/2042030702 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fbf5401b070 con 0x7fbf5c1f0cd0 2024-04-28T13:33:30.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.096+0000 7fbf53400640 1 --2- 172.21.15.196:0/2042030702 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbf3c06c980 0x7fbf3c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.096+0000 7fbf53400640 1 -- 172.21.15.196:0/2042030702 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7fbf5408b1c0 con 0x7fbf5c1f0cd0 2024-04-28T13:33:30.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.097+0000 7fbf5b400640 1 --2- 172.21.15.196:0/2042030702 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbf3c06c980 0x7fbf3c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.098+0000 7fbf5b400640 1 --2- 172.21.15.196:0/2042030702 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbf3c06c980 0x7fbf3c06ee40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c001eb0 tx=0x7fbf4c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:30.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.100+0000 7fbf53400640 1 -- 172.21.15.196:0/2042030702 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbf54059740 con 0x7fbf5c1f0cd0 2024-04-28T13:33:30.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.304+0000 7fbf62600640 1 -- 172.21.15.196:0/2042030702 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf5c072270 con 0x7fbf5c1f0cd0 2024-04-28T13:33:30.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.304+0000 7fbf53400640 1 -- 172.21.15.196:0/2042030702 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7fbf54059040 con 0x7fbf5c1f0cd0 2024-04-28T13:33:30.304 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:33:30.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.307+0000 7fbf62600640 1 -- 172.21.15.196:0/2042030702 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbf3c06c980 msgr2=0x7fbf3c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.307+0000 7fbf62600640 1 --2- 172.21.15.196:0/2042030702 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbf3c06c980 0x7fbf3c06ee40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c001eb0 tx=0x7fbf4c009040 comp rx=0 tx=0).stop 2024-04-28T13:33:30.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.307+0000 7fbf62600640 1 -- 172.21.15.196:0/2042030702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf5c1f0cd0 msgr2=0x7fbf5c1f1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.307+0000 7fbf62600640 1 --2- 172.21.15.196:0/2042030702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf5c1f0cd0 0x7fbf5c1f1160 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbf540012e0 tx=0x7fbf5400dc00 comp rx=0 tx=0).stop 2024-04-28T13:33:30.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.307+0000 7fbf62600640 1 -- 172.21.15.196:0/2042030702 shutdown_connections 2024-04-28T13:33:30.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.307+0000 7fbf62600640 1 --2- 172.21.15.196:0/2042030702 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbf3c06c980 0x7fbf3c06ee40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.307+0000 7fbf62600640 1 --2- 172.21.15.196:0/2042030702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf5c1f0cd0 0x7fbf5c1f1160 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.307+0000 7fbf62600640 1 --2- 172.21.15.196:0/2042030702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf5c143f40 0x7fbf5c1443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.307 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.307+0000 7fbf62600640 1 --2- 172.21.15.196:0/2042030702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf400061c0 0x7fbf5c1edfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.307+0000 7fbf62600640 1 -- 172.21.15.196:0/2042030702 >> 172.21.15.196:0/2042030702 conn(0x7fbf5c0cd8a0 msgr2=0x7fbf5c0cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:30.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.307+0000 7fbf62600640 1 -- 172.21.15.196:0/2042030702 shutdown_connections 2024-04-28T13:33:30.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.307+0000 7fbf62600640 1 -- 172.21.15.196:0/2042030702 wait complete. 2024-04-28T13:33:30.326 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-28T13:33:30.513 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.513+0000 7f6149e00640 1 Processor -- start 2024-04-28T13:33:30.513 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.513+0000 7f6149e00640 1 -- start start 2024-04-28T13:33:30.515 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.514+0000 7f6149e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f61440add70 0x7f61440d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.515 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.514+0000 7f6149e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6144160b00 con 0x7f61440d15c0 2024-04-28T13:33:30.515 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.515+0000 7f6149e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6144160de0 con 0x7f614415a9a0 2024-04-28T13:33:30.515 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.515+0000 7f6149e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f61440d1a20 con 0x7f61440add70 2024-04-28T13:33:30.515 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.515+0000 7f6143e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f61440add70 0x7f61440d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.515 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.515+0000 7f6143e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f61440add70 0x7f61440d1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37584/0 (socket says 172.21.15.196:37584) 2024-04-28T13:33:30.515 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.515+0000 7f6149400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f61440d15c0 0x7f614415d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:54162/0 (socket says 172.21.15.196:54162) 2024-04-28T13:33:30.515 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.515+0000 7f6143e00640 1 -- 172.21.15.196:0/2183266855 learned_addr learned my addr 172.21.15.196:0/2183266855 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:30.516 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.516+0000 7f6143e00640 1 -- 172.21.15.196:0/2183266855 >> v1:172.21.15.203:6789/0 conn(0x7f614415a9a0 legacy=0x7f61440cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.516 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.516+0000 7f6143e00640 1 -- 172.21.15.196:0/2183266855 >> v1:172.21.15.196:6789/0 conn(0x7f61440d15c0 legacy=0x7f614415d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.516 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.516+0000 7f6143e00640 1 -- 172.21.15.196:0/2183266855 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61441610c0 con 0x7f61440add70 2024-04-28T13:33:30.516 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.516+0000 7f6143400640 1 -- 172.21.15.196:0/2183266855 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2735399971 0 0) 0x7f6144160b00 con 0x7f61440d15c0 2024-04-28T13:33:30.516 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.517+0000 7f6143e00640 1 --2- 172.21.15.196:0/2183266855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61440add70 0x7f61440d1080 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6130009870 tx=0x7f613002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5571e250f9bc3ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:30.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.517+0000 7f6143400640 1 -- 172.21.15.196:0/2183266855 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130037070 con 0x7f61440add70 2024-04-28T13:33:30.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.517+0000 7f6143400640 1 -- 172.21.15.196:0/2183266855 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613002ae50 con 0x7f61440add70 2024-04-28T13:33:30.517 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.517+0000 7f6143400640 1 -- 172.21.15.196:0/2183266855 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130032860 con 0x7f61440add70 2024-04-28T13:33:30.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.518+0000 7f6149e00640 1 -- 172.21.15.196:0/2183266855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61440add70 msgr2=0x7f61440d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.518+0000 7f6149e00640 1 --2- 172.21.15.196:0/2183266855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61440add70 0x7f61440d1080 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6130009870 tx=0x7f613002a300 comp rx=0 tx=0).stop 2024-04-28T13:33:30.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.518+0000 7f6149e00640 1 -- 172.21.15.196:0/2183266855 shutdown_connections 2024-04-28T13:33:30.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.518+0000 7f6149e00640 1 --2- 172.21.15.196:0/2183266855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61440add70 0x7f61440d1080 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.518+0000 7f6149e00640 1 -- 172.21.15.196:0/2183266855 >> 172.21.15.196:0/2183266855 conn(0x7f614406c6c0 msgr2=0x7f614406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:30.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.518+0000 7f6149e00640 1 -- 172.21.15.196:0/2183266855 shutdown_connections 2024-04-28T13:33:30.518 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.518+0000 7f6149e00640 1 -- 172.21.15.196:0/2183266855 wait complete. 2024-04-28T13:33:30.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.520+0000 7f6149e00640 1 Processor -- start 2024-04-28T13:33:30.519 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.520+0000 7f6149e00640 1 -- start start 2024-04-28T13:33:30.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.521+0000 7f6149e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61440add70 0x7f614414b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.521 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.521+0000 7f6149e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61440d15c0 0x7f614409d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.521+0000 7f6149e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f614415a9a0 0x7f614409df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.521+0000 7f6149e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61440d06e0 con 0x7f614415a9a0 2024-04-28T13:33:30.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.521+0000 7f6149e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f61441618c0 con 0x7f61440d15c0 2024-04-28T13:33:30.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.521+0000 7f6149e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f614415ad80 con 0x7f61440add70 2024-04-28T13:33:30.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.521+0000 7f6149400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f614415a9a0 0x7f614409df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.521+0000 7f6149400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f614415a9a0 0x7f614409df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37594/0 (socket says 172.21.15.196:37594) 2024-04-28T13:33:30.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.521+0000 7f6149400640 1 -- 172.21.15.196:0/4077644633 learned_addr learned my addr 172.21.15.196:0/4077644633 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:30.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.521+0000 7f6148a00640 1 --2- 172.21.15.196:0/4077644633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61440add70 0x7f614414b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.521+0000 7f6143e00640 1 --2- 172.21.15.196:0/4077644633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61440d15c0 0x7f614409d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.522+0000 7f6149400640 1 -- 172.21.15.196:0/4077644633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61440add70 msgr2=0x7f614414b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.522+0000 7f6149400640 1 --2- 172.21.15.196:0/4077644633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61440add70 0x7f614414b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.522+0000 7f6149400640 1 -- 172.21.15.196:0/4077644633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61440d15c0 msgr2=0x7f614409d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.522+0000 7f6149400640 1 --2- 172.21.15.196:0/4077644633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61440d15c0 0x7f614409d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.523 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.522+0000 7f6149400640 1 -- 172.21.15.196:0/4077644633 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614409bf40 con 0x7f614415a9a0 2024-04-28T13:33:30.523 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.522+0000 7f6143e00640 1 --2- 172.21.15.196:0/4077644633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61440d15c0 0x7f614409d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:30.523 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.522+0000 7f6149400640 1 --2- 172.21.15.196:0/4077644633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f614415a9a0 0x7f614409df30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f613c00c730 tx=0x7f613c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:30.523 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.522+0000 7f6140c00640 1 -- 172.21.15.196:0/4077644633 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c018070 con 0x7f614415a9a0 2024-04-28T13:33:30.524 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.522+0000 7f6140c00640 1 -- 172.21.15.196:0/4077644633 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613c00ce40 con 0x7f614415a9a0 2024-04-28T13:33:30.524 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.522+0000 7f6149e00640 1 -- 172.21.15.196:0/4077644633 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614409c220 con 0x7f614415a9a0 2024-04-28T13:33:30.524 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.522+0000 7f6140c00640 1 -- 172.21.15.196:0/4077644633 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c00a860 con 0x7f614415a9a0 2024-04-28T13:33:30.524 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.522+0000 7f6149e00640 1 -- 172.21.15.196:0/4077644633 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f614409c7e0 con 0x7f614415a9a0 2024-04-28T13:33:30.524 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.524+0000 7f6149e00640 1 -- 172.21.15.196:0/4077644633 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614409cd00 con 0x7f614415a9a0 2024-04-28T13:33:30.524 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.525+0000 7f6140c00640 1 -- 172.21.15.196:0/4077644633 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f613c01b070 con 0x7f614415a9a0 2024-04-28T13:33:30.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.526+0000 7f6140c00640 1 --2- 172.21.15.196:0/4077644633 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f612406c9d0 0x7f612406ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.526+0000 7f6140c00640 1 -- 172.21.15.196:0/4077644633 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f613c08aee0 con 0x7f614415a9a0 2024-04-28T13:33:30.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.527+0000 7f6148a00640 1 --2- 172.21.15.196:0/4077644633 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f612406c9d0 0x7f612406ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.527 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.528+0000 7f6148a00640 1 --2- 172.21.15.196:0/4077644633 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f612406c9d0 0x7f612406ee90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6138000f60 tx=0x7f6138009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:30.530 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.531+0000 7f6140c00640 1 -- 172.21.15.196:0/4077644633 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f613c0596f0 con 0x7f614415a9a0 2024-04-28T13:33:30.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.732+0000 7f6149e00640 1 -- 172.21.15.196:0/4077644633 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f614409f0f0 con 0x7f614415a9a0 2024-04-28T13:33:30.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.733+0000 7f6140c00640 1 -- 172.21.15.196:0/4077644633 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) ==== 84+0+65 (secure 0 0 0) 0x7f613c058ff0 con 0x7f614415a9a0 2024-04-28T13:33:30.733 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:30.733 INFO:teuthology.orchestra.run.smithi196.stdout:[ 2024-04-28T13:33:30.733 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-04-28T13:33:30.733 INFO:teuthology.orchestra.run.smithi196.stdout: "poolnum": 1, 2024-04-28T13:33:30.733 INFO:teuthology.orchestra.run.smithi196.stdout: "poolname": ".mgr" 2024-04-28T13:33:30.733 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-28T13:33:30.734 INFO:teuthology.orchestra.run.smithi196.stdout:] 2024-04-28T13:33:30.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.736+0000 7f6149e00640 1 -- 172.21.15.196:0/4077644633 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f612406c9d0 msgr2=0x7f612406ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.736+0000 7f6149e00640 1 --2- 172.21.15.196:0/4077644633 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f612406c9d0 0x7f612406ee90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6138000f60 tx=0x7f6138009040 comp rx=0 tx=0).stop 2024-04-28T13:33:30.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.736+0000 7f6149e00640 1 -- 172.21.15.196:0/4077644633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f614415a9a0 msgr2=0x7f614409df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.736+0000 7f6149e00640 1 --2- 172.21.15.196:0/4077644633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f614415a9a0 0x7f614409df30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f613c00c730 tx=0x7f613c00cbf0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.736+0000 7f6149e00640 1 -- 172.21.15.196:0/4077644633 shutdown_connections 2024-04-28T13:33:30.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.736+0000 7f6149e00640 1 --2- 172.21.15.196:0/4077644633 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f612406c9d0 0x7f612406ee90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.736+0000 7f6149e00640 1 --2- 172.21.15.196:0/4077644633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f614415a9a0 0x7f614409df30 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.736+0000 7f6149e00640 1 --2- 172.21.15.196:0/4077644633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61440d15c0 0x7f614409d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.736+0000 7f6149e00640 1 --2- 172.21.15.196:0/4077644633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61440add70 0x7f614414b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.736+0000 7f6149e00640 1 -- 172.21.15.196:0/4077644633 >> 172.21.15.196:0/4077644633 conn(0x7f614406c6c0 msgr2=0x7f6144072bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:30.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.736+0000 7f6149e00640 1 -- 172.21.15.196:0/4077644633 shutdown_connections 2024-04-28T13:33:30.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.736+0000 7f6149e00640 1 -- 172.21.15.196:0/4077644633 wait complete. 2024-04-28T13:33:30.755 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-28T13:33:30.755 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-28T13:33:30.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.964+0000 7fdcb7800640 1 Processor -- start 2024-04-28T13:33:30.964 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.964+0000 7fdcb7800640 1 -- start start 2024-04-28T13:33:30.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.966+0000 7fdcb7800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdcb006f600 0x7fdcb0160e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.966+0000 7fdcb7800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcb0161570 con 0x7fdcb00d1190 2024-04-28T13:33:30.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.966+0000 7fdcb7800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcb0161850 con 0x7fdcb01437f0 2024-04-28T13:33:30.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.966+0000 7fdcb7800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fdcb0073440 con 0x7fdcb006f600 2024-04-28T13:33:30.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.966+0000 7fdcafe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdcb006f600 0x7fdcb0160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.966+0000 7fdcafe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdcb006f600 0x7fdcb0160e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37610/0 (socket says 172.21.15.196:37610) 2024-04-28T13:33:30.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.966+0000 7fdcafe00640 1 -- 172.21.15.196:0/586149573 learned_addr learned my addr 172.21.15.196:0/586149573 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:30.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.966+0000 7fdcb5800640 1 --1- 172.21.15.196:0/586149573 >> v1:172.21.15.196:6789/0 conn(0x7fdcb00d1190 0x7fdcb0073bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:54168/0 (socket says 172.21.15.196:54168) 2024-04-28T13:33:30.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.966+0000 7fdcafe00640 1 -- 172.21.15.196:0/586149573 >> v1:172.21.15.203:6789/0 conn(0x7fdcb01437f0 legacy=0x7fdcb0144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:30.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.967+0000 7fdcafe00640 1 -- 172.21.15.196:0/586149573 >> v1:172.21.15.196:6789/0 conn(0x7fdcb00d1190 legacy=0x7fdcb0073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.967+0000 7fdcafe00640 1 -- 172.21.15.196:0/586149573 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb0161b30 con 0x7fdcb006f600 2024-04-28T13:33:30.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.967+0000 7fdcaf400640 1 -- 172.21.15.196:0/586149573 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 212475152 0 0) 0x7fdcb0161570 con 0x7fdcb00d1190 2024-04-28T13:33:30.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.968+0000 7fdcafe00640 1 --2- 172.21.15.196:0/586149573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcb006f600 0x7fdcb0160e70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009870 tx=0x7fdc9c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275b832434900013 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:30.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.968+0000 7fdcaf400640 1 -- 172.21.15.196:0/586149573 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c037070 con 0x7fdcb006f600 2024-04-28T13:33:30.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.968+0000 7fdcaf400640 1 -- 172.21.15.196:0/586149573 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc9c032440 con 0x7fdcb006f600 2024-04-28T13:33:30.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.968+0000 7fdcaf400640 1 -- 172.21.15.196:0/586149573 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c032810 con 0x7fdcb006f600 2024-04-28T13:33:30.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.969+0000 7fdcb7800640 1 -- 172.21.15.196:0/586149573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcb006f600 msgr2=0x7fdcb0160e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.969 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.969+0000 7fdcb7800640 1 --2- 172.21.15.196:0/586149573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcb006f600 0x7fdcb0160e70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009870 tx=0x7fdc9c02a2a0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.969+0000 7fdcb7800640 1 -- 172.21.15.196:0/586149573 shutdown_connections 2024-04-28T13:33:30.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.969+0000 7fdcb7800640 1 --2- 172.21.15.196:0/586149573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcb006f600 0x7fdcb0160e70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.969+0000 7fdcb7800640 1 -- 172.21.15.196:0/586149573 >> 172.21.15.196:0/586149573 conn(0x7fdcb006c690 msgr2=0x7fdcb006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:30.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.969+0000 7fdcb7800640 1 -- 172.21.15.196:0/586149573 shutdown_connections 2024-04-28T13:33:30.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.969+0000 7fdcb7800640 1 -- 172.21.15.196:0/586149573 wait complete. 2024-04-28T13:33:30.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.970+0000 7fdcb7800640 1 Processor -- start 2024-04-28T13:33:30.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.971+0000 7fdcb7800640 1 -- start start 2024-04-28T13:33:30.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.971+0000 7fdcb7800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb006f600 0x7fdcb0141af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.971+0000 7fdcb7800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb00d1190 0x7fdcb01e8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.972+0000 7fdcb7800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcb01437f0 0x7fdcb015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.972+0000 7fdcb7800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb0143d50 con 0x7fdcb01437f0 2024-04-28T13:33:30.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.972+0000 7fdcb7800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb0143bd0 con 0x7fdcb006f600 2024-04-28T13:33:30.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.972+0000 7fdcb7800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdcb00742d0 con 0x7fdcb00d1190 2024-04-28T13:33:30.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.972+0000 7fdcb5800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcb01437f0 0x7fdcb015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.972+0000 7fdcb5800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcb01437f0 0x7fdcb015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37618/0 (socket says 172.21.15.196:37618) 2024-04-28T13:33:30.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.972+0000 7fdcb5800640 1 -- 172.21.15.196:0/2969264735 learned_addr learned my addr 172.21.15.196:0/2969264735 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:30.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.972+0000 7fdcb4e00640 1 --2- 172.21.15.196:0/2969264735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb006f600 0x7fdcb0141af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.972+0000 7fdcafe00640 1 --2- 172.21.15.196:0/2969264735 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb00d1190 0x7fdcb01e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.972+0000 7fdcb5800640 1 -- 172.21.15.196:0/2969264735 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb00d1190 msgr2=0x7fdcb01e8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.973 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.972+0000 7fdcb5800640 1 --2- 172.21.15.196:0/2969264735 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb00d1190 0x7fdcb01e8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.972+0000 7fdcb5800640 1 -- 172.21.15.196:0/2969264735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb006f600 msgr2=0x7fdcb0141af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:30.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.972+0000 7fdcb5800640 1 --2- 172.21.15.196:0/2969264735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb006f600 0x7fdcb0141af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:30.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.972+0000 7fdcb5800640 1 -- 172.21.15.196:0/2969264735 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb0142030 con 0x7fdcb01437f0 2024-04-28T13:33:30.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.972+0000 7fdcb4e00640 1 --2- 172.21.15.196:0/2969264735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb006f600 0x7fdcb0141af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:30.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.973+0000 7fdcb5800640 1 --2- 172.21.15.196:0/2969264735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcb01437f0 0x7fdcb015abd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdca800c7d0 tx=0x7fdca800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:30.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.973+0000 7fdcacc00640 1 -- 172.21.15.196:0/2969264735 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8018070 con 0x7fdcb01437f0 2024-04-28T13:33:30.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.973+0000 7fdcacc00640 1 -- 172.21.15.196:0/2969264735 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdca800a5e0 con 0x7fdcb01437f0 2024-04-28T13:33:30.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.973+0000 7fdcacc00640 1 -- 172.21.15.196:0/2969264735 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca800aa10 con 0x7fdcb01437f0 2024-04-28T13:33:30.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.973+0000 7fdcb7800640 1 -- 172.21.15.196:0/2969264735 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb01e78d0 con 0x7fdcb01437f0 2024-04-28T13:33:30.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.973+0000 7fdcb7800640 1 -- 172.21.15.196:0/2969264735 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb01e7e90 con 0x7fdcb01437f0 2024-04-28T13:33:30.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.974+0000 7fdcb7800640 1 -- 172.21.15.196:0/2969264735 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb006a290 con 0x7fdcb01437f0 2024-04-28T13:33:30.975 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.975+0000 7fdcacc00640 1 -- 172.21.15.196:0/2969264735 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fdca800abb0 con 0x7fdcb01437f0 2024-04-28T13:33:30.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.976+0000 7fdcacc00640 1 --2- 172.21.15.196:0/2969264735 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdc9006c980 0x7fdc9006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:30.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.976+0000 7fdcacc00640 1 -- 172.21.15.196:0/2969264735 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7fdca808aed0 con 0x7fdcb01437f0 2024-04-28T13:33:30.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.977+0000 7fdcb4e00640 1 --2- 172.21.15.196:0/2969264735 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdc9006c980 0x7fdc9006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:30.978 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.978+0000 7fdcb4e00640 1 --2- 172.21.15.196:0/2969264735 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdc9006c980 0x7fdc9006ee40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdca400a830 tx=0x7fdca4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:30.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:30.980+0000 7fdcacc00640 1 -- 172.21.15.196:0/2969264735 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdca80595f0 con 0x7fdcb01437f0 2024-04-28T13:33:31.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:31.179+0000 7fdcb7800640 1 -- 172.21.15.196:0/2969264735 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fdcb006eb80 con 0x7fdcb01437f0 2024-04-28T13:33:31.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:31.946+0000 7fdcacc00640 1 -- 172.21.15.196:0/2969264735 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) ==== 138+0+0 (secure 0 0 0) 0x7fdca8058ef0 con 0x7fdcb01437f0 2024-04-28T13:33:31.946 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'cephfs_metadata' created 2024-04-28T13:33:31.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:31.950+0000 7fdcb7800640 1 -- 172.21.15.196:0/2969264735 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdc9006c980 msgr2=0x7fdc9006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:31.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:31.950+0000 7fdcb7800640 1 --2- 172.21.15.196:0/2969264735 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdc9006c980 0x7fdc9006ee40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdca400a830 tx=0x7fdca4008040 comp rx=0 tx=0).stop 2024-04-28T13:33:31.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:31.950+0000 7fdcb7800640 1 -- 172.21.15.196:0/2969264735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcb01437f0 msgr2=0x7fdcb015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:31.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:31.950+0000 7fdcb7800640 1 --2- 172.21.15.196:0/2969264735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcb01437f0 0x7fdcb015abd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdca800c7d0 tx=0x7fdca800cc90 comp rx=0 tx=0).stop 2024-04-28T13:33:31.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:31.950+0000 7fdcb7800640 1 -- 172.21.15.196:0/2969264735 shutdown_connections 2024-04-28T13:33:31.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:31.950+0000 7fdcb7800640 1 --2- 172.21.15.196:0/2969264735 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fdc9006c980 0x7fdc9006ee40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:31.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:31.950+0000 7fdcb7800640 1 --2- 172.21.15.196:0/2969264735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcb01437f0 0x7fdcb015abd0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:31.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:31.950+0000 7fdcb7800640 1 --2- 172.21.15.196:0/2969264735 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb00d1190 0x7fdcb01e8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:31.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:31.950+0000 7fdcb7800640 1 --2- 172.21.15.196:0/2969264735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb006f600 0x7fdcb0141af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:31.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:31.950+0000 7fdcb7800640 1 -- 172.21.15.196:0/2969264735 >> 172.21.15.196:0/2969264735 conn(0x7fdcb006c690 msgr2=0x7fdcb0160420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:31.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:31.950+0000 7fdcb7800640 1 -- 172.21.15.196:0/2969264735 shutdown_connections 2024-04-28T13:33:31.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:31.950+0000 7fdcb7800640 1 -- 172.21.15.196:0/2969264735 wait complete. 2024-04-28T13:33:31.979 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-28T13:33:32.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.178+0000 7f1caa400640 1 Processor -- start 2024-04-28T13:33:32.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.178+0000 7f1caa400640 1 -- start start 2024-04-28T13:33:32.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.180+0000 7f1caa400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1ca4140090 0x7f1ca4142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:32.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.180+0000 7f1caa400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1ca4142c60 0x7f1ca40cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:32.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.180+0000 7f1caa400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca4143560 con 0x7f1ca40cfb00 2024-04-28T13:33:32.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.180+0000 7f1caa400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca40d0280 con 0x7f1ca4142c60 2024-04-28T13:33:32.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.180+0000 7f1caa400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca40d0560 con 0x7f1ca4140090 2024-04-28T13:33:32.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.180+0000 7f1ca2a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1ca4142c60 0x7f1ca40cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:32.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.180+0000 7f1ca2a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1ca4142c60 0x7f1ca40cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59600/0 (socket says 172.21.15.196:59600) 2024-04-28T13:33:32.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.180+0000 7f1ca2a00640 1 -- 172.21.15.196:0/2876930886 learned_addr learned my addr 172.21.15.196:0/2876930886 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:32.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.181+0000 7f1ca3400640 1 --2- 172.21.15.196:0/2876930886 >> v2:172.21.15.203:3300/0 conn(0x7f1ca4140090 0x7f1ca4142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:32.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.181+0000 7f1ca2a00640 1 -- 172.21.15.196:0/2876930886 >> v2:172.21.15.203:3300/0 conn(0x7f1ca4140090 msgr2=0x7f1ca4142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:32.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.181+0000 7f1ca2a00640 1 --2- 172.21.15.196:0/2876930886 >> v2:172.21.15.203:3300/0 conn(0x7f1ca4140090 0x7f1ca4142720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:32.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.181+0000 7f1ca2a00640 1 -- 172.21.15.196:0/2876930886 >> v1:172.21.15.196:6789/0 conn(0x7f1ca40cfb00 legacy=0x7f1ca4160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:32.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.181+0000 7f1ca2a00640 1 -- 172.21.15.196:0/2876930886 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca40d0840 con 0x7f1ca4142c60 2024-04-28T13:33:32.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.181+0000 7f1ca2000640 1 -- 172.21.15.196:0/2876930886 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 848508175 0 0) 0x7f1ca4143560 con 0x7f1ca40cfb00 2024-04-28T13:33:32.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.182+0000 7f1ca2a00640 1 --2- 172.21.15.196:0/2876930886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca4142c60 0x7f1ca40cf5c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4070520 tx=0x7f1c9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f872fb82a7f48dda server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:32.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.182+0000 7f1ca2000640 1 -- 172.21.15.196:0/2876930886 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90037070 con 0x7f1ca4142c60 2024-04-28T13:33:32.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.182+0000 7f1ca2000640 1 -- 172.21.15.196:0/2876930886 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c90032440 con 0x7f1ca4142c60 2024-04-28T13:33:32.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.182+0000 7f1ca2000640 1 -- 172.21.15.196:0/2876930886 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90032810 con 0x7f1ca4142c60 2024-04-28T13:33:32.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.183+0000 7f1caa400640 1 -- 172.21.15.196:0/2876930886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca4142c60 msgr2=0x7f1ca40cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:32.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.183+0000 7f1caa400640 1 --2- 172.21.15.196:0/2876930886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca4142c60 0x7f1ca40cf5c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4070520 tx=0x7f1c9002a270 comp rx=0 tx=0).stop 2024-04-28T13:33:32.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.183+0000 7f1caa400640 1 -- 172.21.15.196:0/2876930886 shutdown_connections 2024-04-28T13:33:32.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.183+0000 7f1caa400640 1 --2- 172.21.15.196:0/2876930886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca4142c60 0x7f1ca40cf5c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:32.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.183+0000 7f1caa400640 1 --2- 172.21.15.196:0/2876930886 >> v2:172.21.15.203:3300/0 conn(0x7f1ca4140090 0x7f1ca4142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:32.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.183+0000 7f1caa400640 1 -- 172.21.15.196:0/2876930886 >> 172.21.15.196:0/2876930886 conn(0x7f1ca40d0d30 msgr2=0x7f1ca40d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:32.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.183+0000 7f1caa400640 1 -- 172.21.15.196:0/2876930886 shutdown_connections 2024-04-28T13:33:32.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.183+0000 7f1caa400640 1 -- 172.21.15.196:0/2876930886 wait complete. 2024-04-28T13:33:32.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.185+0000 7f1caa400640 1 Processor -- start 2024-04-28T13:33:32.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.186+0000 7f1caa400640 1 -- start start 2024-04-28T13:33:32.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.187+0000 7f1caa400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40cfb00 0x7f1ca4153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:32.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.187+0000 7f1caa400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca4140090 0x7f1ca413dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:32.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.187+0000 7f1caa400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca4142c60 0x7f1ca4140af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:32.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.187+0000 7f1caa400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca406fa40 con 0x7f1ca4140090 2024-04-28T13:33:32.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.187+0000 7f1caa400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca40cff50 con 0x7f1ca4142c60 2024-04-28T13:33:32.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.187+0000 7f1caa400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca406fbc0 con 0x7f1ca40cfb00 2024-04-28T13:33:32.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.187+0000 7f1ca2a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca4140090 0x7f1ca413dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:32.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.187+0000 7f1ca2a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca4140090 0x7f1ca413dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59614/0 (socket says 172.21.15.196:59614) 2024-04-28T13:33:32.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.187+0000 7f1ca2a00640 1 -- 172.21.15.196:0/3682489018 learned_addr learned my addr 172.21.15.196:0/3682489018 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:32.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.188+0000 7f1ca3e00640 1 --2- 172.21.15.196:0/3682489018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca4142c60 0x7f1ca4140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:32.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.188+0000 7f1ca3400640 1 --2- 172.21.15.196:0/3682489018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40cfb00 0x7f1ca4153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:32.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.188+0000 7f1ca2a00640 1 -- 172.21.15.196:0/3682489018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40cfb00 msgr2=0x7f1ca4153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:32.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.188+0000 7f1ca2a00640 1 --2- 172.21.15.196:0/3682489018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40cfb00 0x7f1ca4153f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:32.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.188+0000 7f1ca2a00640 1 -- 172.21.15.196:0/3682489018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca4142c60 msgr2=0x7f1ca4140af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:32.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.188+0000 7f1ca2a00640 1 --2- 172.21.15.196:0/3682489018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca4142c60 0x7f1ca4140af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:32.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.188+0000 7f1ca2a00640 1 -- 172.21.15.196:0/3682489018 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca413e5d0 con 0x7f1ca4140090 2024-04-28T13:33:32.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.188+0000 7f1ca3e00640 1 --2- 172.21.15.196:0/3682489018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca4142c60 0x7f1ca4140af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:32.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.188+0000 7f1ca2a00640 1 --2- 172.21.15.196:0/3682489018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca4140090 0x7f1ca413dcd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1c9002a2b0 tx=0x7f1c900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:32.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.188+0000 7f1c9b400640 1 -- 172.21.15.196:0/3682489018 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90041020 con 0x7f1ca4140090 2024-04-28T13:33:32.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.188+0000 7f1c9b400640 1 -- 172.21.15.196:0/3682489018 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c900306c0 con 0x7f1ca4140090 2024-04-28T13:33:32.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.188+0000 7f1c9b400640 1 -- 172.21.15.196:0/3682489018 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90037040 con 0x7f1ca4140090 2024-04-28T13:33:32.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.188+0000 7f1caa400640 1 -- 172.21.15.196:0/3682489018 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca413e8b0 con 0x7f1ca4140090 2024-04-28T13:33:32.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.189+0000 7f1caa400640 1 -- 172.21.15.196:0/3682489018 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca413ee70 con 0x7f1ca4140090 2024-04-28T13:33:32.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.190+0000 7f1caa400640 1 -- 172.21.15.196:0/3682489018 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca406a290 con 0x7f1ca4140090 2024-04-28T13:33:32.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.191+0000 7f1c9b400640 1 -- 172.21.15.196:0/3682489018 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f1c90044020 con 0x7f1ca4140090 2024-04-28T13:33:32.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.192+0000 7f1c9b400640 1 --2- 172.21.15.196:0/3682489018 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1c8406c980 0x7f1c8406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:32.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.192+0000 7f1c9b400640 1 -- 172.21.15.196:0/3682489018 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5227+0+0 (secure 0 0 0) 0x7f1c900ac2e0 con 0x7f1ca4140090 2024-04-28T13:33:32.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.193+0000 7f1ca3400640 1 --2- 172.21.15.196:0/3682489018 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1c8406c980 0x7f1c8406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:32.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.194+0000 7f1ca3400640 1 --2- 172.21.15.196:0/3682489018 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1c8406c980 0x7f1c8406ee40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1c94006fd0 tx=0x7f1c94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:32.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.197+0000 7f1c9b400640 1 -- 172.21.15.196:0/3682489018 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1c9004b030 con 0x7f1ca4140090 2024-04-28T13:33:32.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.398+0000 7f1caa400640 1 -- 172.21.15.196:0/3682489018 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1ca40b4650 con 0x7f1ca4140090 2024-04-28T13:33:32.961 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.961+0000 7f1c9b400640 1 -- 172.21.15.196:0/3682489018 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) ==== 170+0+0 (secure 0 0 0) 0x7f1c9007a890 con 0x7f1ca4140090 2024-04-28T13:33:32.961 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'cephfs_data' created 2024-04-28T13:33:32.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.965+0000 7f1caa400640 1 -- 172.21.15.196:0/3682489018 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1c8406c980 msgr2=0x7f1c8406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:32.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.965+0000 7f1caa400640 1 --2- 172.21.15.196:0/3682489018 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1c8406c980 0x7f1c8406ee40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1c94006fd0 tx=0x7f1c94008040 comp rx=0 tx=0).stop 2024-04-28T13:33:32.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.965+0000 7f1caa400640 1 -- 172.21.15.196:0/3682489018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca4140090 msgr2=0x7f1ca413dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:32.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.965+0000 7f1caa400640 1 --2- 172.21.15.196:0/3682489018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca4140090 0x7f1ca413dcd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1c9002a2b0 tx=0x7f1c900046c0 comp rx=0 tx=0).stop 2024-04-28T13:33:32.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.965+0000 7f1caa400640 1 -- 172.21.15.196:0/3682489018 shutdown_connections 2024-04-28T13:33:32.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.965+0000 7f1caa400640 1 --2- 172.21.15.196:0/3682489018 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1c8406c980 0x7f1c8406ee40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:32.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.965+0000 7f1caa400640 1 --2- 172.21.15.196:0/3682489018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca4142c60 0x7f1ca4140af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:32.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.965+0000 7f1caa400640 1 --2- 172.21.15.196:0/3682489018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ca4140090 0x7f1ca413dcd0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:32.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.965+0000 7f1caa400640 1 --2- 172.21.15.196:0/3682489018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40cfb00 0x7f1ca4153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:32.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.965+0000 7f1caa400640 1 -- 172.21.15.196:0/3682489018 >> 172.21.15.196:0/3682489018 conn(0x7f1ca40d0d30 msgr2=0x7f1ca40afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:32.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.965+0000 7f1caa400640 1 -- 172.21.15.196:0/3682489018 shutdown_connections 2024-04-28T13:33:32.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:32.965+0000 7f1caa400640 1 -- 172.21.15.196:0/3682489018 wait complete. 2024-04-28T13:33:32.999 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-28T13:33:33.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.177+0000 7fd87a800640 1 Processor -- start 2024-04-28T13:33:33.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.177+0000 7fd87a800640 1 -- start start 2024-04-28T13:33:33.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.179+0000 7fd87a800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd874143840 0x7fd8740b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:33.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.179+0000 7fd87a800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd8740b4520 0x7fd8740b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:33.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.179+0000 7fd87a800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd87415dca0 con 0x7fd8740b53f0 2024-04-28T13:33:33.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.179+0000 7fd87a800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd874143c20 con 0x7fd8740b4520 2024-04-28T13:33:33.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.179+0000 7fd87a800640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd87415df80 con 0x7fd874143840 2024-04-28T13:33:33.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.179+0000 7fd86be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd8740b4520 0x7fd8740b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:33.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.179+0000 7fd86be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd8740b4520 0x7fd8740b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59624/0 (socket says 172.21.15.196:59624) 2024-04-28T13:33:33.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.179+0000 7fd86be00640 1 -- 172.21.15.196:0/2084227527 learned_addr learned my addr 172.21.15.196:0/2084227527 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:33.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.180+0000 7fd873400640 1 --2- 172.21.15.196:0/2084227527 >> v2:172.21.15.203:3300/0 conn(0x7fd874143840 0x7fd8740b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:33.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.180+0000 7fd86be00640 1 -- 172.21.15.196:0/2084227527 >> v2:172.21.15.203:3300/0 conn(0x7fd874143840 msgr2=0x7fd8740b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:33.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.180+0000 7fd86be00640 1 --2- 172.21.15.196:0/2084227527 >> v2:172.21.15.203:3300/0 conn(0x7fd874143840 0x7fd8740b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:33.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.180+0000 7fd86be00640 1 -- 172.21.15.196:0/2084227527 >> v1:172.21.15.203:6789/0 conn(0x7fd8740b53f0 legacy=0x7fd874160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:33.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.180+0000 7fd86be00640 1 -- 172.21.15.196:0/2084227527 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87415e260 con 0x7fd8740b4520 2024-04-28T13:33:33.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.181+0000 7fd86be00640 1 --2- 172.21.15.196:0/2084227527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8740b4520 0x7fd8740b4eb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd874076130 tx=0x7fd85c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fdafb6b6402b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:33.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.181+0000 7fd872a00640 1 -- 172.21.15.196:0/2084227527 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c037070 con 0x7fd8740b4520 2024-04-28T13:33:33.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.181+0000 7fd872a00640 1 -- 172.21.15.196:0/2084227527 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd85c032440 con 0x7fd8740b4520 2024-04-28T13:33:33.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.182+0000 7fd872a00640 1 -- 172.21.15.196:0/2084227527 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c032810 con 0x7fd8740b4520 2024-04-28T13:33:33.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.182+0000 7fd87a800640 1 -- 172.21.15.196:0/2084227527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8740b4520 msgr2=0x7fd8740b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:33.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.182+0000 7fd87a800640 1 --2- 172.21.15.196:0/2084227527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8740b4520 0x7fd8740b4eb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd874076130 tx=0x7fd85c02a230 comp rx=0 tx=0).stop 2024-04-28T13:33:33.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.182+0000 7fd87a800640 1 -- 172.21.15.196:0/2084227527 shutdown_connections 2024-04-28T13:33:33.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.182+0000 7fd87a800640 1 --2- 172.21.15.196:0/2084227527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8740b4520 0x7fd8740b4eb0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:33.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.182+0000 7fd87a800640 1 --2- 172.21.15.196:0/2084227527 >> v2:172.21.15.203:3300/0 conn(0x7fd874143840 0x7fd8740b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:33.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.182+0000 7fd87a800640 1 -- 172.21.15.196:0/2084227527 >> 172.21.15.196:0/2084227527 conn(0x7fd8740cd8a0 msgr2=0x7fd8740cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:33.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.182+0000 7fd87a800640 1 -- 172.21.15.196:0/2084227527 shutdown_connections 2024-04-28T13:33:33.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.182+0000 7fd87a800640 1 -- 172.21.15.196:0/2084227527 wait complete. 2024-04-28T13:33:33.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.184+0000 7fd87a800640 1 Processor -- start 2024-04-28T13:33:33.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.184+0000 7fd87a800640 1 -- start start 2024-04-28T13:33:33.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.185+0000 7fd87a800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8740b4520 0x7fd87415aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:33.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.185+0000 7fd87a800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8740b53f0 0x7fd874130fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:33.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.185+0000 7fd87a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd874143840 0x7fd874144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:33.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.185+0000 7fd87a800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd87415da40 con 0x7fd874143840 2024-04-28T13:33:33.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.185+0000 7fd87a800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd87415d8c0 con 0x7fd8740b53f0 2024-04-28T13:33:33.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.185+0000 7fd87a800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd8740d0000 con 0x7fd8740b4520 2024-04-28T13:33:33.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.186+0000 7fd873e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd874143840 0x7fd874144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:33.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.186+0000 7fd873e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd874143840 0x7fd874144800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59630/0 (socket says 172.21.15.196:59630) 2024-04-28T13:33:33.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.186+0000 7fd873e00640 1 -- 172.21.15.196:0/3720542226 learned_addr learned my addr 172.21.15.196:0/3720542226 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:33.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.186+0000 7fd873e00640 1 -- 172.21.15.196:0/3720542226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8740b4520 msgr2=0x7fd87415aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:33.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.186+0000 7fd873400640 1 --2- 172.21.15.196:0/3720542226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8740b4520 0x7fd87415aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:33.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.186+0000 7fd873e00640 1 --2- 172.21.15.196:0/3720542226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8740b4520 0x7fd87415aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:33.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.186+0000 7fd873e00640 1 -- 172.21.15.196:0/3720542226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8740b53f0 msgr2=0x7fd874130fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:33.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.186+0000 7fd86be00640 1 --2- 172.21.15.196:0/3720542226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8740b53f0 0x7fd874130fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:33.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.186+0000 7fd873e00640 1 --2- 172.21.15.196:0/3720542226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8740b53f0 0x7fd874130fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:33.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.186+0000 7fd873e00640 1 -- 172.21.15.196:0/3720542226 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8741451e0 con 0x7fd874143840 2024-04-28T13:33:33.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.186+0000 7fd873e00640 1 --2- 172.21.15.196:0/3720542226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd874143840 0x7fd874144800 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00c780 tx=0x7fd86c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:33.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.187+0000 7fd86b400640 1 -- 172.21.15.196:0/3720542226 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c004b60 con 0x7fd874143840 2024-04-28T13:33:33.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.187+0000 7fd86b400640 1 -- 172.21.15.196:0/3720542226 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd86c004d00 con 0x7fd874143840 2024-04-28T13:33:33.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.187+0000 7fd87a800640 1 -- 172.21.15.196:0/3720542226 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8741454c0 con 0x7fd874143840 2024-04-28T13:33:33.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.187+0000 7fd86b400640 1 -- 172.21.15.196:0/3720542226 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c00aa30 con 0x7fd874143840 2024-04-28T13:33:33.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.187+0000 7fd87a800640 1 -- 172.21.15.196:0/3720542226 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd874145a80 con 0x7fd874143840 2024-04-28T13:33:33.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.188+0000 7fd87a800640 1 -- 172.21.15.196:0/3720542226 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd874069c80 con 0x7fd874143840 2024-04-28T13:33:33.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.189+0000 7fd86b400640 1 -- 172.21.15.196:0/3720542226 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd86c00abd0 con 0x7fd874143840 2024-04-28T13:33:33.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.190+0000 7fd86b400640 1 --2- 172.21.15.196:0/3720542226 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd85006c980 0x7fd85006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:33.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.191+0000 7fd86b400640 1 -- 172.21.15.196:0/3720542226 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5609+0+0 (secure 0 0 0) 0x7fd86c08bf30 con 0x7fd874143840 2024-04-28T13:33:33.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.191+0000 7fd873400640 1 --2- 172.21.15.196:0/3720542226 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd85006c980 0x7fd85006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:33.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.192+0000 7fd873400640 1 --2- 172.21.15.196:0/3720542226 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd85006c980 0x7fd85006ee40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd864006fd0 tx=0x7fd864008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:33.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.200+0000 7fd86b400640 1 -- 172.21.15.196:0/3720542226 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd874069c80 con 0x7fd874143840 2024-04-28T13:33:33.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.438+0000 7fd87a800640 1 -- 172.21.15.196:0/3720542226 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd8740ce420 con 0x7fd874143840 2024-04-28T13:33:33.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.965+0000 7fd86b400640 1 -- 172.21.15.196:0/3720542226 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v27) ==== 172+0+0 (secure 0 0 0) 0x7fd86c020070 con 0x7fd874143840 2024-04-28T13:33:33.966 INFO:teuthology.orchestra.run.smithi196.stderr:new fs with metadata pool 6 and data pool 7 2024-04-28T13:33:33.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.969+0000 7fd87a800640 1 -- 172.21.15.196:0/3720542226 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd85006c980 msgr2=0x7fd85006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:33.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.969+0000 7fd87a800640 1 --2- 172.21.15.196:0/3720542226 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd85006c980 0x7fd85006ee40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd864006fd0 tx=0x7fd864008040 comp rx=0 tx=0).stop 2024-04-28T13:33:33.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.969+0000 7fd87a800640 1 -- 172.21.15.196:0/3720542226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd874143840 msgr2=0x7fd874144800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:33.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.969+0000 7fd87a800640 1 --2- 172.21.15.196:0/3720542226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd874143840 0x7fd874144800 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00c780 tx=0x7fd86c00cc40 comp rx=0 tx=0).stop 2024-04-28T13:33:33.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.970+0000 7fd87a800640 1 -- 172.21.15.196:0/3720542226 shutdown_connections 2024-04-28T13:33:33.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.970+0000 7fd87a800640 1 --2- 172.21.15.196:0/3720542226 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd85006c980 0x7fd85006ee40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:33.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.970+0000 7fd87a800640 1 --2- 172.21.15.196:0/3720542226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd874143840 0x7fd874144800 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:33.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.970+0000 7fd87a800640 1 --2- 172.21.15.196:0/3720542226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8740b53f0 0x7fd874130fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:33.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.970+0000 7fd87a800640 1 --2- 172.21.15.196:0/3720542226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8740b4520 0x7fd87415aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:33.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.970+0000 7fd87a800640 1 -- 172.21.15.196:0/3720542226 >> 172.21.15.196:0/3720542226 conn(0x7fd8740cd8a0 msgr2=0x7fd874074100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:33.970 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.970+0000 7fd87a800640 1 -- 172.21.15.196:0/3720542226 shutdown_connections 2024-04-28T13:33:33.971 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:33.970+0000 7fd87a800640 1 -- 172.21.15.196:0/3720542226 wait complete. 2024-04-28T13:33:34.006 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:33:34.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.179+0000 7fc8bf000640 1 Processor -- start 2024-04-28T13:33:34.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.179+0000 7fc8bf000640 1 -- start start 2024-04-28T13:33:34.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.181+0000 7fc8bf000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc8b81469f0 0x7fc8b80ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:34.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.181+0000 7fc8bf000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc8b80d1410 0x7fc8b80b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:34.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.181+0000 7fc8bf000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8b8160f40 con 0x7fc8b80cd420 2024-04-28T13:33:34.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.181+0000 7fc8bf000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b80b5960 con 0x7fc8b81469f0 2024-04-28T13:33:34.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.181+0000 7fc8bf000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b8161ac0 con 0x7fc8b80d1410 2024-04-28T13:33:34.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.181+0000 7fc8b7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc8b81469f0 0x7fc8b80ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:34.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.181+0000 7fc8b7e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc8b81469f0 0x7fc8b80ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59632/0 (socket says 172.21.15.196:59632) 2024-04-28T13:33:34.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.181+0000 7fc8b7e00640 1 -- 172.21.15.196:0/1298231640 learned_addr learned my addr 172.21.15.196:0/1298231640 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:34.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.182+0000 7fc8bd000640 1 --2- 172.21.15.196:0/1298231640 >> v2:172.21.15.203:3300/0 conn(0x7fc8b80d1410 0x7fc8b80b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:34.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.182+0000 7fc8b7e00640 1 -- 172.21.15.196:0/1298231640 >> v2:172.21.15.203:3300/0 conn(0x7fc8b80d1410 msgr2=0x7fc8b80b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:34.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.182+0000 7fc8b7e00640 1 --2- 172.21.15.196:0/1298231640 >> v2:172.21.15.203:3300/0 conn(0x7fc8b80d1410 0x7fc8b80b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.182+0000 7fc8b7e00640 1 -- 172.21.15.196:0/1298231640 >> v1:172.21.15.203:6789/0 conn(0x7fc8b80cd420 legacy=0x7fc8b80d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:34.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.182+0000 7fc8b7e00640 1 -- 172.21.15.196:0/1298231640 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b8161220 con 0x7fc8b81469f0 2024-04-28T13:33:34.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.182+0000 7fc8bd000640 1 --2- 172.21.15.196:0/1298231640 >> v2:172.21.15.203:3300/0 conn(0x7fc8b80d1410 0x7fc8b80b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:34.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.183+0000 7fc8b7e00640 1 --2- 172.21.15.196:0/1298231640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b81469f0 0x7fc8b80ccee0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8161c40 tx=0x7fc8a802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e738f03af967615e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:34.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.183+0000 7fc8b6a00640 1 -- 172.21.15.196:0/1298231640 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8037070 con 0x7fc8b81469f0 2024-04-28T13:33:34.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.183+0000 7fc8b6a00640 1 -- 172.21.15.196:0/1298231640 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a8032440 con 0x7fc8b81469f0 2024-04-28T13:33:34.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.183+0000 7fc8b6a00640 1 -- 172.21.15.196:0/1298231640 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8032810 con 0x7fc8b81469f0 2024-04-28T13:33:34.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.184+0000 7fc8bf000640 1 -- 172.21.15.196:0/1298231640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b81469f0 msgr2=0x7fc8b80ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:34.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.184+0000 7fc8bf000640 1 --2- 172.21.15.196:0/1298231640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b81469f0 0x7fc8b80ccee0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8161c40 tx=0x7fc8a802a210 comp rx=0 tx=0).stop 2024-04-28T13:33:34.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.184+0000 7fc8bf000640 1 -- 172.21.15.196:0/1298231640 shutdown_connections 2024-04-28T13:33:34.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.184+0000 7fc8bf000640 1 --2- 172.21.15.196:0/1298231640 >> v2:172.21.15.203:3300/0 conn(0x7fc8b80d1410 0x7fc8b80b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.184+0000 7fc8bf000640 1 --2- 172.21.15.196:0/1298231640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b81469f0 0x7fc8b80ccee0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.184+0000 7fc8bf000640 1 -- 172.21.15.196:0/1298231640 >> 172.21.15.196:0/1298231640 conn(0x7fc8b80cd830 msgr2=0x7fc8b80cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:34.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.184+0000 7fc8bf000640 1 -- 172.21.15.196:0/1298231640 shutdown_connections 2024-04-28T13:33:34.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.184+0000 7fc8bf000640 1 -- 172.21.15.196:0/1298231640 wait complete. 2024-04-28T13:33:34.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.186+0000 7fc8bf000640 1 Processor -- start 2024-04-28T13:33:34.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.186+0000 7fc8bf000640 1 -- start start 2024-04-28T13:33:34.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.187+0000 7fc8bf000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8b80cd420 0x7fc8b813c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:34.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.187+0000 7fc8bf000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b80d1410 0x7fc8b80aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:34.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.187+0000 7fc8bf000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8b81469f0 0x7fc8b80ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:34.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.187+0000 7fc8bf000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b8072020 con 0x7fc8b80d1410 2024-04-28T13:33:34.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.187+0000 7fc8bf000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b81617e0 con 0x7fc8b81469f0 2024-04-28T13:33:34.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.187+0000 7fc8bf000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b8160b80 con 0x7fc8b80cd420 2024-04-28T13:33:34.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.187+0000 7fc8b7400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b80d1410 0x7fc8b80aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:34.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.187+0000 7fc8b7400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b80d1410 0x7fc8b80aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59646/0 (socket says 172.21.15.196:59646) 2024-04-28T13:33:34.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.187+0000 7fc8b7400640 1 -- 172.21.15.196:0/626864564 learned_addr learned my addr 172.21.15.196:0/626864564 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:34.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.187+0000 7fc8b7400640 1 -- 172.21.15.196:0/626864564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8b80cd420 msgr2=0x7fc8b813c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:34.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.187+0000 7fc8b7400640 1 --2- 172.21.15.196:0/626864564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8b80cd420 0x7fc8b813c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.187+0000 7fc8b7400640 1 -- 172.21.15.196:0/626864564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8b81469f0 msgr2=0x7fc8b80ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:34.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.187+0000 7fc8b7400640 1 --2- 172.21.15.196:0/626864564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8b81469f0 0x7fc8b80ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.187+0000 7fc8b7400640 1 -- 172.21.15.196:0/626864564 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b813ca60 con 0x7fc8b80d1410 2024-04-28T13:33:34.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.188+0000 7fc8b7400640 1 --2- 172.21.15.196:0/626864564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b80d1410 0x7fc8b80aff10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400c430 tx=0x7fc8a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:34.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.188+0000 7fc8afe00640 1 -- 172.21.15.196:0/626864564 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4011790 con 0x7fc8b80d1410 2024-04-28T13:33:34.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.188+0000 7fc8afe00640 1 -- 172.21.15.196:0/626864564 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a4017070 con 0x7fc8b80d1410 2024-04-28T13:33:34.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.188+0000 7fc8afe00640 1 -- 172.21.15.196:0/626864564 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4011bb0 con 0x7fc8b80d1410 2024-04-28T13:33:34.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.188+0000 7fc8bf000640 1 -- 172.21.15.196:0/626864564 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b8145430 con 0x7fc8b80d1410 2024-04-28T13:33:34.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.188+0000 7fc8bf000640 1 -- 172.21.15.196:0/626864564 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b8072a40 con 0x7fc8b80d1410 2024-04-28T13:33:34.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.189+0000 7fc8bf000640 1 -- 172.21.15.196:0/626864564 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b8145600 con 0x7fc8b80d1410 2024-04-28T13:33:34.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.190+0000 7fc8afe00640 1 -- 172.21.15.196:0/626864564 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc8a401c460 con 0x7fc8b80d1410 2024-04-28T13:33:34.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.191+0000 7fc8afe00640 1 --2- 172.21.15.196:0/626864564 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc89806c980 0x7fc89806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:34.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.192+0000 7fc8afe00640 1 -- 172.21.15.196:0/626864564 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fc8a408bb10 con 0x7fc8b80d1410 2024-04-28T13:33:34.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.192+0000 7fc8b7e00640 1 --2- 172.21.15.196:0/626864564 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc89806c980 0x7fc89806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:34.193 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.193+0000 7fc8b7e00640 1 --2- 172.21.15.196:0/626864564 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc89806c980 0x7fc89806ee40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8034000 tx=0x7fc8a802c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:34.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.196+0000 7fc8afe00640 1 -- 172.21.15.196:0/626864564 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc8a4059ee0 con 0x7fc8b80d1410 2024-04-28T13:33:34.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.390+0000 7fc8bf000640 1 -- 172.21.15.196:0/626864564 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8b806a290 con 0x7fc8b80d1410 2024-04-28T13:33:34.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.393+0000 7fc8afe00640 1 -- 172.21.15.196:0/626864564 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16662 (secure 0 0 0) 0x7fc8a40597e0 con 0x7fc8b80d1410 2024-04-28T13:33:34.394 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:34.394 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":44,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:33:33.440260+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:33:31.193265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-28T13:33:32.415392+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:33:34.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.395+0000 7fc8bf000640 1 -- 172.21.15.196:0/626864564 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc89806c980 msgr2=0x7fc89806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:34.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.395+0000 7fc8bf000640 1 --2- 172.21.15.196:0/626864564 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc89806c980 0x7fc89806ee40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8034000 tx=0x7fc8a802c000 comp rx=0 tx=0).stop 2024-04-28T13:33:34.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.395+0000 7fc8bf000640 1 -- 172.21.15.196:0/626864564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b80d1410 msgr2=0x7fc8b80aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:34.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.395+0000 7fc8bf000640 1 --2- 172.21.15.196:0/626864564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b80d1410 0x7fc8b80aff10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400c430 tx=0x7fc8a400c8f0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.395+0000 7fc8bf000640 1 -- 172.21.15.196:0/626864564 shutdown_connections 2024-04-28T13:33:34.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.395+0000 7fc8bf000640 1 --2- 172.21.15.196:0/626864564 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc89806c980 0x7fc89806ee40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.395+0000 7fc8bf000640 1 --2- 172.21.15.196:0/626864564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8b81469f0 0x7fc8b80ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.395+0000 7fc8bf000640 1 --2- 172.21.15.196:0/626864564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b80d1410 0x7fc8b80aff10 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.395+0000 7fc8bf000640 1 --2- 172.21.15.196:0/626864564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8b80cd420 0x7fc8b813c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.395+0000 7fc8bf000640 1 -- 172.21.15.196:0/626864564 >> 172.21.15.196:0/626864564 conn(0x7fc8b80cd830 msgr2=0x7fc8b806e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:34.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.396+0000 7fc8bf000640 1 -- 172.21.15.196:0/626864564 shutdown_connections 2024-04-28T13:33:34.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.396+0000 7fc8bf000640 1 -- 172.21.15.196:0/626864564 wait complete. 2024-04-28T13:33:34.414 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:33:34.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.609+0000 7fa88bc00640 1 Processor -- start 2024-04-28T13:33:34.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.609+0000 7fa88bc00640 1 -- start start 2024-04-28T13:33:34.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.611+0000 7fa88bc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa8840cfda0 0x7fa8840d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:34.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.611+0000 7fa88bc00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa8840b60a0 0x7fa8840b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:34.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.611+0000 7fa88bc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8840d0220 con 0x7fa8840d1b80 2024-04-28T13:33:34.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.611+0000 7fa88bc00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa8840d0500 con 0x7fa8840cfda0 2024-04-28T13:33:34.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.611+0000 7fa88bc00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa8840d07e0 con 0x7fa8840b60a0 2024-04-28T13:33:34.612 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.611+0000 7fa889200640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa8840d1b80 0x7fa8840d3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:38180/0 (socket says 172.21.15.196:38180) 2024-04-28T13:33:34.612 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.611+0000 7fa889200640 1 -- 172.21.15.196:0/44503676 learned_addr learned my addr 172.21.15.196:0/44503676 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:34.612 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.611+0000 7fa883e00640 1 --2- 172.21.15.196:0/44503676 >> v2:172.21.15.196:3300/0 conn(0x7fa8840cfda0 0x7fa8840d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.612+0000 7fa883400640 1 -- 172.21.15.196:0/44503676 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2362536134 0 0) 0x7fa8840d0220 con 0x7fa8840d1b80 2024-04-28T13:33:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.612+0000 7fa883400640 1 -- 172.21.15.196:0/44503676 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa86c003520 con 0x7fa8840d1b80 2024-04-28T13:33:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.612+0000 7fa889c00640 1 --2- 172.21.15.196:0/44503676 >> v2:172.21.15.203:3300/0 conn(0x7fa8840b60a0 0x7fa8840b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.612+0000 7fa883e00640 1 -- 172.21.15.196:0/44503676 >> v2:172.21.15.203:3300/0 conn(0x7fa8840b60a0 msgr2=0x7fa8840b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.612+0000 7fa883e00640 1 --2- 172.21.15.196:0/44503676 >> v2:172.21.15.203:3300/0 conn(0x7fa8840b60a0 0x7fa8840b6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.612+0000 7fa883e00640 1 -- 172.21.15.196:0/44503676 >> v1:172.21.15.196:6789/0 conn(0x7fa8840d1b80 legacy=0x7fa8840d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.612+0000 7fa883e00640 1 -- 172.21.15.196:0/44503676 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8840d0ac0 con 0x7fa8840cfda0 2024-04-28T13:33:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.612+0000 7fa889c00640 1 --2- 172.21.15.196:0/44503676 >> v2:172.21.15.203:3300/0 conn(0x7fa8840b60a0 0x7fa8840b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.613+0000 7fa883e00640 1 --2- 172.21.15.196:0/44503676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8840cfda0 0x7fa8840d1f60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa88409e9e0 tx=0x7fa87002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4d4e5c5b73c710 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.613+0000 7fa883400640 1 -- 172.21.15.196:0/44503676 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870037070 con 0x7fa8840cfda0 2024-04-28T13:33:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.613+0000 7fa883400640 1 -- 172.21.15.196:0/44503676 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa870032440 con 0x7fa8840cfda0 2024-04-28T13:33:34.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.613+0000 7fa883400640 1 -- 172.21.15.196:0/44503676 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870032810 con 0x7fa8840cfda0 2024-04-28T13:33:34.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.614+0000 7fa88bc00640 1 -- 172.21.15.196:0/44503676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8840cfda0 msgr2=0x7fa8840d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:34.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.614+0000 7fa88bc00640 1 --2- 172.21.15.196:0/44503676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8840cfda0 0x7fa8840d1f60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa88409e9e0 tx=0x7fa87002a1a0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.614+0000 7fa88bc00640 1 -- 172.21.15.196:0/44503676 shutdown_connections 2024-04-28T13:33:34.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.614+0000 7fa88bc00640 1 --2- 172.21.15.196:0/44503676 >> v2:172.21.15.203:3300/0 conn(0x7fa8840b60a0 0x7fa8840b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.614+0000 7fa88bc00640 1 --2- 172.21.15.196:0/44503676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8840cfda0 0x7fa8840d1f60 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.614+0000 7fa88bc00640 1 -- 172.21.15.196:0/44503676 >> 172.21.15.196:0/44503676 conn(0x7fa88409e540 msgr2=0x7fa8840d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:34.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.614+0000 7fa88bc00640 1 -- 172.21.15.196:0/44503676 shutdown_connections 2024-04-28T13:33:34.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.614+0000 7fa88bc00640 1 -- 172.21.15.196:0/44503676 wait complete. 2024-04-28T13:33:34.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.616+0000 7fa88bc00640 1 Processor -- start 2024-04-28T13:33:34.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.616+0000 7fa88bc00640 1 -- start start 2024-04-28T13:33:34.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.617+0000 7fa88bc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8840b60a0 0x7fa8840b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:34.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.617+0000 7fa88bc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8840cfda0 0x7fa8840b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:34.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.617+0000 7fa88bc00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8840d1b80 0x7fa8840ba310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:34.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.617+0000 7fa88bc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8840d4170 con 0x7fa8840cfda0 2024-04-28T13:33:34.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.617+0000 7fa88bc00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa8840b6a70 con 0x7fa8840b60a0 2024-04-28T13:33:34.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.617+0000 7fa88bc00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa88415ee50 con 0x7fa8840d1b80 2024-04-28T13:33:34.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.617+0000 7fa883e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8840cfda0 0x7fa8840b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:34.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.617+0000 7fa883e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8840cfda0 0x7fa8840b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59656/0 (socket says 172.21.15.196:59656) 2024-04-28T13:33:34.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.618+0000 7fa883e00640 1 -- 172.21.15.196:0/3527017416 learned_addr learned my addr 172.21.15.196:0/3527017416 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:34.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.618+0000 7fa889200640 1 --2- 172.21.15.196:0/3527017416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8840b60a0 0x7fa8840b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:34.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.618+0000 7fa889c00640 1 --2- 172.21.15.196:0/3527017416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8840d1b80 0x7fa8840ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:34.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.618+0000 7fa883e00640 1 -- 172.21.15.196:0/3527017416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8840d1b80 msgr2=0x7fa8840ba310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:34.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.618+0000 7fa883e00640 1 --2- 172.21.15.196:0/3527017416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8840d1b80 0x7fa8840ba310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.618+0000 7fa883e00640 1 -- 172.21.15.196:0/3527017416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8840b60a0 msgr2=0x7fa8840b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:34.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.618+0000 7fa883e00640 1 --2- 172.21.15.196:0/3527017416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8840b60a0 0x7fa8840b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.618+0000 7fa883e00640 1 -- 172.21.15.196:0/3527017416 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8840b2410 con 0x7fa8840cfda0 2024-04-28T13:33:34.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.618+0000 7fa889200640 1 --2- 172.21.15.196:0/3527017416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8840b60a0 0x7fa8840b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:34.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.618+0000 7fa883e00640 1 --2- 172.21.15.196:0/3527017416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8840cfda0 0x7fa8840b34b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa870009890 tx=0x7fa8700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:34.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.618+0000 7fa880c00640 1 -- 172.21.15.196:0/3527017416 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870037040 con 0x7fa8840cfda0 2024-04-28T13:33:34.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.619+0000 7fa880c00640 1 -- 172.21.15.196:0/3527017416 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa870042070 con 0x7fa8840cfda0 2024-04-28T13:33:34.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.619+0000 7fa880c00640 1 -- 172.21.15.196:0/3527017416 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870030960 con 0x7fa8840cfda0 2024-04-28T13:33:34.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.619+0000 7fa88bc00640 1 -- 172.21.15.196:0/3527017416 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8840b26f0 con 0x7fa8840cfda0 2024-04-28T13:33:34.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.619+0000 7fa88bc00640 1 -- 172.21.15.196:0/3527017416 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8840b2cb0 con 0x7fa8840cfda0 2024-04-28T13:33:34.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.620+0000 7fa88bc00640 1 -- 172.21.15.196:0/3527017416 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa88406a290 con 0x7fa8840cfda0 2024-04-28T13:33:34.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.621+0000 7fa880c00640 1 -- 172.21.15.196:0/3527017416 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa870030b00 con 0x7fa8840cfda0 2024-04-28T13:33:34.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.622+0000 7fa880c00640 1 --2- 172.21.15.196:0/3527017416 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa86406c980 0x7fa86406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:34.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.622+0000 7fa880c00640 1 -- 172.21.15.196:0/3527017416 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fa8700ac5d0 con 0x7fa8840cfda0 2024-04-28T13:33:34.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.622+0000 7fa889200640 1 --2- 172.21.15.196:0/3527017416 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa86406c980 0x7fa86406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:34.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.624+0000 7fa889200640 1 --2- 172.21.15.196:0/3527017416 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa86406c980 0x7fa86406ee40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa878005880 tx=0x7fa878009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:34.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.627+0000 7fa880c00640 1 -- 172.21.15.196:0/3527017416 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa870045360 con 0x7fa8840cfda0 2024-04-28T13:33:34.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.829+0000 7fa88bc00640 1 -- 172.21.15.196:0/3527017416 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8840d27b0 con 0x7fa8840cfda0 2024-04-28T13:33:34.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.830+0000 7fa880c00640 1 -- 172.21.15.196:0/3527017416 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16662 (secure 0 0 0) 0x7fa87007a990 con 0x7fa8840cfda0 2024-04-28T13:33:34.830 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:34.831 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":44,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:33:33.440260+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:33:31.193265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-28T13:33:32.415392+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:33:34.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.833+0000 7fa88bc00640 1 -- 172.21.15.196:0/3527017416 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa86406c980 msgr2=0x7fa86406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:34.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.833+0000 7fa88bc00640 1 --2- 172.21.15.196:0/3527017416 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa86406c980 0x7fa86406ee40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa878005880 tx=0x7fa878009040 comp rx=0 tx=0).stop 2024-04-28T13:33:34.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.833+0000 7fa88bc00640 1 -- 172.21.15.196:0/3527017416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8840cfda0 msgr2=0x7fa8840b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:34.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.833+0000 7fa88bc00640 1 --2- 172.21.15.196:0/3527017416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8840cfda0 0x7fa8840b34b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa870009890 tx=0x7fa8700046c0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.833+0000 7fa88bc00640 1 -- 172.21.15.196:0/3527017416 shutdown_connections 2024-04-28T13:33:34.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.833+0000 7fa88bc00640 1 --2- 172.21.15.196:0/3527017416 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa86406c980 0x7fa86406ee40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.833+0000 7fa88bc00640 1 --2- 172.21.15.196:0/3527017416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8840d1b80 0x7fa8840ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.833+0000 7fa88bc00640 1 --2- 172.21.15.196:0/3527017416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8840cfda0 0x7fa8840b34b0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.833+0000 7fa88bc00640 1 --2- 172.21.15.196:0/3527017416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8840b60a0 0x7fa8840b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:34.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.833+0000 7fa88bc00640 1 -- 172.21.15.196:0/3527017416 >> 172.21.15.196:0/3527017416 conn(0x7fa88409e540 msgr2=0x7fa8840ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:34.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.833+0000 7fa88bc00640 1 -- 172.21.15.196:0/3527017416 shutdown_connections 2024-04-28T13:33:34.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:34.833+0000 7fa88bc00640 1 -- 172.21.15.196:0/3527017416 wait complete. 2024-04-28T13:33:34.851 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-28T13:33:35.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.049+0000 7f36ec800640 1 Processor -- start 2024-04-28T13:33:35.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.049+0000 7f36ec800640 1 -- start start 2024-04-28T13:33:35.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.051+0000 7f36ec800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f36e406f340 0x7f36e40b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:35.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.051+0000 7f36ec800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36e40b46a0 con 0x7f36e40b3e50 2024-04-28T13:33:35.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.051+0000 7f36ec800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36e40b5db0 con 0x7f36e40d0300 2024-04-28T13:33:35.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.051+0000 7f36ec800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f36e40b6090 con 0x7f36e406f340 2024-04-28T13:33:35.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.051+0000 7f36e9e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f36e406f340 0x7f36e40b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:35.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.051+0000 7f36e9e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f36e406f340 0x7f36e40b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59672/0 (socket says 172.21.15.196:59672) 2024-04-28T13:33:35.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.051+0000 7f36e9e00640 1 -- 172.21.15.196:0/4154497509 learned_addr learned my addr 172.21.15.196:0/4154497509 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:35.051 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.051+0000 7f36ea800640 1 --1- 172.21.15.196:0/4154497509 >> v1:172.21.15.196:6789/0 conn(0x7f36e40b3e50 0x7f36e40720f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:38190/0 (socket says 172.21.15.196:38190) 2024-04-28T13:33:35.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.052+0000 7f36e9e00640 1 -- 172.21.15.196:0/4154497509 >> v1:172.21.15.203:6789/0 conn(0x7f36e40d0300 legacy=0x7f36e406f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:35.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.052+0000 7f36e9e00640 1 -- 172.21.15.196:0/4154497509 >> v1:172.21.15.196:6789/0 conn(0x7f36e40b3e50 legacy=0x7f36e40720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:35.052 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.052+0000 7f36e9e00640 1 -- 172.21.15.196:0/4154497509 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e40b6370 con 0x7f36e406f340 2024-04-28T13:33:35.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.053+0000 7f36e9e00640 1 --2- 172.21.15.196:0/4154497509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36e406f340 0x7f36e40b6b70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009870 tx=0x7f36d802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0992f3eafa1537 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:35.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.053+0000 7f36e8a00640 1 -- 172.21.15.196:0/4154497509 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8037070 con 0x7f36e406f340 2024-04-28T13:33:35.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.053+0000 7f36e8a00640 1 -- 172.21.15.196:0/4154497509 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36d8032440 con 0x7f36e406f340 2024-04-28T13:33:35.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.053+0000 7f36e8a00640 1 -- 172.21.15.196:0/4154497509 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8032810 con 0x7f36e406f340 2024-04-28T13:33:35.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.053+0000 7f36ec800640 1 -- 172.21.15.196:0/4154497509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36e406f340 msgr2=0x7f36e40b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:35.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.053+0000 7f36ec800640 1 --2- 172.21.15.196:0/4154497509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36e406f340 0x7f36e40b6b70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009870 tx=0x7f36d802a180 comp rx=0 tx=0).stop 2024-04-28T13:33:35.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.054+0000 7f36ec800640 1 -- 172.21.15.196:0/4154497509 shutdown_connections 2024-04-28T13:33:35.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.054+0000 7f36ec800640 1 --2- 172.21.15.196:0/4154497509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36e406f340 0x7f36e40b6b70 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:35.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.054+0000 7f36ec800640 1 -- 172.21.15.196:0/4154497509 >> 172.21.15.196:0/4154497509 conn(0x7f36e40cd8a0 msgr2=0x7f36e40cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:35.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.054+0000 7f36ec800640 1 -- 172.21.15.196:0/4154497509 shutdown_connections 2024-04-28T13:33:35.054 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.054+0000 7f36ec800640 1 -- 172.21.15.196:0/4154497509 wait complete. 2024-04-28T13:33:35.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.055+0000 7f36ec800640 1 Processor -- start 2024-04-28T13:33:35.055 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.056+0000 7f36ec800640 1 -- start start 2024-04-28T13:33:35.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.056+0000 7f36ec800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36e406f340 0x7f36e4073840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:35.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.056+0000 7f36ec800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36e40b3e50 0x7f36e41414f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:35.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.056+0000 7f36ec800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36e40d0300 0x7f36e41f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:35.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.056+0000 7f36ec800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36e4142bb0 con 0x7f36e40b3e50 2024-04-28T13:33:35.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.056+0000 7f36ec800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f36e4142a30 con 0x7f36e40d0300 2024-04-28T13:33:35.057 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.057+0000 7f36ec800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f36e40b4260 con 0x7f36e406f340 2024-04-28T13:33:35.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.057+0000 7f36e9400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36e40b3e50 0x7f36e41414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:35.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.057+0000 7f36e9400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36e40b3e50 0x7f36e41414f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59678/0 (socket says 172.21.15.196:59678) 2024-04-28T13:33:35.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.057+0000 7f36e9400640 1 -- 172.21.15.196:0/1100216277 learned_addr learned my addr 172.21.15.196:0/1100216277 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:35.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.057+0000 7f36ea800640 1 --2- 172.21.15.196:0/1100216277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36e40d0300 0x7f36e41f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:35.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.057+0000 7f36e9e00640 1 --2- 172.21.15.196:0/1100216277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36e406f340 0x7f36e4073840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:35.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.057+0000 7f36e9400640 1 -- 172.21.15.196:0/1100216277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36e406f340 msgr2=0x7f36e4073840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:35.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.057+0000 7f36e9400640 1 --2- 172.21.15.196:0/1100216277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36e406f340 0x7f36e4073840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:35.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.057+0000 7f36e9400640 1 -- 172.21.15.196:0/1100216277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36e40d0300 msgr2=0x7f36e41f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:35.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.057+0000 7f36e9400640 1 --2- 172.21.15.196:0/1100216277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36e40d0300 0x7f36e41f5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:35.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.057+0000 7f36e9400640 1 -- 172.21.15.196:0/1100216277 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e4141a30 con 0x7f36e40b3e50 2024-04-28T13:33:35.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.058+0000 7f36e9400640 1 --2- 172.21.15.196:0/1100216277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36e40b3e50 0x7f36e41414f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f36d400b3e0 tx=0x7f36d400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:35.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.058+0000 7f36de000640 1 -- 172.21.15.196:0/1100216277 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4015020 con 0x7f36e40b3e50 2024-04-28T13:33:35.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.058+0000 7f36de000640 1 -- 172.21.15.196:0/1100216277 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36d4011850 con 0x7f36e40b3e50 2024-04-28T13:33:35.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.058+0000 7f36de000640 1 -- 172.21.15.196:0/1100216277 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4011c40 con 0x7f36e40b3e50 2024-04-28T13:33:35.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.058+0000 7f36ec800640 1 -- 172.21.15.196:0/1100216277 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e41517c0 con 0x7f36e40b3e50 2024-04-28T13:33:35.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.058+0000 7f36ec800640 1 -- 172.21.15.196:0/1100216277 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e4151d80 con 0x7f36e40b3e50 2024-04-28T13:33:35.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.059+0000 7f36ec800640 1 -- 172.21.15.196:0/1100216277 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e406a290 con 0x7f36e40b3e50 2024-04-28T13:33:35.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.060+0000 7f36de000640 1 -- 172.21.15.196:0/1100216277 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f36d4018020 con 0x7f36e40b3e50 2024-04-28T13:33:35.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.061+0000 7f36de000640 1 --2- 172.21.15.196:0/1100216277 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f36c806c980 0x7f36c806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:35.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.061+0000 7f36de000640 1 -- 172.21.15.196:0/1100216277 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f36d408bc10 con 0x7f36e40b3e50 2024-04-28T13:33:35.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.062+0000 7f36e9e00640 1 --2- 172.21.15.196:0/1100216277 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f36c806c980 0x7f36c806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:35.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.063+0000 7f36e9e00640 1 --2- 172.21.15.196:0/1100216277 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f36c806c980 0x7f36c806ee40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f36d8034000 tx=0x7f36d802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:35.065 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.065+0000 7f36de000640 1 -- 172.21.15.196:0/1100216277 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f36d401f020 con 0x7f36e40b3e50 2024-04-28T13:33:35.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:35.309+0000 7f36ec800640 1 -- 172.21.15.196:0/1100216277 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f36e406d2b0 con 0x7f36e40b3e50 2024-04-28T13:33:36.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.006+0000 7f36de000640 1 -- 172.21.15.196:0/1100216277 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) ==== 118+0+0 (secure 0 0 0) 0x7f36d40172c0 con 0x7f36e40b3e50 2024-04-28T13:33:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.010+0000 7f36ec800640 1 -- 172.21.15.196:0/1100216277 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f36c806c980 msgr2=0x7f36c806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.010+0000 7f36ec800640 1 --2- 172.21.15.196:0/1100216277 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f36c806c980 0x7f36c806ee40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f36d8034000 tx=0x7f36d802e000 comp rx=0 tx=0).stop 2024-04-28T13:33:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.010+0000 7f36ec800640 1 -- 172.21.15.196:0/1100216277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36e40b3e50 msgr2=0x7f36e41414f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.010+0000 7f36ec800640 1 --2- 172.21.15.196:0/1100216277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36e40b3e50 0x7f36e41414f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f36d400b3e0 tx=0x7f36d400b8a0 comp rx=0 tx=0).stop 2024-04-28T13:33:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.010+0000 7f36ec800640 1 -- 172.21.15.196:0/1100216277 shutdown_connections 2024-04-28T13:33:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.010+0000 7f36ec800640 1 --2- 172.21.15.196:0/1100216277 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f36c806c980 0x7f36c806ee40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.010+0000 7f36ec800640 1 --2- 172.21.15.196:0/1100216277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36e40d0300 0x7f36e41f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.010+0000 7f36ec800640 1 --2- 172.21.15.196:0/1100216277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36e40b3e50 0x7f36e41414f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.010+0000 7f36ec800640 1 --2- 172.21.15.196:0/1100216277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36e406f340 0x7f36e4073840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.010+0000 7f36ec800640 1 -- 172.21.15.196:0/1100216277 >> 172.21.15.196:0/1100216277 conn(0x7f36e40cd8a0 msgr2=0x7f36e41433e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.010+0000 7f36ec800640 1 -- 172.21.15.196:0/1100216277 shutdown_connections 2024-04-28T13:33:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.010+0000 7f36ec800640 1 -- 172.21.15.196:0/1100216277 wait complete. 2024-04-28T13:33:36.032 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-28T13:33:36.032 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-28T13:33:36.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.233+0000 7f6520000640 1 Processor -- start 2024-04-28T13:33:36.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.234+0000 7f6520000640 1 -- start start 2024-04-28T13:33:36.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.235+0000 7f6520000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f65180b5330 0x7f65180cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:36.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.235+0000 7f6520000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65180b57c0 con 0x7f65180b4eb0 2024-04-28T13:33:36.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.235+0000 7f6520000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f651815dca0 con 0x7f6518143840 2024-04-28T13:33:36.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.235+0000 7f6520000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f6518143c20 con 0x7f65180b5330 2024-04-28T13:33:36.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.236+0000 7f651cc00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f65180b4eb0 0x7f65180ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:38204/0 (socket says 172.21.15.196:38204) 2024-04-28T13:33:36.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.236+0000 7f651cc00640 1 -- 172.21.15.196:0/2984026007 learned_addr learned my addr 172.21.15.196:0/2984026007 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:36.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.236+0000 7f651e000640 1 --2- 172.21.15.196:0/2984026007 >> v2:172.21.15.203:3300/0 conn(0x7f65180b5330 0x7f65180cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:36.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.236+0000 7f6517e00640 1 -- 172.21.15.196:0/2984026007 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 751721963 0 0) 0x7f65180b57c0 con 0x7f65180b4eb0 2024-04-28T13:33:36.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.236+0000 7f6517e00640 1 -- 172.21.15.196:0/2984026007 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6500003520 con 0x7f65180b4eb0 2024-04-28T13:33:36.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.236+0000 7f6517e00640 1 -- 172.21.15.196:0/2984026007 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3225624764 0 0) 0x7f6500003520 con 0x7f65180b4eb0 2024-04-28T13:33:36.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.237+0000 7f6517e00640 1 -- 172.21.15.196:0/2984026007 >> v2:172.21.15.203:3300/0 conn(0x7f65180b5330 msgr2=0x7f65180cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:36.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.237+0000 7f6517e00640 1 --2- 172.21.15.196:0/2984026007 >> v2:172.21.15.203:3300/0 conn(0x7f65180b5330 0x7f65180cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:36.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.237+0000 7f6517e00640 1 -- 172.21.15.196:0/2984026007 >> v1:172.21.15.203:6789/0 conn(0x7f6518143840 legacy=0x7f65180b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:36.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.237+0000 7f6517e00640 1 -- 172.21.15.196:0/2984026007 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651815df80 con 0x7f65180b4eb0 2024-04-28T13:33:36.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.237+0000 7f6517e00640 1 -- 172.21.15.196:0/2984026007 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f6504001670 con 0x7f65180b4eb0 2024-04-28T13:33:36.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.237+0000 7f6517e00640 1 -- 172.21.15.196:0/2984026007 >> v1:172.21.15.196:6789/0 conn(0x7f65180b4eb0 legacy=0x7f65180ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:36.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.237+0000 7f6517e00640 1 --2- 172.21.15.196:0/2984026007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65000039b0 0x7f6500003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:36.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.237+0000 7f6517e00640 1 --2- 172.21.15.196:0/2984026007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6500004800 0x7f6500004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:36.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.237+0000 7f6517e00640 1 --2- 172.21.15.196:0/2984026007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65000051c0 0x7f650000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:36.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.237+0000 7f6517e00640 1 -- 172.21.15.196:0/2984026007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6518143c20 con 0x7f65000051c0 2024-04-28T13:33:36.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.237+0000 7f6517e00640 1 -- 172.21.15.196:0/2984026007 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f650000c060 con 0x7f6500004800 2024-04-28T13:33:36.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.237+0000 7f6517e00640 1 -- 172.21.15.196:0/2984026007 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f650000c260 con 0x7f65000039b0 2024-04-28T13:33:36.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.237+0000 7f651e000640 1 --2- 172.21.15.196:0/2984026007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65000051c0 0x7f650000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:36.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.237+0000 7f651cc00640 1 --2- 172.21.15.196:0/2984026007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6500004800 0x7f6500004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:36.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.237+0000 7f651d600640 1 --2- 172.21.15.196:0/2984026007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65000039b0 0x7f6500003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:36.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.238+0000 7f651e000640 1 -- 172.21.15.196:0/2984026007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65000039b0 msgr2=0x7f6500003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:36.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.238+0000 7f651e000640 1 --2- 172.21.15.196:0/2984026007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65000039b0 0x7f6500003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:36.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.238+0000 7f651e000640 1 -- 172.21.15.196:0/2984026007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6500004800 msgr2=0x7f6500004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:36.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.238+0000 7f651e000640 1 --2- 172.21.15.196:0/2984026007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6500004800 0x7f6500004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:36.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.238+0000 7f651e000640 1 -- 172.21.15.196:0/2984026007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6500003520 con 0x7f65000051c0 2024-04-28T13:33:36.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.238+0000 7f651d600640 1 --2- 172.21.15.196:0/2984026007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65000039b0 0x7f6500003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:36.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.238+0000 7f651e000640 1 -- 172.21.15.196:0/2984026007 reap_dead start 2024-04-28T13:33:36.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.239+0000 7f651e000640 1 --2- 172.21.15.196:0/2984026007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65000051c0 0x7f650000b9c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6510009fc0 tx=0x7f6500005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168e3e944f246d37 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:36.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.239+0000 7f6517e00640 1 -- 172.21.15.196:0/2984026007 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6510035020 con 0x7f65000051c0 2024-04-28T13:33:36.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.239+0000 7f6517e00640 1 -- 172.21.15.196:0/2984026007 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6510003bd0 con 0x7f65000051c0 2024-04-28T13:33:36.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.239+0000 7f6517e00640 1 -- 172.21.15.196:0/2984026007 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6510009c10 con 0x7f65000051c0 2024-04-28T13:33:36.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.239+0000 7f6520000640 1 -- 172.21.15.196:0/2984026007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65000051c0 msgr2=0x7f650000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:36.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.239+0000 7f6520000640 1 --2- 172.21.15.196:0/2984026007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65000051c0 0x7f650000b9c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6510009fc0 tx=0x7f6500005830 comp rx=0 tx=0).stop 2024-04-28T13:33:36.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.240+0000 7f6520000640 1 -- 172.21.15.196:0/2984026007 shutdown_connections 2024-04-28T13:33:36.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.240+0000 7f6520000640 1 --2- 172.21.15.196:0/2984026007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65000051c0 0x7f650000b9c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:36.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.240+0000 7f6520000640 1 -- 172.21.15.196:0/2984026007 >> 172.21.15.196:0/2984026007 conn(0x7f65180cd8a0 msgr2=0x7f65180cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:36.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.240+0000 7f6520000640 1 -- 172.21.15.196:0/2984026007 shutdown_connections 2024-04-28T13:33:36.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.240+0000 7f6520000640 1 -- 172.21.15.196:0/2984026007 wait complete. 2024-04-28T13:33:36.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.241+0000 7f6520000640 1 Processor -- start 2024-04-28T13:33:36.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.242+0000 7f6520000640 1 -- start start 2024-04-28T13:33:36.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.242+0000 7f6520000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65000051c0 0x7f651815a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:36.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.242+0000 7f6520000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65180ca1d0 0x7f65180ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:36.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.243+0000 7f6520000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f651813d590 0x7f651813da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:36.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.243+0000 7f6520000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f651815d8c0 con 0x7f65000051c0 2024-04-28T13:33:36.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.243+0000 7f6520000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f651815e540 con 0x7f65180ca1d0 2024-04-28T13:33:36.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.243+0000 7f6520000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f651815da40 con 0x7f651813d590 2024-04-28T13:33:36.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.243+0000 7f651d600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65000051c0 0x7f651815a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:36.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.243+0000 7f651d600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65000051c0 0x7f651815a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59704/0 (socket says 172.21.15.196:59704) 2024-04-28T13:33:36.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.243+0000 7f651d600640 1 -- 172.21.15.196:0/2976596322 learned_addr learned my addr 172.21.15.196:0/2976596322 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:36.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.243+0000 7f651d600640 1 -- 172.21.15.196:0/2976596322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f651813d590 msgr2=0x7f651813da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:36.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.243+0000 7f651d600640 1 --2- 172.21.15.196:0/2976596322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f651813d590 0x7f651813da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:36.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.243+0000 7f651cc00640 1 --2- 172.21.15.196:0/2976596322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65180ca1d0 0x7f65180ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:36.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.243+0000 7f651d600640 1 -- 172.21.15.196:0/2976596322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65180ca1d0 msgr2=0x7f65180ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:36.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.243+0000 7f651d600640 1 --2- 172.21.15.196:0/2976596322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65180ca1d0 0x7f65180ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:36.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.243+0000 7f651d600640 1 -- 172.21.15.196:0/2976596322 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65180cc2d0 con 0x7f65000051c0 2024-04-28T13:33:36.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.243+0000 7f651d600640 1 --2- 172.21.15.196:0/2976596322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65000051c0 0x7f651815a690 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f650c00d770 tx=0x7f650c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:36.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.244+0000 7f6515600640 1 -- 172.21.15.196:0/2976596322 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c005270 con 0x7f65000051c0 2024-04-28T13:33:36.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.244+0000 7f6515600640 1 -- 172.21.15.196:0/2976596322 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f650c017070 con 0x7f65000051c0 2024-04-28T13:33:36.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.244+0000 7f6515600640 1 -- 172.21.15.196:0/2976596322 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c00b6b0 con 0x7f65000051c0 2024-04-28T13:33:36.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.244+0000 7f6520000640 1 -- 172.21.15.196:0/2976596322 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651815abd0 con 0x7f65000051c0 2024-04-28T13:33:36.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.244+0000 7f6520000640 1 -- 172.21.15.196:0/2976596322 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65180cc850 con 0x7f65000051c0 2024-04-28T13:33:36.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.245+0000 7f6520000640 1 -- 172.21.15.196:0/2976596322 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f651806a290 con 0x7f65000051c0 2024-04-28T13:33:36.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.246+0000 7f6515600640 1 -- 172.21.15.196:0/2976596322 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f650c00b850 con 0x7f65000051c0 2024-04-28T13:33:36.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.247+0000 7f6515600640 1 --2- 172.21.15.196:0/2976596322 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f64f806c9b0 0x7f64f806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:36.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.247+0000 7f6515600640 1 -- 172.21.15.196:0/2976596322 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f650c08bbc0 con 0x7f65000051c0 2024-04-28T13:33:36.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.248+0000 7f651cc00640 1 --2- 172.21.15.196:0/2976596322 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f64f806c9b0 0x7f64f806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:36.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.249+0000 7f651cc00640 1 --2- 172.21.15.196:0/2976596322 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f64f806c9b0 0x7f64f806ee70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6504001eb0 tx=0x7f6504000ef0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:36.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.251+0000 7f6515600640 1 -- 172.21.15.196:0/2976596322 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f650c019310 con 0x7f65000051c0 2024-04-28T13:33:36.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:36.490+0000 7f6520000640 1 -- 172.21.15.196:0/2976596322 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f65180ce420 con 0x7f65000051c0 2024-04-28T13:33:37.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.017+0000 7f6515600640 1 -- 172.21.15.196:0/2976596322 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) ==== 122+0+0 (secure 0 0 0) 0x7f650c001230 con 0x7f65000051c0 2024-04-28T13:33:37.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.021+0000 7f6520000640 1 -- 172.21.15.196:0/2976596322 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f64f806c9b0 msgr2=0x7f64f806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:37.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.021+0000 7f6520000640 1 --2- 172.21.15.196:0/2976596322 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f64f806c9b0 0x7f64f806ee70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6504001eb0 tx=0x7f6504000ef0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.021+0000 7f6520000640 1 -- 172.21.15.196:0/2976596322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65000051c0 msgr2=0x7f651815a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:37.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.021+0000 7f6520000640 1 --2- 172.21.15.196:0/2976596322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65000051c0 0x7f651815a690 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f650c00d770 tx=0x7f650c00dc30 comp rx=0 tx=0).stop 2024-04-28T13:33:37.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.021+0000 7f6520000640 1 -- 172.21.15.196:0/2976596322 shutdown_connections 2024-04-28T13:33:37.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.021+0000 7f6520000640 1 --2- 172.21.15.196:0/2976596322 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f64f806c9b0 0x7f64f806ee70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.021+0000 7f6520000640 1 --2- 172.21.15.196:0/2976596322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f651813d590 0x7f651813da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.021+0000 7f6520000640 1 --2- 172.21.15.196:0/2976596322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65180ca1d0 0x7f65180ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.021+0000 7f6520000640 1 --2- 172.21.15.196:0/2976596322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65000051c0 0x7f651815a690 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.021+0000 7f6520000640 1 -- 172.21.15.196:0/2976596322 >> 172.21.15.196:0/2976596322 conn(0x7f65180cd8a0 msgr2=0x7f6518141460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:37.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.021+0000 7f6520000640 1 -- 172.21.15.196:0/2976596322 shutdown_connections 2024-04-28T13:33:37.022 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.021+0000 7f6520000640 1 -- 172.21.15.196:0/2976596322 wait complete. 2024-04-28T13:33:37.045 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-28T13:33:37.046 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:33:37.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.244+0000 7fd807e00640 1 Processor -- start 2024-04-28T13:33:37.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.244+0000 7fd807e00640 1 -- start start 2024-04-28T13:33:37.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.246+0000 7fd807e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8080b3c20 0x7fd8080b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:37.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.246+0000 7fd807e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8080b47a0 con 0x7fd808073370 2024-04-28T13:33:37.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.246+0000 7fd807e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8080738e0 con 0x7fd8080d0540 2024-04-28T13:33:37.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.246+0000 7fd807e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd8080aea80 con 0x7fd8080b3c20 2024-04-28T13:33:37.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.246+0000 7fd806000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fd808073370 0x7fd808072900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:38212/0 (socket says 172.21.15.196:38212) 2024-04-28T13:33:37.246 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.246+0000 7fd806000640 1 -- 172.21.15.196:0/3942770263 learned_addr learned my addr 172.21.15.196:0/3942770263 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:37.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.246+0000 7fd805600640 1 -- 172.21.15.196:0/3942770263 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3913251005 0 0) 0x7fd8080b47a0 con 0x7fd808073370 2024-04-28T13:33:37.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.246+0000 7fd807400640 1 --2- 172.21.15.196:0/3942770263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8080b3c20 0x7fd8080b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:37.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.246+0000 7fd805600640 1 -- 172.21.15.196:0/3942770263 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7ec003520 con 0x7fd808073370 2024-04-28T13:33:37.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.247+0000 7fd805600640 1 -- 172.21.15.196:0/3942770263 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2499591930 0 0) 0x7fd7ec003520 con 0x7fd808073370 2024-04-28T13:33:37.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.247+0000 7fd805600640 1 -- 172.21.15.196:0/3942770263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8080b3c20 msgr2=0x7fd8080b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:37.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.247+0000 7fd805600640 1 --2- 172.21.15.196:0/3942770263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8080b3c20 0x7fd8080b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.247+0000 7fd805600640 1 -- 172.21.15.196:0/3942770263 >> v1:172.21.15.203:6789/0 conn(0x7fd8080d0540 legacy=0x7fd8080af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:37.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.247+0000 7fd805600640 1 -- 172.21.15.196:0/3942770263 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8080aed60 con 0x7fd808073370 2024-04-28T13:33:37.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.247+0000 7fd805600640 1 -- 172.21.15.196:0/3942770263 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fd7f0001670 con 0x7fd808073370 2024-04-28T13:33:37.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.247+0000 7fd805600640 1 -- 172.21.15.196:0/3942770263 >> v1:172.21.15.196:6789/0 conn(0x7fd808073370 legacy=0x7fd808072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:37.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.247+0000 7fd805600640 1 --2- 172.21.15.196:0/3942770263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8080b3c20 0x7fd7ec0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:37.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.247+0000 7fd805600640 1 --2- 172.21.15.196:0/3942770263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7ec004010 0x7fd7ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:37.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.247+0000 7fd805600640 1 --2- 172.21.15.196:0/3942770263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7ec004d40 0x7fd7ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:37.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.247+0000 7fd805600640 1 -- 172.21.15.196:0/3942770263 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd8080aea80 con 0x7fd7ec004010 2024-04-28T13:33:37.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.247+0000 7fd805600640 1 -- 172.21.15.196:0/3942770263 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec00ff90 con 0x7fd7ec004d40 2024-04-28T13:33:37.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.247+0000 7fd805600640 1 -- 172.21.15.196:0/3942770263 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd7ec010190 con 0x7fd8080b3c20 2024-04-28T13:33:37.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.247+0000 7fd806000640 1 --2- 172.21.15.196:0/3942770263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7ec004010 0x7fd7ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:37.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.247+0000 7fd806a00640 1 --2- 172.21.15.196:0/3942770263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8080b3c20 0x7fd7ec0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:37.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.248+0000 7fd807400640 1 --2- 172.21.15.196:0/3942770263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7ec004d40 0x7fd7ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:37.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.248+0000 7fd806000640 1 -- 172.21.15.196:0/3942770263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8080b3c20 msgr2=0x7fd7ec0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:37.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.248+0000 7fd806000640 1 --2- 172.21.15.196:0/3942770263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8080b3c20 0x7fd7ec0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.248+0000 7fd806000640 1 -- 172.21.15.196:0/3942770263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7ec004d40 msgr2=0x7fd7ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:37.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.248+0000 7fd806000640 1 --2- 172.21.15.196:0/3942770263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7ec004d40 0x7fd7ec0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.248+0000 7fd806000640 1 -- 172.21.15.196:0/3942770263 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ec003520 con 0x7fd7ec004010 2024-04-28T13:33:37.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.249+0000 7fd806000640 1 --2- 172.21.15.196:0/3942770263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7ec004010 0x7fd7ec004800 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000c210 tx=0x7fd7ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd667bbcb0f44b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:37.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.249+0000 7fd805600640 1 -- 172.21.15.196:0/3942770263 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0007d30 con 0x7fd7ec004010 2024-04-28T13:33:37.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.249+0000 7fd805600640 1 -- 172.21.15.196:0/3942770263 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f0005ad0 con 0x7fd7ec004010 2024-04-28T13:33:37.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.249+0000 7fd805600640 1 -- 172.21.15.196:0/3942770263 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0007070 con 0x7fd7ec004010 2024-04-28T13:33:37.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.249+0000 7fd807e00640 1 -- 172.21.15.196:0/3942770263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7ec004010 msgr2=0x7fd7ec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:37.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.249+0000 7fd807e00640 1 --2- 172.21.15.196:0/3942770263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7ec004010 0x7fd7ec004800 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000c210 tx=0x7fd7ec005830 comp rx=0 tx=0).stop 2024-04-28T13:33:37.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.249+0000 7fd807400640 1 -- 172.21.15.196:0/3942770263 reap_dead start 2024-04-28T13:33:37.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.249+0000 7fd807e00640 1 -- 172.21.15.196:0/3942770263 shutdown_connections 2024-04-28T13:33:37.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.250+0000 7fd807e00640 1 -- 172.21.15.196:0/3942770263 >> 172.21.15.196:0/3942770263 conn(0x7fd80806c6c0 msgr2=0x7fd80806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:37.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.250+0000 7fd807e00640 1 -- 172.21.15.196:0/3942770263 shutdown_connections 2024-04-28T13:33:37.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.250+0000 7fd807e00640 1 -- 172.21.15.196:0/3942770263 wait complete. 2024-04-28T13:33:37.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.251+0000 7fd807e00640 1 Processor -- start 2024-04-28T13:33:37.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.251+0000 7fd807e00640 1 -- start start 2024-04-28T13:33:37.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.252+0000 7fd807e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7ec004010 0x7fd80809cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:37.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.252+0000 7fd807e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8081e5580 0x7fd8081435e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:37.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.252+0000 7fd807e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd80815cc10 0x7fd80815d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:37.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.252+0000 7fd807e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd8080d0920 con 0x7fd8081e5580 2024-04-28T13:33:37.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.252+0000 7fd807e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd8080ae6c0 con 0x7fd7ec004010 2024-04-28T13:33:37.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.252+0000 7fd807e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd8081e5a00 con 0x7fd80815cc10 2024-04-28T13:33:37.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.253+0000 7fd806000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8081e5580 0x7fd8081435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:37.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.253+0000 7fd806000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8081e5580 0x7fd8081435e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59722/0 (socket says 172.21.15.196:59722) 2024-04-28T13:33:37.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.253+0000 7fd806000640 1 -- 172.21.15.196:0/1868886992 learned_addr learned my addr 172.21.15.196:0/1868886992 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:37.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.253+0000 7fd806a00640 1 --2- 172.21.15.196:0/1868886992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7ec004010 0x7fd80809cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:37.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.253+0000 7fd807400640 1 --2- 172.21.15.196:0/1868886992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd80815cc10 0x7fd80815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:37.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.253+0000 7fd806000640 1 -- 172.21.15.196:0/1868886992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd80815cc10 msgr2=0x7fd80815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:37.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.253+0000 7fd806000640 1 --2- 172.21.15.196:0/1868886992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd80815cc10 0x7fd80815d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.253+0000 7fd806000640 1 -- 172.21.15.196:0/1868886992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7ec004010 msgr2=0x7fd80809cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:37.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.253+0000 7fd806000640 1 --2- 172.21.15.196:0/1868886992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7ec004010 0x7fd80809cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.253+0000 7fd806000640 1 -- 172.21.15.196:0/1868886992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80809d2c0 con 0x7fd8081e5580 2024-04-28T13:33:37.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.253+0000 7fd806a00640 1 --2- 172.21.15.196:0/1868886992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7ec004010 0x7fd80809cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:37.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.253+0000 7fd806000640 1 --2- 172.21.15.196:0/1868886992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8081e5580 0x7fd8081435e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000c330 tx=0x7fd7f0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:37.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.254+0000 7fd7fea00640 1 -- 172.21.15.196:0/1868886992 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0042020 con 0x7fd8081e5580 2024-04-28T13:33:37.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.254+0000 7fd7fea00640 1 -- 172.21.15.196:0/1868886992 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f0004bf0 con 0x7fd8081e5580 2024-04-28T13:33:37.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.254+0000 7fd7fea00640 1 -- 172.21.15.196:0/1868886992 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0007d10 con 0x7fd8081e5580 2024-04-28T13:33:37.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.254+0000 7fd807e00640 1 -- 172.21.15.196:0/1868886992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80809d5a0 con 0x7fd8081e5580 2024-04-28T13:33:37.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.254+0000 7fd807e00640 1 -- 172.21.15.196:0/1868886992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80809db60 con 0x7fd8081e5580 2024-04-28T13:33:37.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.255+0000 7fd807e00640 1 -- 172.21.15.196:0/1868886992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd80806a290 con 0x7fd8081e5580 2024-04-28T13:33:37.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.256+0000 7fd7fea00640 1 -- 172.21.15.196:0/1868886992 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd7f0045020 con 0x7fd8081e5580 2024-04-28T13:33:37.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.257+0000 7fd7fea00640 1 --2- 172.21.15.196:0/1868886992 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7e406c980 0x7fd7e406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:37.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.257+0000 7fd7fea00640 1 -- 172.21.15.196:0/1868886992 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fd7f00abcf0 con 0x7fd8081e5580 2024-04-28T13:33:37.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.257+0000 7fd806a00640 1 --2- 172.21.15.196:0/1868886992 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7e406c980 0x7fd7e406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:37.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.258+0000 7fd806a00640 1 --2- 172.21.15.196:0/1868886992 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7e406c980 0x7fd7e406ee40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8001d60 tx=0x7fd7f8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:37.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.261+0000 7fd7fea00640 1 -- 172.21.15.196:0/1868886992 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd7f004c020 con 0x7fd8081e5580 2024-04-28T13:33:37.502 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.502+0000 7fd807e00640 1 -- 172.21.15.196:0/1868886992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd80806e670 con 0x7fd8081e5580 2024-04-28T13:33:37.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.504+0000 7fd7fea00640 1 -- 172.21.15.196:0/1868886992 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5193 (secure 0 0 0) 0x7fd7f0007070 con 0x7fd8081e5580 2024-04-28T13:33:37.504 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:37.504 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2957566398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2957566398},{"type":"v1","addr":"172.21.15.203:6836","nonce":2957566398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5766,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1101263363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1101263363},{"type":"v1","addr":"172.21.15.196:6834","nonce":1101263363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1157781293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1157781293},{"type":"v1","addr":"172.21.15.196:6835","nonce":1157781293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:33:33.440220+0000","modified":"2024-04-28T13:33:36.491827+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5781},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5781":{"gid":5781,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/1525791340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1525791340},{"type":"v1","addr":"172.21.15.203:6837","nonce":1525791340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5781,"qdb_cluster":[5781]},"id":3}]} 2024-04-28T13:33:37.504 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 31 2024-04-28T13:33:37.507 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.507+0000 7fd807e00640 1 -- 172.21.15.196:0/1868886992 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7e406c980 msgr2=0x7fd7e406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:37.507 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.507+0000 7fd807e00640 1 --2- 172.21.15.196:0/1868886992 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7e406c980 0x7fd7e406ee40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8001d60 tx=0x7fd7f8009040 comp rx=0 tx=0).stop 2024-04-28T13:33:37.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.507+0000 7fd807e00640 1 -- 172.21.15.196:0/1868886992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8081e5580 msgr2=0x7fd8081435e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:37.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.507+0000 7fd807e00640 1 --2- 172.21.15.196:0/1868886992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8081e5580 0x7fd8081435e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000c330 tx=0x7fd7f0001a40 comp rx=0 tx=0).stop 2024-04-28T13:33:37.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.507+0000 7fd807e00640 1 -- 172.21.15.196:0/1868886992 shutdown_connections 2024-04-28T13:33:37.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.507+0000 7fd807e00640 1 --2- 172.21.15.196:0/1868886992 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd7e406c980 0x7fd7e406ee40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.507+0000 7fd807e00640 1 --2- 172.21.15.196:0/1868886992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd80815cc10 0x7fd80815d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.507+0000 7fd807e00640 1 --2- 172.21.15.196:0/1868886992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8081e5580 0x7fd8081435e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.507+0000 7fd807e00640 1 --2- 172.21.15.196:0/1868886992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7ec004010 0x7fd80809cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.507+0000 7fd807e00640 1 -- 172.21.15.196:0/1868886992 >> 172.21.15.196:0/1868886992 conn(0x7fd80806c6c0 msgr2=0x7fd80806dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:37.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.507+0000 7fd807e00640 1 -- 172.21.15.196:0/1868886992 shutdown_connections 2024-04-28T13:33:37.508 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.507+0000 7fd807e00640 1 -- 172.21.15.196:0/1868886992 wait complete. 2024-04-28T13:33:37.526 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:33:37.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.720+0000 7fbf6b200640 1 Processor -- start 2024-04-28T13:33:37.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.720+0000 7fbf6b200640 1 -- start start 2024-04-28T13:33:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.721+0000 7fbf6b200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbf6c0d0540 0x7fbf6c0af560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.721+0000 7fbf6b200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf6c0afaa0 con 0x7fbf6c0b3c20 2024-04-28T13:33:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.722+0000 7fbf6b200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf6c0b47a0 con 0x7fbf6c073370 2024-04-28T13:33:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.722+0000 7fbf6b200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbf6c0738e0 con 0x7fbf6c0d0540 2024-04-28T13:33:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.722+0000 7fbf69e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbf6c0d0540 0x7fbf6c0af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.722+0000 7fbf6a800640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fbf6c0b3c20 0x7fbf6c06ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:38228/0 (socket says 172.21.15.196:38228) 2024-04-28T13:33:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.722+0000 7fbf6a800640 1 -- 172.21.15.196:0/172275195 learned_addr learned my addr 172.21.15.196:0/172275195 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.722+0000 7fbf69e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbf6c0d0540 0x7fbf6c0af560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59724/0 (socket says 172.21.15.196:59724) 2024-04-28T13:33:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.722+0000 7fbf69e00640 1 -- 172.21.15.196:0/172275195 >> v1:172.21.15.203:6789/0 conn(0x7fbf6c073370 legacy=0x7fbf6c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.722+0000 7fbf69e00640 1 -- 172.21.15.196:0/172275195 >> v1:172.21.15.196:6789/0 conn(0x7fbf6c0b3c20 legacy=0x7fbf6c06ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.722+0000 7fbf69e00640 1 -- 172.21.15.196:0/172275195 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c0aea80 con 0x7fbf6c0d0540 2024-04-28T13:33:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.723+0000 7fbf69e00640 1 --2- 172.21.15.196:0/172275195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c0d0540 0x7fbf6c0af560 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0097a0 tx=0x7fbf5c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35695ee95c07a615 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.724+0000 7fbf68a00640 1 -- 172.21.15.196:0/172275195 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c037070 con 0x7fbf6c0d0540 2024-04-28T13:33:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.724+0000 7fbf68a00640 1 -- 172.21.15.196:0/172275195 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf5c032440 con 0x7fbf6c0d0540 2024-04-28T13:33:37.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.724+0000 7fbf68a00640 1 -- 172.21.15.196:0/172275195 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c032810 con 0x7fbf6c0d0540 2024-04-28T13:33:37.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.724+0000 7fbf6b200640 1 -- 172.21.15.196:0/172275195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c0d0540 msgr2=0x7fbf6c0af560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:37.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.724+0000 7fbf6b200640 1 --2- 172.21.15.196:0/172275195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c0d0540 0x7fbf6c0af560 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0097a0 tx=0x7fbf5c02a1e0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.724+0000 7fbf6b200640 1 -- 172.21.15.196:0/172275195 shutdown_connections 2024-04-28T13:33:37.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.724+0000 7fbf6b200640 1 --2- 172.21.15.196:0/172275195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c0d0540 0x7fbf6c0af560 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.724+0000 7fbf6b200640 1 -- 172.21.15.196:0/172275195 >> 172.21.15.196:0/172275195 conn(0x7fbf6c06c6c0 msgr2=0x7fbf6c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:37.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.725+0000 7fbf6b200640 1 -- 172.21.15.196:0/172275195 shutdown_connections 2024-04-28T13:33:37.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.725+0000 7fbf6b200640 1 -- 172.21.15.196:0/172275195 wait complete. 2024-04-28T13:33:37.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.726+0000 7fbf6b200640 1 Processor -- start 2024-04-28T13:33:37.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.726+0000 7fbf6b200640 1 -- start start 2024-04-28T13:33:37.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.727+0000 7fbf6b200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf6c073370 0x7fbf6c09d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.727+0000 7fbf6b200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c0b3c20 0x7fbf6c1445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.727+0000 7fbf6b200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf6c0d0540 0x7fbf6c1e8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.727+0000 7fbf6b200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c0d0920 con 0x7fbf6c0b3c20 2024-04-28T13:33:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.727+0000 7fbf6b200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c0af040 con 0x7fbf6c073370 2024-04-28T13:33:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.727+0000 7fbf6b200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6c09da70 con 0x7fbf6c0d0540 2024-04-28T13:33:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.728+0000 7fbf69400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c0b3c20 0x7fbf6c1445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.728+0000 7fbf69400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c0b3c20 0x7fbf6c1445f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59726/0 (socket says 172.21.15.196:59726) 2024-04-28T13:33:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.728+0000 7fbf69400640 1 -- 172.21.15.196:0/3057456823 learned_addr learned my addr 172.21.15.196:0/3057456823 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.728+0000 7fbf69e00640 1 --2- 172.21.15.196:0/3057456823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf6c073370 0x7fbf6c09d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.728+0000 7fbf69400640 1 -- 172.21.15.196:0/3057456823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf6c0d0540 msgr2=0x7fbf6c1e8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.728+0000 7fbf69400640 1 --2- 172.21.15.196:0/3057456823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf6c0d0540 0x7fbf6c1e8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.728+0000 7fbf69400640 1 -- 172.21.15.196:0/3057456823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf6c073370 msgr2=0x7fbf6c09d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.728+0000 7fbf69400640 1 --2- 172.21.15.196:0/3057456823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf6c073370 0x7fbf6c09d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.728+0000 7fbf69400640 1 -- 172.21.15.196:0/3057456823 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c1e95b0 con 0x7fbf6c0b3c20 2024-04-28T13:33:37.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.728+0000 7fbf69e00640 1 --2- 172.21.15.196:0/3057456823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf6c073370 0x7fbf6c09d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:37.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.728+0000 7fbf69400640 1 --2- 172.21.15.196:0/3057456823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c0b3c20 0x7fbf6c1445f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800b3f0 tx=0x7fbf5800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:37.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.729+0000 7fbf62000640 1 -- 172.21.15.196:0/3057456823 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58015020 con 0x7fbf6c0b3c20 2024-04-28T13:33:37.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.729+0000 7fbf62000640 1 -- 172.21.15.196:0/3057456823 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf58011830 con 0x7fbf6c0b3c20 2024-04-28T13:33:37.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.729+0000 7fbf62000640 1 -- 172.21.15.196:0/3057456823 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58011c20 con 0x7fbf6c0b3c20 2024-04-28T13:33:37.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.729+0000 7fbf6b200640 1 -- 172.21.15.196:0/3057456823 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6c09c770 con 0x7fbf6c0b3c20 2024-04-28T13:33:37.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.729+0000 7fbf6b200640 1 -- 172.21.15.196:0/3057456823 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6c09cd30 con 0x7fbf6c0b3c20 2024-04-28T13:33:37.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.730+0000 7fbf6b200640 1 -- 172.21.15.196:0/3057456823 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6c06a290 con 0x7fbf6c0b3c20 2024-04-28T13:33:37.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.731+0000 7fbf62000640 1 -- 172.21.15.196:0/3057456823 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fbf58018020 con 0x7fbf6c0b3c20 2024-04-28T13:33:37.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.732+0000 7fbf62000640 1 --2- 172.21.15.196:0/3057456823 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbf4c06c980 0x7fbf4c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:37.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.732+0000 7fbf62000640 1 -- 172.21.15.196:0/3057456823 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fbf5808b800 con 0x7fbf6c0b3c20 2024-04-28T13:33:37.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.733+0000 7fbf69e00640 1 --2- 172.21.15.196:0/3057456823 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbf4c06c980 0x7fbf4c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:37.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.734+0000 7fbf69e00640 1 --2- 172.21.15.196:0/3057456823 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbf4c06c980 0x7fbf4c06ee40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c034000 tx=0x7fbf5c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:37.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.737+0000 7fbf62000640 1 -- 172.21.15.196:0/3057456823 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbf5801f030 con 0x7fbf6c0b3c20 2024-04-28T13:33:37.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.942+0000 7fbf6b200640 1 -- 172.21.15.196:0/3057456823 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf6c15f830 con 0x7fbf6c0b3c20 2024-04-28T13:33:37.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.945+0000 7fbf62000640 1 -- 172.21.15.196:0/3057456823 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16649 (secure 0 0 0) 0x7fbf580172c0 con 0x7fbf6c0b3c20 2024-04-28T13:33:37.945 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:37.945 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":45,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:33:34.981346+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:33:31.193265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-28T13:33:32.415392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:33:37.947 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.947+0000 7fbf6b200640 1 -- 172.21.15.196:0/3057456823 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbf4c06c980 msgr2=0x7fbf4c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:37.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.947+0000 7fbf6b200640 1 --2- 172.21.15.196:0/3057456823 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbf4c06c980 0x7fbf4c06ee40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c034000 tx=0x7fbf5c02e000 comp rx=0 tx=0).stop 2024-04-28T13:33:37.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.947+0000 7fbf6b200640 1 -- 172.21.15.196:0/3057456823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c0b3c20 msgr2=0x7fbf6c1445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:37.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.947+0000 7fbf6b200640 1 --2- 172.21.15.196:0/3057456823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c0b3c20 0x7fbf6c1445f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800b3f0 tx=0x7fbf5800b8b0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.947+0000 7fbf6b200640 1 -- 172.21.15.196:0/3057456823 shutdown_connections 2024-04-28T13:33:37.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.947+0000 7fbf6b200640 1 --2- 172.21.15.196:0/3057456823 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbf4c06c980 0x7fbf4c06ee40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.947+0000 7fbf6b200640 1 --2- 172.21.15.196:0/3057456823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf6c0d0540 0x7fbf6c1e8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.947+0000 7fbf6b200640 1 --2- 172.21.15.196:0/3057456823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf6c0b3c20 0x7fbf6c1445f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.947+0000 7fbf6b200640 1 --2- 172.21.15.196:0/3057456823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf6c073370 0x7fbf6c09d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:37.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.947+0000 7fbf6b200640 1 -- 172.21.15.196:0/3057456823 >> 172.21.15.196:0/3057456823 conn(0x7fbf6c06c6c0 msgr2=0x7fbf6c0b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:37.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.948+0000 7fbf6b200640 1 -- 172.21.15.196:0/3057456823 shutdown_connections 2024-04-28T13:33:37.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:37.948+0000 7fbf6b200640 1 -- 172.21.15.196:0/3057456823 wait complete. 2024-04-28T13:33:37.968 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-28T13:33:37.968 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T13:33:38.170 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.170+0000 7f787dc00640 1 Processor -- start 2024-04-28T13:33:38.170 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.170+0000 7f787dc00640 1 -- start start 2024-04-28T13:33:38.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.172+0000 7f787dc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f78780d0d00 0x7f78780d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:38.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.172+0000 7f787dc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7878160f40 con 0x7f78781469f0 2024-04-28T13:33:38.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.172+0000 7f787dc00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7878161220 con 0x7f78780b5dd0 2024-04-28T13:33:38.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.172+0000 7f787dc00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f78780b5960 con 0x7f78780d0d00 2024-04-28T13:33:38.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.172+0000 7f786fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f78780d0d00 0x7f78780d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:38.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.172+0000 7f7876a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f78781469f0 0x7f78780ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:38232/0 (socket says 172.21.15.196:38232) 2024-04-28T13:33:38.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.172+0000 7f7876a00640 1 -- 172.21.15.196:0/2912497332 learned_addr learned my addr 172.21.15.196:0/2912497332 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:38.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.172+0000 7f7876000640 1 -- 172.21.15.196:0/2912497332 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 78868972 0 0) 0x7f7878160f40 con 0x7f78781469f0 2024-04-28T13:33:38.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.173+0000 7f7876000640 1 -- 172.21.15.196:0/2912497332 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f785c003520 con 0x7f78781469f0 2024-04-28T13:33:38.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.173+0000 7f786fe00640 1 -- 172.21.15.196:0/2912497332 >> v1:172.21.15.203:6789/0 conn(0x7f78780b5dd0 legacy=0x7f78780b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:38.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.173+0000 7f786fe00640 1 -- 172.21.15.196:0/2912497332 >> v1:172.21.15.196:6789/0 conn(0x7f78781469f0 legacy=0x7f78780ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:38.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.173+0000 7f786fe00640 1 -- 172.21.15.196:0/2912497332 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7878161500 con 0x7f78780d0d00 2024-04-28T13:33:38.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.174+0000 7f786fe00640 1 --2- 172.21.15.196:0/2912497332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78780d0d00 0x7f78780d1190 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f78600097a0 tx=0x7f786002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0275e0bb0a4f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:38.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.174+0000 7f7876000640 1 -- 172.21.15.196:0/2912497332 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860037070 con 0x7f78780d0d00 2024-04-28T13:33:38.174 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.174+0000 7f7876000640 1 -- 172.21.15.196:0/2912497332 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7860032440 con 0x7f78780d0d00 2024-04-28T13:33:38.174 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.174+0000 7f7876000640 1 -- 172.21.15.196:0/2912497332 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78600327f0 con 0x7f78780d0d00 2024-04-28T13:33:38.174 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.174+0000 7f787dc00640 1 -- 172.21.15.196:0/2912497332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78780d0d00 msgr2=0x7f78780d1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:38.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.174+0000 7f787dc00640 1 --2- 172.21.15.196:0/2912497332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78780d0d00 0x7f78780d1190 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f78600097a0 tx=0x7f786002a1e0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.175+0000 7f787dc00640 1 -- 172.21.15.196:0/2912497332 shutdown_connections 2024-04-28T13:33:38.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.175+0000 7f787dc00640 1 --2- 172.21.15.196:0/2912497332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78780d0d00 0x7f78780d1190 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.175+0000 7f787dc00640 1 -- 172.21.15.196:0/2912497332 >> 172.21.15.196:0/2912497332 conn(0x7f78780cd830 msgr2=0x7f78780cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:38.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.175+0000 7f787dc00640 1 -- 172.21.15.196:0/2912497332 shutdown_connections 2024-04-28T13:33:38.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.175+0000 7f787dc00640 1 -- 172.21.15.196:0/2912497332 wait complete. 2024-04-28T13:33:38.176 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.176+0000 7f787dc00640 1 Processor -- start 2024-04-28T13:33:38.176 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.177+0000 7f787dc00640 1 -- start start 2024-04-28T13:33:38.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.177+0000 7f787dc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78780b5dd0 0x7f78781f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:38.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.177+0000 7f787dc00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78780d0d00 0x7f78781f8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:38.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.178+0000 7f787dc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78781469f0 0x7f78780b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:38.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.178+0000 7f787dc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78780cd5f0 con 0x7f78780b5dd0 2024-04-28T13:33:38.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.178+0000 7f787dc00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7878161ac0 con 0x7f78781469f0 2024-04-28T13:33:38.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.178+0000 7f787dc00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7878072020 con 0x7f78780d0d00 2024-04-28T13:33:38.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.178+0000 7f7876a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78780b5dd0 0x7f78781f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:38.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.178+0000 7f7876a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78780b5dd0 0x7f78781f6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59754/0 (socket says 172.21.15.196:59754) 2024-04-28T13:33:38.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.178+0000 7f7876a00640 1 -- 172.21.15.196:0/3606728484 learned_addr learned my addr 172.21.15.196:0/3606728484 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:38.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.178+0000 7f7876a00640 1 -- 172.21.15.196:0/3606728484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78780d0d00 msgr2=0x7f78781f8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:38.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.178+0000 7f7876a00640 1 --2- 172.21.15.196:0/3606728484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78780d0d00 0x7f78781f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.178+0000 7f7876a00640 1 -- 172.21.15.196:0/3606728484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78781469f0 msgr2=0x7f78780b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:38.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.178+0000 7f7876a00640 1 --2- 172.21.15.196:0/3606728484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78781469f0 0x7f78780b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.178+0000 7f7876a00640 1 -- 172.21.15.196:0/3606728484 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78781f5d20 con 0x7f78780b5dd0 2024-04-28T13:33:38.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.178+0000 7f7876a00640 1 --2- 172.21.15.196:0/3606728484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78780b5dd0 0x7f78781f6970 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f786800c390 tx=0x7f786800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:38.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.179+0000 7f786ea00640 1 -- 172.21.15.196:0/3606728484 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868017070 con 0x7f78780b5dd0 2024-04-28T13:33:38.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.179+0000 7f786ea00640 1 -- 172.21.15.196:0/3606728484 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7868005ce0 con 0x7f78780b5dd0 2024-04-28T13:33:38.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.179+0000 7f786ea00640 1 -- 172.21.15.196:0/3606728484 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78680072f0 con 0x7f78780b5dd0 2024-04-28T13:33:38.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.179+0000 7f787dc00640 1 -- 172.21.15.196:0/3606728484 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78781f6000 con 0x7f78780b5dd0 2024-04-28T13:33:38.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.179+0000 7f787dc00640 1 -- 172.21.15.196:0/3606728484 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78781f65c0 con 0x7f78780b5dd0 2024-04-28T13:33:38.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.180+0000 7f787dc00640 1 -- 172.21.15.196:0/3606728484 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f787806a290 con 0x7f78780b5dd0 2024-04-28T13:33:38.181 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.181+0000 7f786ea00640 1 -- 172.21.15.196:0/3606728484 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f786801a070 con 0x7f78780b5dd0 2024-04-28T13:33:38.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.182+0000 7f786ea00640 1 --2- 172.21.15.196:0/3606728484 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f785406c980 0x7f785406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:38.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.182+0000 7f786ea00640 1 -- 172.21.15.196:0/3606728484 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f786808af20 con 0x7f78780b5dd0 2024-04-28T13:33:38.182 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.183+0000 7f786fe00640 1 --2- 172.21.15.196:0/3606728484 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f785406c980 0x7f785406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:38.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.184+0000 7f786fe00640 1 --2- 172.21.15.196:0/3606728484 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f785406c980 0x7f785406ee40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7860034000 tx=0x7f786002c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:38.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.187+0000 7f786ea00640 1 -- 172.21.15.196:0/3606728484 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f78680594b0 con 0x7f78780b5dd0 2024-04-28T13:33:38.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.386+0000 7f787dc00640 1 -- 172.21.15.196:0/3606728484 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78780d0930 con 0x7f785406c980 2024-04-28T13:33:38.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.407+0000 7f786ea00640 1 -- 172.21.15.196:0/3606728484 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251913 (secure 0 0 0) 0x7f78780d0930 con 0x7f785406c980 2024-04-28T13:33:38.411 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:38.413 INFO:teuthology.orchestra.run.smithi196.stderr:dumped all 2024-04-28T13:33:38.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.414+0000 7f787dc00640 1 -- 172.21.15.196:0/3606728484 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f785406c980 msgr2=0x7f785406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:38.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.414+0000 7f787dc00640 1 --2- 172.21.15.196:0/3606728484 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f785406c980 0x7f785406ee40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7860034000 tx=0x7f786002c000 comp rx=0 tx=0).stop 2024-04-28T13:33:38.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.414+0000 7f787dc00640 1 -- 172.21.15.196:0/3606728484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78780b5dd0 msgr2=0x7f78781f6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:38.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.414+0000 7f787dc00640 1 --2- 172.21.15.196:0/3606728484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78780b5dd0 0x7f78781f6970 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f786800c390 tx=0x7f786800c850 comp rx=0 tx=0).stop 2024-04-28T13:33:38.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.414+0000 7f787dc00640 1 -- 172.21.15.196:0/3606728484 shutdown_connections 2024-04-28T13:33:38.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.414+0000 7f787dc00640 1 --2- 172.21.15.196:0/3606728484 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f785406c980 0x7f785406ee40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.414+0000 7f787dc00640 1 --2- 172.21.15.196:0/3606728484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78781469f0 0x7f78780b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.414+0000 7f787dc00640 1 --2- 172.21.15.196:0/3606728484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78780d0d00 0x7f78781f8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.414+0000 7f787dc00640 1 --2- 172.21.15.196:0/3606728484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78780b5dd0 0x7f78781f6970 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.414+0000 7f787dc00640 1 -- 172.21.15.196:0/3606728484 >> 172.21.15.196:0/3606728484 conn(0x7f78780cd830 msgr2=0x7f7878073660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:38.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.414+0000 7f787dc00640 1 -- 172.21.15.196:0/3606728484 shutdown_connections 2024-04-28T13:33:38.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.414+0000 7f787dc00640 1 -- 172.21.15.196:0/3606728484 wait complete. 2024-04-28T13:33:38.432 INFO:teuthology.orchestra.run.smithi196.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-04-28T13:33:38.230620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225636,"kb_used_data":5280,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749084,"statfs":{"total":773094113280,"available":772863062016,"internally_reserved":0,"allocated":5406720,"data_stored":2878244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002207"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.001905+0000","last_change":"2024-04-28T13:33:33.999818+0000","last_active":"2024-04-28T13:33:34.001905+0000","last_peered":"2024-04-28T13:33:34.001905+0000","last_clean":"2024-04-28T13:33:34.001905+0000","last_became_active":"2024-04-28T13:33:33.999486+0000","last_became_peered":"2024-04-28T13:33:33.999486+0000","last_unstale":"2024-04-28T13:33:34.001905+0000","last_undegraded":"2024-04-28T13:33:34.001905+0000","last_fullsized":"2024-04-28T13:33:34.001905+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001490+0000","last_change":"2024-04-28T13:33:34.003919+0000","last_active":"2024-04-28T13:33:35.001490+0000","last_peered":"2024-04-28T13:33:35.001490+0000","last_clean":"2024-04-28T13:33:35.001490+0000","last_became_active":"2024-04-28T13:33:34.003551+0000","last_became_peered":"2024-04-28T13:33:34.003551+0000","last_unstale":"2024-04-28T13:33:35.001490+0000","last_undegraded":"2024-04-28T13:33:35.001490+0000","last_fullsized":"2024-04-28T13:33:35.001490+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999420+0000","last_change":"2024-04-28T13:33:33.996733+0000","last_active":"2024-04-28T13:33:34.999420+0000","last_peered":"2024-04-28T13:33:34.999420+0000","last_clean":"2024-04-28T13:33:34.999420+0000","last_became_active":"2024-04-28T13:33:33.996413+0000","last_became_peered":"2024-04-28T13:33:33.996413+0000","last_unstale":"2024-04-28T13:33:34.999420+0000","last_undegraded":"2024-04-28T13:33:34.999420+0000","last_fullsized":"2024-04-28T13:33:34.999420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:46.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.007952+0000","last_change":"2024-04-28T13:33:34.004787+0000","last_active":"2024-04-28T13:33:34.007952+0000","last_peered":"2024-04-28T13:33:34.007952+0000","last_clean":"2024-04-28T13:33:34.007952+0000","last_became_active":"2024-04-28T13:33:33.999818+0000","last_became_peered":"2024-04-28T13:33:33.999818+0000","last_unstale":"2024-04-28T13:33:34.007952+0000","last_undegraded":"2024-04-28T13:33:34.007952+0000","last_fullsized":"2024-04-28T13:33:34.007952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:52.292838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001180+0000","last_change":"2024-04-28T13:33:34.004715+0000","last_active":"2024-04-28T13:33:35.001180+0000","last_peered":"2024-04-28T13:33:35.001180+0000","last_clean":"2024-04-28T13:33:35.001180+0000","last_became_active":"2024-04-28T13:33:34.004436+0000","last_became_peered":"2024-04-28T13:33:34.004436+0000","last_unstale":"2024-04-28T13:33:35.001180+0000","last_undegraded":"2024-04-28T13:33:35.001180+0000","last_fullsized":"2024-04-28T13:33:35.001180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:46.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.018650+0000","last_change":"2024-04-28T13:33:34.012959+0000","last_active":"2024-04-28T13:33:34.018650+0000","last_peered":"2024-04-28T13:33:34.018650+0000","last_clean":"2024-04-28T13:33:34.018650+0000","last_became_active":"2024-04-28T13:33:34.010839+0000","last_became_peered":"2024-04-28T13:33:34.010839+0000","last_unstale":"2024-04-28T13:33:34.018650+0000","last_undegraded":"2024-04-28T13:33:34.018650+0000","last_fullsized":"2024-04-28T13:33:34.018650+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:32:47.409486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.010263+0000","last_change":"2024-04-28T13:33:34.003448+0000","last_active":"2024-04-28T13:33:34.010263+0000","last_peered":"2024-04-28T13:33:34.010263+0000","last_clean":"2024-04-28T13:33:34.010263+0000","last_became_active":"2024-04-28T13:33:34.002948+0000","last_became_peered":"2024-04-28T13:33:34.002948+0000","last_unstale":"2024-04-28T13:33:34.010263+0000","last_undegraded":"2024-04-28T13:33:34.010263+0000","last_fullsized":"2024-04-28T13:33:34.010263+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014060+0000","last_change":"2024-04-28T13:33:34.007814+0000","last_active":"2024-04-28T13:33:34.014060+0000","last_peered":"2024-04-28T13:33:34.014060+0000","last_clean":"2024-04-28T13:33:34.014060+0000","last_became_active":"2024-04-28T13:33:34.007485+0000","last_became_peered":"2024-04-28T13:33:34.007485+0000","last_unstale":"2024-04-28T13:33:34.014060+0000","last_undegraded":"2024-04-28T13:33:34.014060+0000","last_fullsized":"2024-04-28T13:33:34.014060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:37:53.261649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.016586+0000","last_change":"2024-04-28T13:33:34.013366+0000","last_active":"2024-04-28T13:33:34.016586+0000","last_peered":"2024-04-28T13:33:34.016586+0000","last_clean":"2024-04-28T13:33:34.016586+0000","last_became_active":"2024-04-28T13:33:34.012900+0000","last_became_peered":"2024-04-28T13:33:34.012900+0000","last_unstale":"2024-04-28T13:33:34.016586+0000","last_undegraded":"2024-04-28T13:33:34.016586+0000","last_fullsized":"2024-04-28T13:33:34.016586+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:44.577439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014683+0000","last_change":"2024-04-28T13:33:34.007098+0000","last_active":"2024-04-28T13:33:34.014683+0000","last_peered":"2024-04-28T13:33:34.014683+0000","last_clean":"2024-04-28T13:33:34.014683+0000","last_became_active":"2024-04-28T13:33:34.005157+0000","last_became_peered":"2024-04-28T13:33:34.005157+0000","last_unstale":"2024-04-28T13:33:34.014683+0000","last_undegraded":"2024-04-28T13:33:34.014683+0000","last_fullsized":"2024-04-28T13:33:34.014683+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:18:56.809253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014801+0000","last_change":"2024-04-28T13:33:34.011118+0000","last_active":"2024-04-28T13:33:34.014801+0000","last_peered":"2024-04-28T13:33:34.014801+0000","last_clean":"2024-04-28T13:33:34.014801+0000","last_became_active":"2024-04-28T13:33:34.010811+0000","last_became_peered":"2024-04-28T13:33:34.010811+0000","last_unstale":"2024-04-28T13:33:34.014801+0000","last_undegraded":"2024-04-28T13:33:34.014801+0000","last_fullsized":"2024-04-28T13:33:34.014801+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:14.152642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001244+0000","last_change":"2024-04-28T13:33:34.011900+0000","last_active":"2024-04-28T13:33:35.001244+0000","last_peered":"2024-04-28T13:33:35.001244+0000","last_clean":"2024-04-28T13:33:35.001244+0000","last_became_active":"2024-04-28T13:33:34.011580+0000","last_became_peered":"2024-04-28T13:33:34.011580+0000","last_unstale":"2024-04-28T13:33:35.001244+0000","last_undegraded":"2024-04-28T13:33:35.001244+0000","last_fullsized":"2024-04-28T13:33:35.001244+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999662+0000","last_change":"2024-04-28T13:33:34.000874+0000","last_active":"2024-04-28T13:33:34.999662+0000","last_peered":"2024-04-28T13:33:34.999662+0000","last_clean":"2024-04-28T13:33:34.999662+0000","last_became_active":"2024-04-28T13:33:34.000568+0000","last_became_peered":"2024-04-28T13:33:34.000568+0000","last_unstale":"2024-04-28T13:33:34.999662+0000","last_undegraded":"2024-04-28T13:33:34.999662+0000","last_fullsized":"2024-04-28T13:33:34.999662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:56:40.722825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.015183+0000","last_change":"2024-04-28T13:33:34.012076+0000","last_active":"2024-04-28T13:33:34.015183+0000","last_peered":"2024-04-28T13:33:34.015183+0000","last_clean":"2024-04-28T13:33:34.015183+0000","last_became_active":"2024-04-28T13:33:34.011795+0000","last_became_peered":"2024-04-28T13:33:34.011795+0000","last_unstale":"2024-04-28T13:33:34.015183+0000","last_undegraded":"2024-04-28T13:33:34.015183+0000","last_fullsized":"2024-04-28T13:33:34.015183+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:29.495657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.004135+0000","last_change":"2024-04-28T13:33:33.998387+0000","last_active":"2024-04-28T13:33:34.004135+0000","last_peered":"2024-04-28T13:33:34.004135+0000","last_clean":"2024-04-28T13:33:34.004135+0000","last_became_active":"2024-04-28T13:33:33.998087+0000","last_became_peered":"2024-04-28T13:33:33.998087+0000","last_unstale":"2024-04-28T13:33:34.004135+0000","last_undegraded":"2024-04-28T13:33:34.004135+0000","last_fullsized":"2024-04-28T13:33:34.004135+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.998844+0000","last_change":"2024-04-28T13:33:33.989154+0000","last_active":"2024-04-28T13:33:33.998844+0000","last_peered":"2024-04-28T13:33:33.998844+0000","last_clean":"2024-04-28T13:33:33.998844+0000","last_became_active":"2024-04-28T13:33:33.988827+0000","last_became_peered":"2024-04-28T13:33:33.988827+0000","last_unstale":"2024-04-28T13:33:33.998844+0000","last_undegraded":"2024-04-28T13:33:33.998844+0000","last_fullsized":"2024-04-28T13:33:33.998844+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:03:40.797677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999849+0000","last_change":"2024-04-28T13:33:34.008467+0000","last_active":"2024-04-28T13:33:34.999849+0000","last_peered":"2024-04-28T13:33:34.999849+0000","last_clean":"2024-04-28T13:33:34.999849+0000","last_became_active":"2024-04-28T13:33:34.008181+0000","last_became_peered":"2024-04-28T13:33:34.008181+0000","last_unstale":"2024-04-28T13:33:34.999849+0000","last_undegraded":"2024-04-28T13:33:34.999849+0000","last_fullsized":"2024-04-28T13:33:34.999849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999757+0000","last_change":"2024-04-28T13:33:34.007505+0000","last_active":"2024-04-28T13:33:34.999757+0000","last_peered":"2024-04-28T13:33:34.999757+0000","last_clean":"2024-04-28T13:33:34.999757+0000","last_became_active":"2024-04-28T13:33:34.006793+0000","last_became_peered":"2024-04-28T13:33:34.006793+0000","last_unstale":"2024-04-28T13:33:34.999757+0000","last_undegraded":"2024-04-28T13:33:34.999757+0000","last_fullsized":"2024-04-28T13:33:34.999757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.011743+0000","last_change":"2024-04-28T13:33:34.009974+0000","last_active":"2024-04-28T13:33:34.011743+0000","last_peered":"2024-04-28T13:33:34.011743+0000","last_clean":"2024-04-28T13:33:34.011743+0000","last_became_active":"2024-04-28T13:33:34.009657+0000","last_became_peered":"2024-04-28T13:33:34.009657+0000","last_unstale":"2024-04-28T13:33:34.011743+0000","last_undegraded":"2024-04-28T13:33:34.011743+0000","last_fullsized":"2024-04-28T13:33:34.011743+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:37:19.643053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999758+0000","last_change":"2024-04-28T13:33:34.004233+0000","last_active":"2024-04-28T13:33:34.999758+0000","last_peered":"2024-04-28T13:33:34.999758+0000","last_clean":"2024-04-28T13:33:34.999758+0000","last_became_active":"2024-04-28T13:33:33.999567+0000","last_became_peered":"2024-04-28T13:33:33.999567+0000","last_unstale":"2024-04-28T13:33:34.999758+0000","last_undegraded":"2024-04-28T13:33:34.999758+0000","last_fullsized":"2024-04-28T13:33:34.999758+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:36.248393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.016459+0000","last_change":"2024-04-28T13:33:34.008932+0000","last_active":"2024-04-28T13:33:34.016459+0000","last_peered":"2024-04-28T13:33:34.016459+0000","last_clean":"2024-04-28T13:33:34.016459+0000","last_became_active":"2024-04-28T13:33:34.008652+0000","last_became_peered":"2024-04-28T13:33:34.008652+0000","last_unstale":"2024-04-28T13:33:34.016459+0000","last_undegraded":"2024-04-28T13:33:34.016459+0000","last_fullsized":"2024-04-28T13:33:34.016459+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:54.852573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014147+0000","last_change":"2024-04-28T13:33:34.006521+0000","last_active":"2024-04-28T13:33:34.014147+0000","last_peered":"2024-04-28T13:33:34.014147+0000","last_clean":"2024-04-28T13:33:34.014147+0000","last_became_active":"2024-04-28T13:33:34.006227+0000","last_became_peered":"2024-04-28T13:33:34.006227+0000","last_unstale":"2024-04-28T13:33:34.014147+0000","last_undegraded":"2024-04-28T13:33:34.014147+0000","last_fullsized":"2024-04-28T13:33:34.014147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:31.671210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.001556+0000","last_change":"2024-04-28T13:33:33.999169+0000","last_active":"2024-04-28T13:33:34.001556+0000","last_peered":"2024-04-28T13:33:34.001556+0000","last_clean":"2024-04-28T13:33:34.001556+0000","last_became_active":"2024-04-28T13:33:33.998179+0000","last_became_peered":"2024-04-28T13:33:33.998179+0000","last_unstale":"2024-04-28T13:33:34.001556+0000","last_undegraded":"2024-04-28T13:33:34.001556+0000","last_fullsized":"2024-04-28T13:33:34.001556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:52:06.849498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.001462+0000","last_change":"2024-04-28T13:33:33.998034+0000","last_active":"2024-04-28T13:33:34.001462+0000","last_peered":"2024-04-28T13:33:34.001462+0000","last_clean":"2024-04-28T13:33:34.001462+0000","last_became_active":"2024-04-28T13:33:33.997648+0000","last_became_peered":"2024-04-28T13:33:33.997648+0000","last_unstale":"2024-04-28T13:33:34.001462+0000","last_undegraded":"2024-04-28T13:33:34.001462+0000","last_fullsized":"2024-04-28T13:33:34.001462+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:12:14.181038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.009634+0000","last_change":"2024-04-28T13:33:34.007275+0000","last_active":"2024-04-28T13:33:34.009634+0000","last_peered":"2024-04-28T13:33:34.009634+0000","last_clean":"2024-04-28T13:33:34.009634+0000","last_became_active":"2024-04-28T13:33:34.005017+0000","last_became_peered":"2024-04-28T13:33:34.005017+0000","last_unstale":"2024-04-28T13:33:34.009634+0000","last_undegraded":"2024-04-28T13:33:34.009634+0000","last_fullsized":"2024-04-28T13:33:34.009634+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:36.248393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999682+0000","last_change":"2024-04-28T13:33:33.995804+0000","last_active":"2024-04-28T13:33:34.999682+0000","last_peered":"2024-04-28T13:33:34.999682+0000","last_clean":"2024-04-28T13:33:34.999682+0000","last_became_active":"2024-04-28T13:33:33.995502+0000","last_became_peered":"2024-04-28T13:33:33.995502+0000","last_unstale":"2024-04-28T13:33:34.999682+0000","last_undegraded":"2024-04-28T13:33:34.999682+0000","last_fullsized":"2024-04-28T13:33:34.999682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:27:53.804713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999622+0000","last_change":"2024-04-28T13:33:34.006447+0000","last_active":"2024-04-28T13:33:34.999622+0000","last_peered":"2024-04-28T13:33:34.999622+0000","last_clean":"2024-04-28T13:33:34.999622+0000","last_became_active":"2024-04-28T13:33:34.005656+0000","last_became_peered":"2024-04-28T13:33:34.005656+0000","last_unstale":"2024-04-28T13:33:34.999622+0000","last_undegraded":"2024-04-28T13:33:34.999622+0000","last_fullsized":"2024-04-28T13:33:34.999622+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001410+0000","last_change":"2024-04-28T13:33:34.005608+0000","last_active":"2024-04-28T13:33:35.001410+0000","last_peered":"2024-04-28T13:33:35.001410+0000","last_clean":"2024-04-28T13:33:35.001410+0000","last_became_active":"2024-04-28T13:33:34.005316+0000","last_became_peered":"2024-04-28T13:33:34.005316+0000","last_unstale":"2024-04-28T13:33:35.001410+0000","last_undegraded":"2024-04-28T13:33:35.001410+0000","last_fullsized":"2024-04-28T13:33:35.001410+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.004506+0000","last_change":"2024-04-28T13:33:34.002648+0000","last_active":"2024-04-28T13:33:34.004506+0000","last_peered":"2024-04-28T13:33:34.004506+0000","last_clean":"2024-04-28T13:33:34.004506+0000","last_became_active":"2024-04-28T13:33:34.002375+0000","last_became_peered":"2024-04-28T13:33:34.002375+0000","last_unstale":"2024-04-28T13:33:34.004506+0000","last_undegraded":"2024-04-28T13:33:34.004506+0000","last_fullsized":"2024-04-28T13:33:34.004506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:56:40.722825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999556+0000","last_change":"2024-04-28T13:33:33.999443+0000","last_active":"2024-04-28T13:33:34.999556+0000","last_peered":"2024-04-28T13:33:34.999556+0000","last_clean":"2024-04-28T13:33:34.999556+0000","last_became_active":"2024-04-28T13:33:33.998905+0000","last_became_peered":"2024-04-28T13:33:33.998905+0000","last_unstale":"2024-04-28T13:33:34.999556+0000","last_undegraded":"2024-04-28T13:33:34.999556+0000","last_fullsized":"2024-04-28T13:33:34.999556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.016079+0000","last_change":"2024-04-28T13:33:34.008881+0000","last_active":"2024-04-28T13:33:34.016079+0000","last_peered":"2024-04-28T13:33:34.016079+0000","last_clean":"2024-04-28T13:33:34.016079+0000","last_became_active":"2024-04-28T13:33:34.008297+0000","last_became_peered":"2024-04-28T13:33:34.008297+0000","last_unstale":"2024-04-28T13:33:34.016079+0000","last_undegraded":"2024-04-28T13:33:34.016079+0000","last_fullsized":"2024-04-28T13:33:34.016079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:20:07.691466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999181+0000","last_change":"2024-04-28T13:33:34.003176+0000","last_active":"2024-04-28T13:33:34.999181+0000","last_peered":"2024-04-28T13:33:34.999181+0000","last_clean":"2024-04-28T13:33:34.999181+0000","last_became_active":"2024-04-28T13:33:34.002915+0000","last_became_peered":"2024-04-28T13:33:34.002915+0000","last_unstale":"2024-04-28T13:33:34.999181+0000","last_undegraded":"2024-04-28T13:33:34.999181+0000","last_fullsized":"2024-04-28T13:33:34.999181+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:52:06.849498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014933+0000","last_change":"2024-04-28T13:33:34.005669+0000","last_active":"2024-04-28T13:33:34.014933+0000","last_peered":"2024-04-28T13:33:34.014933+0000","last_clean":"2024-04-28T13:33:34.014933+0000","last_became_active":"2024-04-28T13:33:34.005365+0000","last_became_peered":"2024-04-28T13:33:34.005365+0000","last_unstale":"2024-04-28T13:33:34.014933+0000","last_undegraded":"2024-04-28T13:33:34.014933+0000","last_fullsized":"2024-04-28T13:33:34.014933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:22:35.458698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.017552+0000","last_change":"2024-04-28T13:33:32.998037+0000","last_active":"2024-04-28T13:33:34.017552+0000","last_peered":"2024-04-28T13:33:34.017552+0000","last_clean":"2024-04-28T13:33:34.017552+0000","last_became_active":"2024-04-28T13:33:32.997694+0000","last_became_peered":"2024-04-28T13:33:32.997694+0000","last_unstale":"2024-04-28T13:33:34.017552+0000","last_undegraded":"2024-04-28T13:33:34.017552+0000","last_fullsized":"2024-04-28T13:33:34.017552+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:02.432193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.998769+0000","last_change":"2024-04-28T13:33:33.989048+0000","last_active":"2024-04-28T13:33:33.998769+0000","last_peered":"2024-04-28T13:33:33.998769+0000","last_clean":"2024-04-28T13:33:33.998769+0000","last_became_active":"2024-04-28T13:33:33.988758+0000","last_became_peered":"2024-04-28T13:33:33.988758+0000","last_unstale":"2024-04-28T13:33:33.998769+0000","last_undegraded":"2024-04-28T13:33:33.998769+0000","last_fullsized":"2024-04-28T13:33:33.998769+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:24:18.185935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999883+0000","last_change":"2024-04-28T13:33:32.999239+0000","last_active":"2024-04-28T13:33:34.999883+0000","last_peered":"2024-04-28T13:33:34.999883+0000","last_clean":"2024-04-28T13:33:34.999883+0000","last_became_active":"2024-04-28T13:33:32.998813+0000","last_became_peered":"2024-04-28T13:33:32.998813+0000","last_unstale":"2024-04-28T13:33:34.999883+0000","last_undegraded":"2024-04-28T13:33:34.999883+0000","last_fullsized":"2024-04-28T13:33:34.999883+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:22:34.444155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001579+0000","last_change":"2024-04-28T13:33:34.003176+0000","last_active":"2024-04-28T13:33:35.001579+0000","last_peered":"2024-04-28T13:33:35.001579+0000","last_clean":"2024-04-28T13:33:35.001579+0000","last_became_active":"2024-04-28T13:33:34.002904+0000","last_became_peered":"2024-04-28T13:33:34.002904+0000","last_unstale":"2024-04-28T13:33:35.001579+0000","last_undegraded":"2024-04-28T13:33:35.001579+0000","last_fullsized":"2024-04-28T13:33:35.001579+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.018798+0000","last_change":"2024-04-28T13:33:32.997729+0000","last_active":"2024-04-28T13:33:34.018798+0000","last_peered":"2024-04-28T13:33:34.018798+0000","last_clean":"2024-04-28T13:33:34.018798+0000","last_became_active":"2024-04-28T13:33:32.996881+0000","last_became_peered":"2024-04-28T13:33:32.996881+0000","last_unstale":"2024-04-28T13:33:34.018798+0000","last_undegraded":"2024-04-28T13:33:34.018798+0000","last_fullsized":"2024-04-28T13:33:34.018798+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.016605+0000","last_change":"2024-04-28T13:33:34.004339+0000","last_active":"2024-04-28T13:33:34.016605+0000","last_peered":"2024-04-28T13:33:34.016605+0000","last_clean":"2024-04-28T13:33:34.016605+0000","last_became_active":"2024-04-28T13:33:34.002361+0000","last_became_peered":"2024-04-28T13:33:34.002361+0000","last_unstale":"2024-04-28T13:33:34.016605+0000","last_undegraded":"2024-04-28T13:33:34.016605+0000","last_fullsized":"2024-04-28T13:33:34.016605+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:52:46.527475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.010939+0000","last_change":"2024-04-28T13:33:33.000754+0000","last_active":"2024-04-28T13:33:34.010939+0000","last_peered":"2024-04-28T13:33:34.010939+0000","last_clean":"2024-04-28T13:33:34.010939+0000","last_became_active":"2024-04-28T13:33:33.000387+0000","last_became_peered":"2024-04-28T13:33:33.000387+0000","last_unstale":"2024-04-28T13:33:34.010939+0000","last_undegraded":"2024-04-28T13:33:34.010939+0000","last_fullsized":"2024-04-28T13:33:34.010939+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:28.481114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999283+0000","last_change":"2024-04-28T13:33:33.994316+0000","last_active":"2024-04-28T13:33:34.999283+0000","last_peered":"2024-04-28T13:33:34.999283+0000","last_clean":"2024-04-28T13:33:34.999283+0000","last_became_active":"2024-04-28T13:33:33.994002+0000","last_became_peered":"2024-04-28T13:33:33.994002+0000","last_unstale":"2024-04-28T13:33:34.999283+0000","last_undegraded":"2024-04-28T13:33:34.999283+0000","last_fullsized":"2024-04-28T13:33:34.999283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:03.446736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.016168+0000","last_change":"2024-04-28T13:33:32.999540+0000","last_active":"2024-04-28T13:33:34.016168+0000","last_peered":"2024-04-28T13:33:34.016168+0000","last_clean":"2024-04-28T13:33:34.016168+0000","last_became_active":"2024-04-28T13:33:32.999147+0000","last_became_peered":"2024-04-28T13:33:32.999147+0000","last_unstale":"2024-04-28T13:33:34.016168+0000","last_undegraded":"2024-04-28T13:33:34.016168+0000","last_fullsized":"2024-04-28T13:33:34.016168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:45.657949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.005003+0000","last_change":"2024-04-28T13:33:34.000424+0000","last_active":"2024-04-28T13:33:34.005003+0000","last_peered":"2024-04-28T13:33:34.005003+0000","last_clean":"2024-04-28T13:33:34.005003+0000","last_became_active":"2024-04-28T13:33:33.999414+0000","last_became_peered":"2024-04-28T13:33:33.999414+0000","last_unstale":"2024-04-28T13:33:34.005003+0000","last_undegraded":"2024-04-28T13:33:34.005003+0000","last_fullsized":"2024-04-28T13:33:34.005003+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:22:35.458698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999411+0000","last_change":"2024-04-28T13:33:32.988943+0000","last_active":"2024-04-28T13:33:34.999411+0000","last_peered":"2024-04-28T13:33:34.999411+0000","last_clean":"2024-04-28T13:33:34.999411+0000","last_became_active":"2024-04-28T13:33:32.986596+0000","last_became_peered":"2024-04-28T13:33:32.986596+0000","last_unstale":"2024-04-28T13:33:34.999411+0000","last_undegraded":"2024-04-28T13:33:34.999411+0000","last_fullsized":"2024-04-28T13:33:34.999411+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.007696+0000","last_change":"2024-04-28T13:33:34.002579+0000","last_active":"2024-04-28T13:33:34.007696+0000","last_peered":"2024-04-28T13:33:34.007696+0000","last_clean":"2024-04-28T13:33:34.007696+0000","last_became_active":"2024-04-28T13:33:34.002254+0000","last_became_peered":"2024-04-28T13:33:34.002254+0000","last_unstale":"2024-04-28T13:33:34.007696+0000","last_undegraded":"2024-04-28T13:33:34.007696+0000","last_fullsized":"2024-04-28T13:33:34.007696+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:30.570296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001072+0000","last_change":"2024-04-28T13:33:32.996052+0000","last_active":"2024-04-28T13:33:35.001072+0000","last_peered":"2024-04-28T13:33:35.001072+0000","last_clean":"2024-04-28T13:33:35.001072+0000","last_became_active":"2024-04-28T13:33:32.990258+0000","last_became_peered":"2024-04-28T13:33:32.990258+0000","last_unstale":"2024-04-28T13:33:35.001072+0000","last_undegraded":"2024-04-28T13:33:35.001072+0000","last_fullsized":"2024-04-28T13:33:35.001072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.001508+0000","last_change":"2024-04-28T13:33:33.998180+0000","last_active":"2024-04-28T13:33:34.001508+0000","last_peered":"2024-04-28T13:33:34.001508+0000","last_clean":"2024-04-28T13:33:34.001508+0000","last_became_active":"2024-04-28T13:33:33.997730+0000","last_became_peered":"2024-04-28T13:33:33.997730+0000","last_unstale":"2024-04-28T13:33:34.001508+0000","last_undegraded":"2024-04-28T13:33:34.001508+0000","last_fullsized":"2024-04-28T13:33:34.001508+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:55:17.195691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.983250+0000","last_change":"2024-04-28T13:33:32.989759+0000","last_active":"2024-04-28T13:33:33.983250+0000","last_peered":"2024-04-28T13:33:33.983250+0000","last_clean":"2024-04-28T13:33:33.983250+0000","last_became_active":"2024-04-28T13:33:32.988565+0000","last_became_peered":"2024-04-28T13:33:32.988565+0000","last_unstale":"2024-04-28T13:33:33.983250+0000","last_undegraded":"2024-04-28T13:33:33.983250+0000","last_fullsized":"2024-04-28T13:33:33.983250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:27:52.790170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.000164+0000","last_change":"2024-04-28T13:33:34.007736+0000","last_active":"2024-04-28T13:33:35.000164+0000","last_peered":"2024-04-28T13:33:35.000164+0000","last_clean":"2024-04-28T13:33:35.000164+0000","last_became_active":"2024-04-28T13:33:34.007170+0000","last_became_peered":"2024-04-28T13:33:34.007170+0000","last_unstale":"2024-04-28T13:33:35.000164+0000","last_undegraded":"2024-04-28T13:33:35.000164+0000","last_fullsized":"2024-04-28T13:33:35.000164+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:01:33.789936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.000108+0000","last_change":"2024-04-28T13:33:32.992483+0000","last_active":"2024-04-28T13:33:35.000108+0000","last_peered":"2024-04-28T13:33:35.000108+0000","last_clean":"2024-04-28T13:33:35.000108+0000","last_became_active":"2024-04-28T13:33:32.992216+0000","last_became_peered":"2024-04-28T13:33:32.992216+0000","last_unstale":"2024-04-28T13:33:35.000108+0000","last_undegraded":"2024-04-28T13:33:35.000108+0000","last_fullsized":"2024-04-28T13:33:35.000108+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:11.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.011091+0000","last_change":"2024-04-28T13:33:34.009420+0000","last_active":"2024-04-28T13:33:34.011091+0000","last_peered":"2024-04-28T13:33:34.011091+0000","last_clean":"2024-04-28T13:33:34.011091+0000","last_became_active":"2024-04-28T13:33:34.009020+0000","last_became_peered":"2024-04-28T13:33:34.009020+0000","last_unstale":"2024-04-28T13:33:34.011091+0000","last_undegraded":"2024-04-28T13:33:34.011091+0000","last_fullsized":"2024-04-28T13:33:34.011091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.991627+0000","last_change":"2024-04-28T13:33:32.997498+0000","last_active":"2024-04-28T13:33:33.991627+0000","last_peered":"2024-04-28T13:33:33.991627+0000","last_clean":"2024-04-28T13:33:33.991627+0000","last_became_active":"2024-04-28T13:33:32.997237+0000","last_became_peered":"2024-04-28T13:33:32.997237+0000","last_unstale":"2024-04-28T13:33:33.991627+0000","last_undegraded":"2024-04-28T13:33:33.991627+0000","last_fullsized":"2024-04-28T13:33:33.991627+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:37:18.628510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.008405+0000","last_change":"2024-04-28T13:33:34.002865+0000","last_active":"2024-04-28T13:33:34.008405+0000","last_peered":"2024-04-28T13:33:34.008405+0000","last_clean":"2024-04-28T13:33:34.008405+0000","last_became_active":"2024-04-28T13:33:34.002421+0000","last_became_peered":"2024-04-28T13:33:34.002421+0000","last_unstale":"2024-04-28T13:33:34.008405+0000","last_undegraded":"2024-04-28T13:33:34.008405+0000","last_fullsized":"2024-04-28T13:33:34.008405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.007950+0000","last_change":"2024-04-28T13:33:33.004559+0000","last_active":"2024-04-28T13:33:34.007950+0000","last_peered":"2024-04-28T13:33:34.007950+0000","last_clean":"2024-04-28T13:33:34.007950+0000","last_became_active":"2024-04-28T13:33:33.004153+0000","last_became_peered":"2024-04-28T13:33:33.004153+0000","last_unstale":"2024-04-28T13:33:34.007950+0000","last_undegraded":"2024-04-28T13:33:34.007950+0000","last_fullsized":"2024-04-28T13:33:34.007950+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:02.432193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.012182+0000","last_change":"2024-04-28T13:33:34.006854+0000","last_active":"2024-04-28T13:33:34.012182+0000","last_peered":"2024-04-28T13:33:34.012182+0000","last_clean":"2024-04-28T13:33:34.012182+0000","last_became_active":"2024-04-28T13:33:34.006561+0000","last_became_peered":"2024-04-28T13:33:34.006561+0000","last_unstale":"2024-04-28T13:33:34.012182+0000","last_undegraded":"2024-04-28T13:33:34.012182+0000","last_fullsized":"2024-04-28T13:33:34.012182+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999932+0000","last_change":"2024-04-28T13:33:32.991367+0000","last_active":"2024-04-28T13:33:34.999932+0000","last_peered":"2024-04-28T13:33:34.999932+0000","last_clean":"2024-04-28T13:33:34.999932+0000","last_became_active":"2024-04-28T13:33:32.991060+0000","last_became_peered":"2024-04-28T13:33:32.991060+0000","last_unstale":"2024-04-28T13:33:34.999932+0000","last_undegraded":"2024-04-28T13:33:34.999932+0000","last_fullsized":"2024-04-28T13:33:34.999932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:00:16.144423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.001266+0000","last_change":"2024-04-28T13:33:33.996658+0000","last_active":"2024-04-28T13:33:34.001266+0000","last_peered":"2024-04-28T13:33:34.001266+0000","last_clean":"2024-04-28T13:33:34.001266+0000","last_became_active":"2024-04-28T13:33:33.996260+0000","last_became_peered":"2024-04-28T13:33:33.996260+0000","last_unstale":"2024-04-28T13:33:34.001266+0000","last_undegraded":"2024-04-28T13:33:34.001266+0000","last_fullsized":"2024-04-28T13:33:34.001266+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:54:02.364417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.021953+0000","last_change":"2024-04-28T13:33:32.990968+0000","last_active":"2024-04-28T13:33:34.021953+0000","last_peered":"2024-04-28T13:33:34.021953+0000","last_clean":"2024-04-28T13:33:34.021953+0000","last_became_active":"2024-04-28T13:33:32.990718+0000","last_became_peered":"2024-04-28T13:33:32.990718+0000","last_unstale":"2024-04-28T13:33:34.021953+0000","last_undegraded":"2024-04-28T13:33:34.021953+0000","last_fullsized":"2024-04-28T13:33:34.021953+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:03.358943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.006429+0000","last_change":"2024-04-28T13:33:34.003961+0000","last_active":"2024-04-28T13:33:34.006429+0000","last_peered":"2024-04-28T13:33:34.006429+0000","last_clean":"2024-04-28T13:33:34.006429+0000","last_became_active":"2024-04-28T13:33:34.003599+0000","last_became_peered":"2024-04-28T13:33:34.003599+0000","last_unstale":"2024-04-28T13:33:34.006429+0000","last_undegraded":"2024-04-28T13:33:34.006429+0000","last_fullsized":"2024-04-28T13:33:34.006429+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:12.546153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.989344+0000","last_change":"2024-04-28T13:33:33.000521+0000","last_active":"2024-04-28T13:33:33.989344+0000","last_peered":"2024-04-28T13:33:33.989344+0000","last_clean":"2024-04-28T13:33:33.989344+0000","last_became_active":"2024-04-28T13:33:32.998867+0000","last_became_peered":"2024-04-28T13:33:32.998867+0000","last_unstale":"2024-04-28T13:33:33.989344+0000","last_undegraded":"2024-04-28T13:33:33.989344+0000","last_fullsized":"2024-04-28T13:33:33.989344+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:43.562896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.015834+0000","last_change":"2024-04-28T13:33:34.006368+0000","last_active":"2024-04-28T13:33:34.015834+0000","last_peered":"2024-04-28T13:33:34.015834+0000","last_clean":"2024-04-28T13:33:34.015834+0000","last_became_active":"2024-04-28T13:33:34.002579+0000","last_became_peered":"2024-04-28T13:33:34.002579+0000","last_unstale":"2024-04-28T13:33:34.015834+0000","last_undegraded":"2024-04-28T13:33:34.015834+0000","last_fullsized":"2024-04-28T13:33:34.015834+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:01:13.143511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.986501+0000","last_change":"2024-04-28T13:33:32.991648+0000","last_active":"2024-04-28T13:33:33.986501+0000","last_peered":"2024-04-28T13:33:33.986501+0000","last_clean":"2024-04-28T13:33:33.986501+0000","last_became_active":"2024-04-28T13:33:32.991369+0000","last_became_peered":"2024-04-28T13:33:32.991369+0000","last_unstale":"2024-04-28T13:33:33.986501+0000","last_undegraded":"2024-04-28T13:33:33.986501+0000","last_fullsized":"2024-04-28T13:33:33.986501+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:43.562896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.013499+0000","last_change":"2024-04-28T13:33:34.010407+0000","last_active":"2024-04-28T13:33:34.013499+0000","last_peered":"2024-04-28T13:33:34.013499+0000","last_clean":"2024-04-28T13:33:34.013499+0000","last_became_active":"2024-04-28T13:33:34.010077+0000","last_became_peered":"2024-04-28T13:33:34.010077+0000","last_unstale":"2024-04-28T13:33:34.013499+0000","last_undegraded":"2024-04-28T13:33:34.013499+0000","last_fullsized":"2024-04-28T13:33:34.013499+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:01:33.789936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.018534+0000","last_change":"2024-04-28T13:33:32.989587+0000","last_active":"2024-04-28T13:33:34.018534+0000","last_peered":"2024-04-28T13:33:34.018534+0000","last_clean":"2024-04-28T13:33:34.018534+0000","last_became_active":"2024-04-28T13:33:32.988579+0000","last_became_peered":"2024-04-28T13:33:32.988579+0000","last_unstale":"2024-04-28T13:33:34.018534+0000","last_undegraded":"2024-04-28T13:33:34.018534+0000","last_fullsized":"2024-04-28T13:33:34.018534+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:35.233850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999875+0000","last_change":"2024-04-28T13:33:34.012322+0000","last_active":"2024-04-28T13:33:34.999875+0000","last_peered":"2024-04-28T13:33:34.999875+0000","last_clean":"2024-04-28T13:33:34.999875+0000","last_became_active":"2024-04-28T13:33:34.012112+0000","last_became_peered":"2024-04-28T13:33:34.012112+0000","last_unstale":"2024-04-28T13:33:34.999875+0000","last_undegraded":"2024-04-28T13:33:34.999875+0000","last_fullsized":"2024-04-28T13:33:34.999875+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:10.015862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.000018+0000","last_change":"2024-04-28T13:33:32.990958+0000","last_active":"2024-04-28T13:33:35.000018+0000","last_peered":"2024-04-28T13:33:35.000018+0000","last_clean":"2024-04-28T13:33:35.000018+0000","last_became_active":"2024-04-28T13:33:32.990659+0000","last_became_peered":"2024-04-28T13:33:32.990659+0000","last_unstale":"2024-04-28T13:33:35.000018+0000","last_undegraded":"2024-04-28T13:33:35.000018+0000","last_fullsized":"2024-04-28T13:33:35.000018+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:24:34.028457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014446+0000","last_change":"2024-04-28T13:33:34.005832+0000","last_active":"2024-04-28T13:33:34.014446+0000","last_peered":"2024-04-28T13:33:34.014446+0000","last_clean":"2024-04-28T13:33:34.014446+0000","last_became_active":"2024-04-28T13:33:34.005364+0000","last_became_peered":"2024-04-28T13:33:34.005364+0000","last_unstale":"2024-04-28T13:33:34.014446+0000","last_undegraded":"2024-04-28T13:33:34.014446+0000","last_fullsized":"2024-04-28T13:33:34.014446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:35.969568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.000007+0000","last_change":"2024-04-28T13:33:32.995338+0000","last_active":"2024-04-28T13:33:35.000007+0000","last_peered":"2024-04-28T13:33:35.000007+0000","last_clean":"2024-04-28T13:33:35.000007+0000","last_became_active":"2024-04-28T13:33:32.994798+0000","last_became_peered":"2024-04-28T13:33:32.994798+0000","last_unstale":"2024-04-28T13:33:35.000007+0000","last_undegraded":"2024-04-28T13:33:35.000007+0000","last_fullsized":"2024-04-28T13:33:35.000007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:29.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.003779+0000","last_change":"2024-04-28T13:33:34.002242+0000","last_active":"2024-04-28T13:33:34.003779+0000","last_peered":"2024-04-28T13:33:34.003779+0000","last_clean":"2024-04-28T13:33:34.003779+0000","last_became_active":"2024-04-28T13:33:34.001893+0000","last_became_peered":"2024-04-28T13:33:34.001893+0000","last_unstale":"2024-04-28T13:33:34.003779+0000","last_undegraded":"2024-04-28T13:33:34.003779+0000","last_fullsized":"2024-04-28T13:33:34.003779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:03:14.657009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001343+0000","last_change":"2024-04-28T13:33:32.991441+0000","last_active":"2024-04-28T13:33:35.001343+0000","last_peered":"2024-04-28T13:33:35.001343+0000","last_clean":"2024-04-28T13:33:35.001343+0000","last_became_active":"2024-04-28T13:33:32.990745+0000","last_became_peered":"2024-04-28T13:33:32.990745+0000","last_unstale":"2024-04-28T13:33:35.001343+0000","last_undegraded":"2024-04-28T13:33:35.001343+0000","last_fullsized":"2024-04-28T13:33:35.001343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:30.656667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.005445+0000","last_change":"2024-04-28T13:33:34.003094+0000","last_active":"2024-04-28T13:33:34.005445+0000","last_peered":"2024-04-28T13:33:34.005445+0000","last_clean":"2024-04-28T13:33:34.005445+0000","last_became_active":"2024-04-28T13:33:34.002228+0000","last_became_peered":"2024-04-28T13:33:34.002228+0000","last_unstale":"2024-04-28T13:33:34.005445+0000","last_undegraded":"2024-04-28T13:33:34.005445+0000","last_fullsized":"2024-04-28T13:33:34.005445+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:27:53.804713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.988463+0000","last_change":"2024-04-28T13:33:33.003720+0000","last_active":"2024-04-28T13:33:33.988463+0000","last_peered":"2024-04-28T13:33:33.988463+0000","last_clean":"2024-04-28T13:33:33.988463+0000","last_became_active":"2024-04-28T13:33:33.003389+0000","last_became_peered":"2024-04-28T13:33:33.003389+0000","last_unstale":"2024-04-28T13:33:33.988463+0000","last_undegraded":"2024-04-28T13:33:33.988463+0000","last_fullsized":"2024-04-28T13:33:33.988463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:18.814381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.015707+0000","last_change":"2024-04-28T13:33:34.006579+0000","last_active":"2024-04-28T13:33:34.015707+0000","last_peered":"2024-04-28T13:33:34.015707+0000","last_clean":"2024-04-28T13:33:34.015707+0000","last_became_active":"2024-04-28T13:33:34.006282+0000","last_became_peered":"2024-04-28T13:33:34.006282+0000","last_unstale":"2024-04-28T13:33:34.015707+0000","last_undegraded":"2024-04-28T13:33:34.015707+0000","last_fullsized":"2024-04-28T13:33:34.015707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:09:39.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.016492+0000","last_change":"2024-04-28T13:33:32.999347+0000","last_active":"2024-04-28T13:33:34.016492+0000","last_peered":"2024-04-28T13:33:34.016492+0000","last_clean":"2024-04-28T13:33:34.016492+0000","last_became_active":"2024-04-28T13:33:32.998923+0000","last_became_peered":"2024-04-28T13:33:32.998923+0000","last_unstale":"2024-04-28T13:33:34.016492+0000","last_undegraded":"2024-04-28T13:33:34.016492+0000","last_fullsized":"2024-04-28T13:33:34.016492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:46.522691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.018494+0000","last_change":"2024-04-28T13:33:34.011875+0000","last_active":"2024-04-28T13:33:34.018494+0000","last_peered":"2024-04-28T13:33:34.018494+0000","last_clean":"2024-04-28T13:33:34.018494+0000","last_became_active":"2024-04-28T13:33:34.011271+0000","last_became_peered":"2024-04-28T13:33:34.011271+0000","last_unstale":"2024-04-28T13:33:34.018494+0000","last_undegraded":"2024-04-28T13:33:34.018494+0000","last_fullsized":"2024-04-28T13:33:34.018494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:42.082730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.000983+0000","last_change":"2024-04-28T13:33:33.002216+0000","last_active":"2024-04-28T13:33:35.000983+0000","last_peered":"2024-04-28T13:33:35.000983+0000","last_clean":"2024-04-28T13:33:35.000983+0000","last_became_active":"2024-04-28T13:33:33.000335+0000","last_became_peered":"2024-04-28T13:33:33.000335+0000","last_unstale":"2024-04-28T13:33:35.000983+0000","last_undegraded":"2024-04-28T13:33:35.000983+0000","last_fullsized":"2024-04-28T13:33:35.000983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:34.955026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999276+0000","last_change":"2024-04-28T13:33:34.000104+0000","last_active":"2024-04-28T13:33:34.999276+0000","last_peered":"2024-04-28T13:33:34.999276+0000","last_clean":"2024-04-28T13:33:34.999276+0000","last_became_active":"2024-04-28T13:33:33.999839+0000","last_became_peered":"2024-04-28T13:33:33.999839+0000","last_unstale":"2024-04-28T13:33:34.999276+0000","last_undegraded":"2024-04-28T13:33:34.999276+0000","last_fullsized":"2024-04-28T13:33:34.999276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:03:14.657009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.982243+0000","last_change":"2024-04-28T13:33:32.992149+0000","last_active":"2024-04-28T13:33:33.982243+0000","last_peered":"2024-04-28T13:33:33.982243+0000","last_clean":"2024-04-28T13:33:33.982243+0000","last_became_active":"2024-04-28T13:33:32.991800+0000","last_became_peered":"2024-04-28T13:33:32.991800+0000","last_unstale":"2024-04-28T13:33:33.982243+0000","last_undegraded":"2024-04-28T13:33:33.982243+0000","last_fullsized":"2024-04-28T13:33:33.982243+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:56:39.708282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.016566+0000","last_change":"2024-04-28T13:33:34.011133+0000","last_active":"2024-04-28T13:33:34.016566+0000","last_peered":"2024-04-28T13:33:34.016566+0000","last_clean":"2024-04-28T13:33:34.016566+0000","last_became_active":"2024-04-28T13:33:34.010896+0000","last_became_peered":"2024-04-28T13:33:34.010896+0000","last_unstale":"2024-04-28T13:33:34.016566+0000","last_undegraded":"2024-04-28T13:33:34.016566+0000","last_fullsized":"2024-04-28T13:33:34.016566+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:19.235757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.988279+0000","last_change":"2024-04-28T13:33:33.000859+0000","last_active":"2024-04-28T13:33:33.988279+0000","last_peered":"2024-04-28T13:33:33.988279+0000","last_clean":"2024-04-28T13:33:33.988279+0000","last_became_active":"2024-04-28T13:33:33.000095+0000","last_became_peered":"2024-04-28T13:33:33.000095+0000","last_unstale":"2024-04-28T13:33:33.988279+0000","last_undegraded":"2024-04-28T13:33:33.988279+0000","last_fullsized":"2024-04-28T13:33:33.988279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:18.221214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.010743+0000","last_change":"2024-04-28T13:33:34.008405+0000","last_active":"2024-04-28T13:33:34.010743+0000","last_peered":"2024-04-28T13:33:34.010743+0000","last_clean":"2024-04-28T13:33:34.010743+0000","last_became_active":"2024-04-28T13:33:34.007425+0000","last_became_peered":"2024-04-28T13:33:34.007425+0000","last_unstale":"2024-04-28T13:33:34.010743+0000","last_undegraded":"2024-04-28T13:33:34.010743+0000","last_fullsized":"2024-04-28T13:33:34.010743+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:10.015862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.985821+0000","last_change":"2024-04-28T13:33:32.998594+0000","last_active":"2024-04-28T13:33:33.985821+0000","last_peered":"2024-04-28T13:33:33.985821+0000","last_clean":"2024-04-28T13:33:33.985821+0000","last_became_active":"2024-04-28T13:33:32.998328+0000","last_became_peered":"2024-04-28T13:33:32.998328+0000","last_unstale":"2024-04-28T13:33:33.985821+0000","last_undegraded":"2024-04-28T13:33:33.985821+0000","last_fullsized":"2024-04-28T13:33:33.985821+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:18.221214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.006713+0000","last_change":"2024-04-28T13:33:33.998134+0000","last_active":"2024-04-28T13:33:34.006713+0000","last_peered":"2024-04-28T13:33:34.006713+0000","last_clean":"2024-04-28T13:33:34.006713+0000","last_became_active":"2024-04-28T13:33:33.997834+0000","last_became_peered":"2024-04-28T13:33:33.997834+0000","last_unstale":"2024-04-28T13:33:34.006713+0000","last_undegraded":"2024-04-28T13:33:34.006713+0000","last_fullsized":"2024-04-28T13:33:34.006713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:52:46.527475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.991330+0000","last_change":"2024-04-28T13:33:33.005737+0000","last_active":"2024-04-28T13:33:33.991330+0000","last_peered":"2024-04-28T13:33:33.991330+0000","last_clean":"2024-04-28T13:33:33.991330+0000","last_became_active":"2024-04-28T13:33:33.005179+0000","last_became_peered":"2024-04-28T13:33:33.005179+0000","last_unstale":"2024-04-28T13:33:33.991330+0000","last_undegraded":"2024-04-28T13:33:33.991330+0000","last_fullsized":"2024-04-28T13:33:33.991330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:45.657949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014333+0000","last_change":"2024-04-28T13:33:34.011725+0000","last_active":"2024-04-28T13:33:34.014333+0000","last_peered":"2024-04-28T13:33:34.014333+0000","last_clean":"2024-04-28T13:33:34.014333+0000","last_became_active":"2024-04-28T13:33:34.011422+0000","last_became_peered":"2024-04-28T13:33:34.011422+0000","last_unstale":"2024-04-28T13:33:34.014333+0000","last_undegraded":"2024-04-28T13:33:34.014333+0000","last_fullsized":"2024-04-28T13:33:34.014333+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:01:13.143511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.007375+0000","last_change":"2024-04-28T13:33:32.997338+0000","last_active":"2024-04-28T13:33:34.007375+0000","last_peered":"2024-04-28T13:33:34.007375+0000","last_clean":"2024-04-28T13:33:34.007375+0000","last_became_active":"2024-04-28T13:33:32.996934+0000","last_became_peered":"2024-04-28T13:33:32.996934+0000","last_unstale":"2024-04-28T13:33:34.007375+0000","last_undegraded":"2024-04-28T13:33:34.007375+0000","last_fullsized":"2024-04-28T13:33:34.007375+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:29.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.021671+0000","last_change":"2024-04-28T13:33:34.020082+0000","last_active":"2024-04-28T13:33:34.021671+0000","last_peered":"2024-04-28T13:33:34.021671+0000","last_clean":"2024-04-28T13:33:34.021671+0000","last_became_active":"2024-04-28T13:33:34.018922+0000","last_became_peered":"2024-04-28T13:33:34.018922+0000","last_unstale":"2024-04-28T13:33:34.021671+0000","last_undegraded":"2024-04-28T13:33:34.021671+0000","last_fullsized":"2024-04-28T13:33:34.021671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:28.548233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.989409+0000","last_change":"2024-04-28T13:33:33.000626+0000","last_active":"2024-04-28T13:33:33.989409+0000","last_peered":"2024-04-28T13:33:33.989409+0000","last_clean":"2024-04-28T13:33:33.989409+0000","last_became_active":"2024-04-28T13:33:33.000049+0000","last_became_peered":"2024-04-28T13:33:33.000049+0000","last_unstale":"2024-04-28T13:33:33.989409+0000","last_undegraded":"2024-04-28T13:33:33.989409+0000","last_fullsized":"2024-04-28T13:33:33.989409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:18.814381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.013632+0000","last_change":"2024-04-28T13:33:34.004230+0000","last_active":"2024-04-28T13:33:34.013632+0000","last_peered":"2024-04-28T13:33:34.013632+0000","last_clean":"2024-04-28T13:33:34.013632+0000","last_became_active":"2024-04-28T13:33:34.003940+0000","last_became_peered":"2024-04-28T13:33:34.003940+0000","last_unstale":"2024-04-28T13:33:34.013632+0000","last_undegraded":"2024-04-28T13:33:34.013632+0000","last_fullsized":"2024-04-28T13:33:34.013632+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:01:33.789936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999548+0000","last_change":"2024-04-28T13:33:32.991869+0000","last_active":"2024-04-28T13:33:34.999548+0000","last_peered":"2024-04-28T13:33:34.999548+0000","last_clean":"2024-04-28T13:33:34.999548+0000","last_became_active":"2024-04-28T13:33:32.991435+0000","last_became_peered":"2024-04-28T13:33:32.991435+0000","last_unstale":"2024-04-28T13:33:34.999548+0000","last_undegraded":"2024-04-28T13:33:34.999548+0000","last_fullsized":"2024-04-28T13:33:34.999548+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:41.068187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.000896+0000","last_change":"2024-04-28T13:33:34.004178+0000","last_active":"2024-04-28T13:33:35.000896+0000","last_peered":"2024-04-28T13:33:35.000896+0000","last_clean":"2024-04-28T13:33:35.000896+0000","last_became_active":"2024-04-28T13:33:34.002092+0000","last_became_peered":"2024-04-28T13:33:34.002092+0000","last_unstale":"2024-04-28T13:33:35.000896+0000","last_undegraded":"2024-04-28T13:33:35.000896+0000","last_fullsized":"2024-04-28T13:33:35.000896+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:03.446736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.008707+0000","last_change":"2024-04-28T13:33:33.003796+0000","last_active":"2024-04-28T13:33:34.008707+0000","last_peered":"2024-04-28T13:33:34.008707+0000","last_clean":"2024-04-28T13:33:34.008707+0000","last_became_active":"2024-04-28T13:33:33.003374+0000","last_became_peered":"2024-04-28T13:33:33.003374+0000","last_unstale":"2024-04-28T13:33:34.008707+0000","last_undegraded":"2024-04-28T13:33:34.008707+0000","last_fullsized":"2024-04-28T13:33:34.008707+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:11.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999264+0000","last_change":"2024-04-28T13:33:34.001551+0000","last_active":"2024-04-28T13:33:34.999264+0000","last_peered":"2024-04-28T13:33:34.999264+0000","last_clean":"2024-04-28T13:33:34.999264+0000","last_became_active":"2024-04-28T13:33:33.999963+0000","last_became_peered":"2024-04-28T13:33:33.999963+0000","last_unstale":"2024-04-28T13:33:34.999264+0000","last_undegraded":"2024-04-28T13:33:34.999264+0000","last_fullsized":"2024-04-28T13:33:34.999264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:19.828924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.983634+0000","last_change":"2024-04-28T13:33:32.991033+0000","last_active":"2024-04-28T13:33:33.983634+0000","last_peered":"2024-04-28T13:33:33.983634+0000","last_clean":"2024-04-28T13:33:33.983634+0000","last_became_active":"2024-04-28T13:33:32.990726+0000","last_became_peered":"2024-04-28T13:33:32.990726+0000","last_unstale":"2024-04-28T13:33:33.983634+0000","last_undegraded":"2024-04-28T13:33:33.983634+0000","last_fullsized":"2024-04-28T13:33:33.983634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:09.001319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014662+0000","last_change":"2024-04-28T13:33:34.005421+0000","last_active":"2024-04-28T13:33:34.014662+0000","last_peered":"2024-04-28T13:33:34.014662+0000","last_clean":"2024-04-28T13:33:34.014662+0000","last_became_active":"2024-04-28T13:33:34.004776+0000","last_became_peered":"2024-04-28T13:33:34.004776+0000","last_unstale":"2024-04-28T13:33:34.014662+0000","last_undegraded":"2024-04-28T13:33:34.014662+0000","last_fullsized":"2024-04-28T13:33:34.014662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:42.082730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001892+0000","last_change":"2024-04-28T13:33:32.991462+0000","last_active":"2024-04-28T13:33:35.001892+0000","last_peered":"2024-04-28T13:33:35.001892+0000","last_clean":"2024-04-28T13:33:35.001892+0000","last_became_active":"2024-04-28T13:33:32.991149+0000","last_became_peered":"2024-04-28T13:33:32.991149+0000","last_unstale":"2024-04-28T13:33:35.001892+0000","last_undegraded":"2024-04-28T13:33:35.001892+0000","last_fullsized":"2024-04-28T13:33:35.001892+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:41.068187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":213,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.988364+0000","last_change":"2024-04-28T13:31:11.831538+0000","last_active":"2024-04-28T13:33:33.988364+0000","last_peered":"2024-04-28T13:33:33.988364+0000","last_clean":"2024-04-28T13:33:33.988364+0000","last_became_active":"2024-04-28T13:31:11.828959+0000","last_became_peered":"2024-04-28T13:31:11.828959+0000","last_unstale":"2024-04-28T13:33:33.988364+0000","last_undegraded":"2024-04-28T13:33:33.988364+0000","last_fullsized":"2024-04-28T13:33:33.988364+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_clean_scrub_stamp":"2024-04-28T13:31:10.444388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:25:27.078911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1114112,"data_stored":800469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":524288,"data_stored":210683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":210081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":217923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":210649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":217945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":210081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1114112,"data_stored":800413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T13:33:38.437 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T13:33:38.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.676+0000 7f1728600640 1 Processor -- start 2024-04-28T13:33:38.676 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.676+0000 7f1728600640 1 -- start start 2024-04-28T13:33:38.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.678+0000 7f1728600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f17200d0d00 0x7f17200d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:38.678 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.678+0000 7f1728600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1720160f40 con 0x7f17200b5dd0 2024-04-28T13:33:38.678 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.678+0000 7f1728600640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1720161220 con 0x7f17201469f0 2024-04-28T13:33:38.678 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.678+0000 7f1728600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f17200b5960 con 0x7f17200d0d00 2024-04-28T13:33:38.678 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.678+0000 7f1726600640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f17200b5dd0 0x7f17200b6200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:38234/0 (socket says 172.21.15.196:38234) 2024-04-28T13:33:38.678 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.678+0000 7f1726600640 1 -- 172.21.15.196:0/1060924100 learned_addr learned my addr 172.21.15.196:0/1060924100 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:38.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.678+0000 7f1725200640 1 --2- 172.21.15.196:0/1060924100 >> v2:172.21.15.203:3300/0 conn(0x7f17200d0d00 0x7f17200d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:38.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.679+0000 7f171fe00640 1 -- 172.21.15.196:0/1060924100 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1792827533 0 0) 0x7f1720160f40 con 0x7f17200b5dd0 2024-04-28T13:33:38.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.679+0000 7f171fe00640 1 -- 172.21.15.196:0/1060924100 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1708003520 con 0x7f17200b5dd0 2024-04-28T13:33:38.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.679+0000 7f171fe00640 1 -- 172.21.15.196:0/1060924100 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 630019832 0 0) 0x7f1708003520 con 0x7f17200b5dd0 2024-04-28T13:33:38.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.679+0000 7f171fe00640 1 -- 172.21.15.196:0/1060924100 >> v2:172.21.15.203:3300/0 conn(0x7f17200d0d00 msgr2=0x7f17200d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:38.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.679+0000 7f171fe00640 1 --2- 172.21.15.196:0/1060924100 >> v2:172.21.15.203:3300/0 conn(0x7f17200d0d00 0x7f17200d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.679+0000 7f171fe00640 1 -- 172.21.15.196:0/1060924100 >> v1:172.21.15.203:6789/0 conn(0x7f17201469f0 legacy=0x7f17200ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:38.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.679+0000 7f171fe00640 1 -- 172.21.15.196:0/1060924100 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1720161500 con 0x7f17200b5dd0 2024-04-28T13:33:38.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.679+0000 7f171fe00640 1 -- 172.21.15.196:0/1060924100 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f17180019f0 con 0x7f17200b5dd0 2024-04-28T13:33:38.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.679+0000 7f171fe00640 1 -- 172.21.15.196:0/1060924100 >> v1:172.21.15.196:6789/0 conn(0x7f17200b5dd0 legacy=0x7f17200b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:38.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.679+0000 7f171fe00640 1 --2- 172.21.15.196:0/1060924100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17080039b0 0x7f1708003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:38.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.679+0000 7f171fe00640 1 --2- 172.21.15.196:0/1060924100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1708004800 0x7f1708004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:38.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.679+0000 7f171fe00640 1 --2- 172.21.15.196:0/1060924100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17080051c0 0x7f170800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:38.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.679+0000 7f171fe00640 1 -- 172.21.15.196:0/1060924100 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f170800c030 con 0x7f1708004800 2024-04-28T13:33:38.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.679+0000 7f171fe00640 1 -- 172.21.15.196:0/1060924100 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f170800c1e0 con 0x7f17080039b0 2024-04-28T13:33:38.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.679+0000 7f171fe00640 1 -- 172.21.15.196:0/1060924100 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f170800c3e0 con 0x7f17080051c0 2024-04-28T13:33:38.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.680+0000 7f1725200640 1 --2- 172.21.15.196:0/1060924100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1708004800 0x7f1708004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:38.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.680+0000 7f1725200640 1 -- 172.21.15.196:0/1060924100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17080051c0 msgr2=0x7f170800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:38.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.680+0000 7f1725200640 1 --2- 172.21.15.196:0/1060924100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17080051c0 0x7f170800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.680+0000 7f1725200640 1 -- 172.21.15.196:0/1060924100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17080039b0 msgr2=0x7f1708003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:38.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.680+0000 7f1725200640 1 --2- 172.21.15.196:0/1060924100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17080039b0 0x7f1708003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.680+0000 7f1725200640 1 -- 172.21.15.196:0/1060924100 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1708003520 con 0x7f1708004800 2024-04-28T13:33:38.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.680+0000 7f1726600640 1 -- 172.21.15.196:0/1060924100 reap_dead start 2024-04-28T13:33:38.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.681+0000 7f1725200640 1 --2- 172.21.15.196:0/1060924100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1708004800 0x7f1708004c80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f170c009ac0 tx=0x7f1708005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b3608b3844376 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:38.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.681+0000 7f171fe00640 1 -- 172.21.15.196:0/1060924100 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c036070 con 0x7f1708004800 2024-04-28T13:33:38.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.681+0000 7f171fe00640 1 -- 172.21.15.196:0/1060924100 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f170c031400 con 0x7f1708004800 2024-04-28T13:33:38.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.681+0000 7f171fe00640 1 -- 172.21.15.196:0/1060924100 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c009690 con 0x7f1708004800 2024-04-28T13:33:38.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.682+0000 7f1728600640 1 -- 172.21.15.196:0/1060924100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1708004800 msgr2=0x7f1708004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:38.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.682+0000 7f1728600640 1 --2- 172.21.15.196:0/1060924100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1708004800 0x7f1708004c80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f170c009ac0 tx=0x7f1708005830 comp rx=0 tx=0).stop 2024-04-28T13:33:38.683 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.682+0000 7f1728600640 1 -- 172.21.15.196:0/1060924100 shutdown_connections 2024-04-28T13:33:38.683 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.682+0000 7f1728600640 1 --2- 172.21.15.196:0/1060924100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1708004800 0x7f1708004c80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.683 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.682+0000 7f1728600640 1 -- 172.21.15.196:0/1060924100 >> 172.21.15.196:0/1060924100 conn(0x7f17200cd830 msgr2=0x7f17200cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:38.683 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.682+0000 7f1728600640 1 -- 172.21.15.196:0/1060924100 shutdown_connections 2024-04-28T13:33:38.683 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.682+0000 7f1728600640 1 -- 172.21.15.196:0/1060924100 wait complete. 2024-04-28T13:33:38.684 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.684+0000 7f1728600640 1 Processor -- start 2024-04-28T13:33:38.684 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.684+0000 7f1728600640 1 -- start start 2024-04-28T13:33:38.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.685+0000 7f1728600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1708004800 0x7f1720072b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:38.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.685+0000 7f1728600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f172015d5d0 0x7f17201e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:38.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.685+0000 7f1728600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f172012f710 0x7f17201e9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:38.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.685+0000 7f1728600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1720072020 con 0x7f172015d5d0 2024-04-28T13:33:38.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.685+0000 7f1728600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1720161ac0 con 0x7f1708004800 2024-04-28T13:33:38.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.685+0000 7f1728600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1720160b80 con 0x7f172012f710 2024-04-28T13:33:38.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.685+0000 7f1725200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f172015d5d0 0x7f17201e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:38.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.685+0000 7f1725200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f172015d5d0 0x7f17201e8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59770/0 (socket says 172.21.15.196:59770) 2024-04-28T13:33:38.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.685+0000 7f1725200640 1 -- 172.21.15.196:0/3541995817 learned_addr learned my addr 172.21.15.196:0/3541995817 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:38.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.686+0000 7f1726600640 1 --2- 172.21.15.196:0/3541995817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f172012f710 0x7f17201e9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:38.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.686+0000 7f1725c00640 1 --2- 172.21.15.196:0/3541995817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1708004800 0x7f1720072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:38.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.686+0000 7f1725200640 1 -- 172.21.15.196:0/3541995817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f172012f710 msgr2=0x7f17201e9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:38.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.686+0000 7f1725200640 1 --2- 172.21.15.196:0/3541995817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f172012f710 0x7f17201e9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.686+0000 7f1725200640 1 -- 172.21.15.196:0/3541995817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1708004800 msgr2=0x7f1720072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:38.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.686+0000 7f1725200640 1 --2- 172.21.15.196:0/3541995817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1708004800 0x7f1720072b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.686+0000 7f1725200640 1 -- 172.21.15.196:0/3541995817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172015da60 con 0x7f172015d5d0 2024-04-28T13:33:38.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.686+0000 7f1725200640 1 --2- 172.21.15.196:0/3541995817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f172015d5d0 0x7f17201e8b20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f170c009be0 tx=0x7f170c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:38.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.686+0000 7f171d600640 1 -- 172.21.15.196:0/3541995817 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c03f020 con 0x7f172015d5d0 2024-04-28T13:33:38.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.687+0000 7f171d600640 1 -- 172.21.15.196:0/3541995817 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f170c037040 con 0x7f172015d5d0 2024-04-28T13:33:38.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.687+0000 7f171d600640 1 -- 172.21.15.196:0/3541995817 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c036070 con 0x7f172015d5d0 2024-04-28T13:33:38.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.687+0000 7f1728600640 1 -- 172.21.15.196:0/3541995817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172015dd40 con 0x7f172015d5d0 2024-04-28T13:33:38.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.687+0000 7f1728600640 1 -- 172.21.15.196:0/3541995817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172015e300 con 0x7f172015d5d0 2024-04-28T13:33:38.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.688+0000 7f1728600640 1 -- 172.21.15.196:0/3541995817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f172006a290 con 0x7f172015d5d0 2024-04-28T13:33:38.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.689+0000 7f171d600640 1 -- 172.21.15.196:0/3541995817 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f170c042020 con 0x7f172015d5d0 2024-04-28T13:33:38.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.690+0000 7f171d600640 1 --2- 172.21.15.196:0/3541995817 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f170006c980 0x7f170006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:38.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.690+0000 7f171d600640 1 -- 172.21.15.196:0/3541995817 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f170c0abca0 con 0x7f172015d5d0 2024-04-28T13:33:38.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.690+0000 7f1725c00640 1 --2- 172.21.15.196:0/3541995817 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f170006c980 0x7f170006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:38.691 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.691+0000 7f1725c00640 1 --2- 172.21.15.196:0/3541995817 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f170006c980 0x7f170006ee40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1714007fd0 tx=0x7f1714009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:38.694 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.694+0000 7f171d600640 1 -- 172.21.15.196:0/3541995817 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f170c049030 con 0x7f172015d5d0 2024-04-28T13:33:38.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.889+0000 7f1728600640 1 -- 172.21.15.196:0/3541995817 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f172006fd90 con 0x7f170006c980 2024-04-28T13:33:38.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.908+0000 7f171d600640 1 -- 172.21.15.196:0/3541995817 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251913 (secure 0 0 0) 0x7f172006fd90 con 0x7f170006c980 2024-04-28T13:33:38.912 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:38.913 INFO:teuthology.orchestra.run.smithi196.stderr:dumped all 2024-04-28T13:33:38.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.914+0000 7f1728600640 1 -- 172.21.15.196:0/3541995817 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f170006c980 msgr2=0x7f170006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:38.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.914+0000 7f1728600640 1 --2- 172.21.15.196:0/3541995817 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f170006c980 0x7f170006ee40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1714007fd0 tx=0x7f1714009040 comp rx=0 tx=0).stop 2024-04-28T13:33:38.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.914+0000 7f1728600640 1 -- 172.21.15.196:0/3541995817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f172015d5d0 msgr2=0x7f17201e8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:38.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.914+0000 7f1728600640 1 --2- 172.21.15.196:0/3541995817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f172015d5d0 0x7f17201e8b20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f170c009be0 tx=0x7f170c004730 comp rx=0 tx=0).stop 2024-04-28T13:33:38.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.914+0000 7f1728600640 1 -- 172.21.15.196:0/3541995817 shutdown_connections 2024-04-28T13:33:38.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.914+0000 7f1728600640 1 --2- 172.21.15.196:0/3541995817 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f170006c980 0x7f170006ee40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.914+0000 7f1728600640 1 --2- 172.21.15.196:0/3541995817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f172012f710 0x7f17201e9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.914+0000 7f1728600640 1 --2- 172.21.15.196:0/3541995817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f172015d5d0 0x7f17201e8b20 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.914+0000 7f1728600640 1 --2- 172.21.15.196:0/3541995817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1708004800 0x7f1720072b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:38.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.914+0000 7f1728600640 1 -- 172.21.15.196:0/3541995817 >> 172.21.15.196:0/3541995817 conn(0x7f17200cd830 msgr2=0x7f17200b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:38.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.914+0000 7f1728600640 1 -- 172.21.15.196:0/3541995817 shutdown_connections 2024-04-28T13:33:38.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:38.914+0000 7f1728600640 1 -- 172.21.15.196:0/3541995817 wait complete. 2024-04-28T13:33:38.934 INFO:teuthology.orchestra.run.smithi196.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-04-28T13:33:38.230620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225636,"kb_used_data":5280,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749084,"statfs":{"total":773094113280,"available":772863062016,"internally_reserved":0,"allocated":5406720,"data_stored":2878244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002207"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.001905+0000","last_change":"2024-04-28T13:33:33.999818+0000","last_active":"2024-04-28T13:33:34.001905+0000","last_peered":"2024-04-28T13:33:34.001905+0000","last_clean":"2024-04-28T13:33:34.001905+0000","last_became_active":"2024-04-28T13:33:33.999486+0000","last_became_peered":"2024-04-28T13:33:33.999486+0000","last_unstale":"2024-04-28T13:33:34.001905+0000","last_undegraded":"2024-04-28T13:33:34.001905+0000","last_fullsized":"2024-04-28T13:33:34.001905+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001490+0000","last_change":"2024-04-28T13:33:34.003919+0000","last_active":"2024-04-28T13:33:35.001490+0000","last_peered":"2024-04-28T13:33:35.001490+0000","last_clean":"2024-04-28T13:33:35.001490+0000","last_became_active":"2024-04-28T13:33:34.003551+0000","last_became_peered":"2024-04-28T13:33:34.003551+0000","last_unstale":"2024-04-28T13:33:35.001490+0000","last_undegraded":"2024-04-28T13:33:35.001490+0000","last_fullsized":"2024-04-28T13:33:35.001490+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999420+0000","last_change":"2024-04-28T13:33:33.996733+0000","last_active":"2024-04-28T13:33:34.999420+0000","last_peered":"2024-04-28T13:33:34.999420+0000","last_clean":"2024-04-28T13:33:34.999420+0000","last_became_active":"2024-04-28T13:33:33.996413+0000","last_became_peered":"2024-04-28T13:33:33.996413+0000","last_unstale":"2024-04-28T13:33:34.999420+0000","last_undegraded":"2024-04-28T13:33:34.999420+0000","last_fullsized":"2024-04-28T13:33:34.999420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:46.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.007952+0000","last_change":"2024-04-28T13:33:34.004787+0000","last_active":"2024-04-28T13:33:34.007952+0000","last_peered":"2024-04-28T13:33:34.007952+0000","last_clean":"2024-04-28T13:33:34.007952+0000","last_became_active":"2024-04-28T13:33:33.999818+0000","last_became_peered":"2024-04-28T13:33:33.999818+0000","last_unstale":"2024-04-28T13:33:34.007952+0000","last_undegraded":"2024-04-28T13:33:34.007952+0000","last_fullsized":"2024-04-28T13:33:34.007952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:52.292838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001180+0000","last_change":"2024-04-28T13:33:34.004715+0000","last_active":"2024-04-28T13:33:35.001180+0000","last_peered":"2024-04-28T13:33:35.001180+0000","last_clean":"2024-04-28T13:33:35.001180+0000","last_became_active":"2024-04-28T13:33:34.004436+0000","last_became_peered":"2024-04-28T13:33:34.004436+0000","last_unstale":"2024-04-28T13:33:35.001180+0000","last_undegraded":"2024-04-28T13:33:35.001180+0000","last_fullsized":"2024-04-28T13:33:35.001180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:46.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.018650+0000","last_change":"2024-04-28T13:33:34.012959+0000","last_active":"2024-04-28T13:33:34.018650+0000","last_peered":"2024-04-28T13:33:34.018650+0000","last_clean":"2024-04-28T13:33:34.018650+0000","last_became_active":"2024-04-28T13:33:34.010839+0000","last_became_peered":"2024-04-28T13:33:34.010839+0000","last_unstale":"2024-04-28T13:33:34.018650+0000","last_undegraded":"2024-04-28T13:33:34.018650+0000","last_fullsized":"2024-04-28T13:33:34.018650+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:32:47.409486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.010263+0000","last_change":"2024-04-28T13:33:34.003448+0000","last_active":"2024-04-28T13:33:34.010263+0000","last_peered":"2024-04-28T13:33:34.010263+0000","last_clean":"2024-04-28T13:33:34.010263+0000","last_became_active":"2024-04-28T13:33:34.002948+0000","last_became_peered":"2024-04-28T13:33:34.002948+0000","last_unstale":"2024-04-28T13:33:34.010263+0000","last_undegraded":"2024-04-28T13:33:34.010263+0000","last_fullsized":"2024-04-28T13:33:34.010263+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014060+0000","last_change":"2024-04-28T13:33:34.007814+0000","last_active":"2024-04-28T13:33:34.014060+0000","last_peered":"2024-04-28T13:33:34.014060+0000","last_clean":"2024-04-28T13:33:34.014060+0000","last_became_active":"2024-04-28T13:33:34.007485+0000","last_became_peered":"2024-04-28T13:33:34.007485+0000","last_unstale":"2024-04-28T13:33:34.014060+0000","last_undegraded":"2024-04-28T13:33:34.014060+0000","last_fullsized":"2024-04-28T13:33:34.014060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:37:53.261649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.016586+0000","last_change":"2024-04-28T13:33:34.013366+0000","last_active":"2024-04-28T13:33:34.016586+0000","last_peered":"2024-04-28T13:33:34.016586+0000","last_clean":"2024-04-28T13:33:34.016586+0000","last_became_active":"2024-04-28T13:33:34.012900+0000","last_became_peered":"2024-04-28T13:33:34.012900+0000","last_unstale":"2024-04-28T13:33:34.016586+0000","last_undegraded":"2024-04-28T13:33:34.016586+0000","last_fullsized":"2024-04-28T13:33:34.016586+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:44.577439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014683+0000","last_change":"2024-04-28T13:33:34.007098+0000","last_active":"2024-04-28T13:33:34.014683+0000","last_peered":"2024-04-28T13:33:34.014683+0000","last_clean":"2024-04-28T13:33:34.014683+0000","last_became_active":"2024-04-28T13:33:34.005157+0000","last_became_peered":"2024-04-28T13:33:34.005157+0000","last_unstale":"2024-04-28T13:33:34.014683+0000","last_undegraded":"2024-04-28T13:33:34.014683+0000","last_fullsized":"2024-04-28T13:33:34.014683+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:18:56.809253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014801+0000","last_change":"2024-04-28T13:33:34.011118+0000","last_active":"2024-04-28T13:33:34.014801+0000","last_peered":"2024-04-28T13:33:34.014801+0000","last_clean":"2024-04-28T13:33:34.014801+0000","last_became_active":"2024-04-28T13:33:34.010811+0000","last_became_peered":"2024-04-28T13:33:34.010811+0000","last_unstale":"2024-04-28T13:33:34.014801+0000","last_undegraded":"2024-04-28T13:33:34.014801+0000","last_fullsized":"2024-04-28T13:33:34.014801+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:14.152642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001244+0000","last_change":"2024-04-28T13:33:34.011900+0000","last_active":"2024-04-28T13:33:35.001244+0000","last_peered":"2024-04-28T13:33:35.001244+0000","last_clean":"2024-04-28T13:33:35.001244+0000","last_became_active":"2024-04-28T13:33:34.011580+0000","last_became_peered":"2024-04-28T13:33:34.011580+0000","last_unstale":"2024-04-28T13:33:35.001244+0000","last_undegraded":"2024-04-28T13:33:35.001244+0000","last_fullsized":"2024-04-28T13:33:35.001244+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999662+0000","last_change":"2024-04-28T13:33:34.000874+0000","last_active":"2024-04-28T13:33:34.999662+0000","last_peered":"2024-04-28T13:33:34.999662+0000","last_clean":"2024-04-28T13:33:34.999662+0000","last_became_active":"2024-04-28T13:33:34.000568+0000","last_became_peered":"2024-04-28T13:33:34.000568+0000","last_unstale":"2024-04-28T13:33:34.999662+0000","last_undegraded":"2024-04-28T13:33:34.999662+0000","last_fullsized":"2024-04-28T13:33:34.999662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:56:40.722825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.015183+0000","last_change":"2024-04-28T13:33:34.012076+0000","last_active":"2024-04-28T13:33:34.015183+0000","last_peered":"2024-04-28T13:33:34.015183+0000","last_clean":"2024-04-28T13:33:34.015183+0000","last_became_active":"2024-04-28T13:33:34.011795+0000","last_became_peered":"2024-04-28T13:33:34.011795+0000","last_unstale":"2024-04-28T13:33:34.015183+0000","last_undegraded":"2024-04-28T13:33:34.015183+0000","last_fullsized":"2024-04-28T13:33:34.015183+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:29.495657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.004135+0000","last_change":"2024-04-28T13:33:33.998387+0000","last_active":"2024-04-28T13:33:34.004135+0000","last_peered":"2024-04-28T13:33:34.004135+0000","last_clean":"2024-04-28T13:33:34.004135+0000","last_became_active":"2024-04-28T13:33:33.998087+0000","last_became_peered":"2024-04-28T13:33:33.998087+0000","last_unstale":"2024-04-28T13:33:34.004135+0000","last_undegraded":"2024-04-28T13:33:34.004135+0000","last_fullsized":"2024-04-28T13:33:34.004135+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.998844+0000","last_change":"2024-04-28T13:33:33.989154+0000","last_active":"2024-04-28T13:33:33.998844+0000","last_peered":"2024-04-28T13:33:33.998844+0000","last_clean":"2024-04-28T13:33:33.998844+0000","last_became_active":"2024-04-28T13:33:33.988827+0000","last_became_peered":"2024-04-28T13:33:33.988827+0000","last_unstale":"2024-04-28T13:33:33.998844+0000","last_undegraded":"2024-04-28T13:33:33.998844+0000","last_fullsized":"2024-04-28T13:33:33.998844+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:03:40.797677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999849+0000","last_change":"2024-04-28T13:33:34.008467+0000","last_active":"2024-04-28T13:33:34.999849+0000","last_peered":"2024-04-28T13:33:34.999849+0000","last_clean":"2024-04-28T13:33:34.999849+0000","last_became_active":"2024-04-28T13:33:34.008181+0000","last_became_peered":"2024-04-28T13:33:34.008181+0000","last_unstale":"2024-04-28T13:33:34.999849+0000","last_undegraded":"2024-04-28T13:33:34.999849+0000","last_fullsized":"2024-04-28T13:33:34.999849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999757+0000","last_change":"2024-04-28T13:33:34.007505+0000","last_active":"2024-04-28T13:33:34.999757+0000","last_peered":"2024-04-28T13:33:34.999757+0000","last_clean":"2024-04-28T13:33:34.999757+0000","last_became_active":"2024-04-28T13:33:34.006793+0000","last_became_peered":"2024-04-28T13:33:34.006793+0000","last_unstale":"2024-04-28T13:33:34.999757+0000","last_undegraded":"2024-04-28T13:33:34.999757+0000","last_fullsized":"2024-04-28T13:33:34.999757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.011743+0000","last_change":"2024-04-28T13:33:34.009974+0000","last_active":"2024-04-28T13:33:34.011743+0000","last_peered":"2024-04-28T13:33:34.011743+0000","last_clean":"2024-04-28T13:33:34.011743+0000","last_became_active":"2024-04-28T13:33:34.009657+0000","last_became_peered":"2024-04-28T13:33:34.009657+0000","last_unstale":"2024-04-28T13:33:34.011743+0000","last_undegraded":"2024-04-28T13:33:34.011743+0000","last_fullsized":"2024-04-28T13:33:34.011743+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:37:19.643053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999758+0000","last_change":"2024-04-28T13:33:34.004233+0000","last_active":"2024-04-28T13:33:34.999758+0000","last_peered":"2024-04-28T13:33:34.999758+0000","last_clean":"2024-04-28T13:33:34.999758+0000","last_became_active":"2024-04-28T13:33:33.999567+0000","last_became_peered":"2024-04-28T13:33:33.999567+0000","last_unstale":"2024-04-28T13:33:34.999758+0000","last_undegraded":"2024-04-28T13:33:34.999758+0000","last_fullsized":"2024-04-28T13:33:34.999758+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:36.248393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.016459+0000","last_change":"2024-04-28T13:33:34.008932+0000","last_active":"2024-04-28T13:33:34.016459+0000","last_peered":"2024-04-28T13:33:34.016459+0000","last_clean":"2024-04-28T13:33:34.016459+0000","last_became_active":"2024-04-28T13:33:34.008652+0000","last_became_peered":"2024-04-28T13:33:34.008652+0000","last_unstale":"2024-04-28T13:33:34.016459+0000","last_undegraded":"2024-04-28T13:33:34.016459+0000","last_fullsized":"2024-04-28T13:33:34.016459+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:54.852573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014147+0000","last_change":"2024-04-28T13:33:34.006521+0000","last_active":"2024-04-28T13:33:34.014147+0000","last_peered":"2024-04-28T13:33:34.014147+0000","last_clean":"2024-04-28T13:33:34.014147+0000","last_became_active":"2024-04-28T13:33:34.006227+0000","last_became_peered":"2024-04-28T13:33:34.006227+0000","last_unstale":"2024-04-28T13:33:34.014147+0000","last_undegraded":"2024-04-28T13:33:34.014147+0000","last_fullsized":"2024-04-28T13:33:34.014147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:31.671210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.001556+0000","last_change":"2024-04-28T13:33:33.999169+0000","last_active":"2024-04-28T13:33:34.001556+0000","last_peered":"2024-04-28T13:33:34.001556+0000","last_clean":"2024-04-28T13:33:34.001556+0000","last_became_active":"2024-04-28T13:33:33.998179+0000","last_became_peered":"2024-04-28T13:33:33.998179+0000","last_unstale":"2024-04-28T13:33:34.001556+0000","last_undegraded":"2024-04-28T13:33:34.001556+0000","last_fullsized":"2024-04-28T13:33:34.001556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:52:06.849498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.001462+0000","last_change":"2024-04-28T13:33:33.998034+0000","last_active":"2024-04-28T13:33:34.001462+0000","last_peered":"2024-04-28T13:33:34.001462+0000","last_clean":"2024-04-28T13:33:34.001462+0000","last_became_active":"2024-04-28T13:33:33.997648+0000","last_became_peered":"2024-04-28T13:33:33.997648+0000","last_unstale":"2024-04-28T13:33:34.001462+0000","last_undegraded":"2024-04-28T13:33:34.001462+0000","last_fullsized":"2024-04-28T13:33:34.001462+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:12:14.181038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.009634+0000","last_change":"2024-04-28T13:33:34.007275+0000","last_active":"2024-04-28T13:33:34.009634+0000","last_peered":"2024-04-28T13:33:34.009634+0000","last_clean":"2024-04-28T13:33:34.009634+0000","last_became_active":"2024-04-28T13:33:34.005017+0000","last_became_peered":"2024-04-28T13:33:34.005017+0000","last_unstale":"2024-04-28T13:33:34.009634+0000","last_undegraded":"2024-04-28T13:33:34.009634+0000","last_fullsized":"2024-04-28T13:33:34.009634+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:36.248393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999682+0000","last_change":"2024-04-28T13:33:33.995804+0000","last_active":"2024-04-28T13:33:34.999682+0000","last_peered":"2024-04-28T13:33:34.999682+0000","last_clean":"2024-04-28T13:33:34.999682+0000","last_became_active":"2024-04-28T13:33:33.995502+0000","last_became_peered":"2024-04-28T13:33:33.995502+0000","last_unstale":"2024-04-28T13:33:34.999682+0000","last_undegraded":"2024-04-28T13:33:34.999682+0000","last_fullsized":"2024-04-28T13:33:34.999682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:27:53.804713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999622+0000","last_change":"2024-04-28T13:33:34.006447+0000","last_active":"2024-04-28T13:33:34.999622+0000","last_peered":"2024-04-28T13:33:34.999622+0000","last_clean":"2024-04-28T13:33:34.999622+0000","last_became_active":"2024-04-28T13:33:34.005656+0000","last_became_peered":"2024-04-28T13:33:34.005656+0000","last_unstale":"2024-04-28T13:33:34.999622+0000","last_undegraded":"2024-04-28T13:33:34.999622+0000","last_fullsized":"2024-04-28T13:33:34.999622+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001410+0000","last_change":"2024-04-28T13:33:34.005608+0000","last_active":"2024-04-28T13:33:35.001410+0000","last_peered":"2024-04-28T13:33:35.001410+0000","last_clean":"2024-04-28T13:33:35.001410+0000","last_became_active":"2024-04-28T13:33:34.005316+0000","last_became_peered":"2024-04-28T13:33:34.005316+0000","last_unstale":"2024-04-28T13:33:35.001410+0000","last_undegraded":"2024-04-28T13:33:35.001410+0000","last_fullsized":"2024-04-28T13:33:35.001410+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.004506+0000","last_change":"2024-04-28T13:33:34.002648+0000","last_active":"2024-04-28T13:33:34.004506+0000","last_peered":"2024-04-28T13:33:34.004506+0000","last_clean":"2024-04-28T13:33:34.004506+0000","last_became_active":"2024-04-28T13:33:34.002375+0000","last_became_peered":"2024-04-28T13:33:34.002375+0000","last_unstale":"2024-04-28T13:33:34.004506+0000","last_undegraded":"2024-04-28T13:33:34.004506+0000","last_fullsized":"2024-04-28T13:33:34.004506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:56:40.722825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999556+0000","last_change":"2024-04-28T13:33:33.999443+0000","last_active":"2024-04-28T13:33:34.999556+0000","last_peered":"2024-04-28T13:33:34.999556+0000","last_clean":"2024-04-28T13:33:34.999556+0000","last_became_active":"2024-04-28T13:33:33.998905+0000","last_became_peered":"2024-04-28T13:33:33.998905+0000","last_unstale":"2024-04-28T13:33:34.999556+0000","last_undegraded":"2024-04-28T13:33:34.999556+0000","last_fullsized":"2024-04-28T13:33:34.999556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.016079+0000","last_change":"2024-04-28T13:33:34.008881+0000","last_active":"2024-04-28T13:33:34.016079+0000","last_peered":"2024-04-28T13:33:34.016079+0000","last_clean":"2024-04-28T13:33:34.016079+0000","last_became_active":"2024-04-28T13:33:34.008297+0000","last_became_peered":"2024-04-28T13:33:34.008297+0000","last_unstale":"2024-04-28T13:33:34.016079+0000","last_undegraded":"2024-04-28T13:33:34.016079+0000","last_fullsized":"2024-04-28T13:33:34.016079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:20:07.691466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999181+0000","last_change":"2024-04-28T13:33:34.003176+0000","last_active":"2024-04-28T13:33:34.999181+0000","last_peered":"2024-04-28T13:33:34.999181+0000","last_clean":"2024-04-28T13:33:34.999181+0000","last_became_active":"2024-04-28T13:33:34.002915+0000","last_became_peered":"2024-04-28T13:33:34.002915+0000","last_unstale":"2024-04-28T13:33:34.999181+0000","last_undegraded":"2024-04-28T13:33:34.999181+0000","last_fullsized":"2024-04-28T13:33:34.999181+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:52:06.849498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014933+0000","last_change":"2024-04-28T13:33:34.005669+0000","last_active":"2024-04-28T13:33:34.014933+0000","last_peered":"2024-04-28T13:33:34.014933+0000","last_clean":"2024-04-28T13:33:34.014933+0000","last_became_active":"2024-04-28T13:33:34.005365+0000","last_became_peered":"2024-04-28T13:33:34.005365+0000","last_unstale":"2024-04-28T13:33:34.014933+0000","last_undegraded":"2024-04-28T13:33:34.014933+0000","last_fullsized":"2024-04-28T13:33:34.014933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:22:35.458698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.017552+0000","last_change":"2024-04-28T13:33:32.998037+0000","last_active":"2024-04-28T13:33:34.017552+0000","last_peered":"2024-04-28T13:33:34.017552+0000","last_clean":"2024-04-28T13:33:34.017552+0000","last_became_active":"2024-04-28T13:33:32.997694+0000","last_became_peered":"2024-04-28T13:33:32.997694+0000","last_unstale":"2024-04-28T13:33:34.017552+0000","last_undegraded":"2024-04-28T13:33:34.017552+0000","last_fullsized":"2024-04-28T13:33:34.017552+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:02.432193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.998769+0000","last_change":"2024-04-28T13:33:33.989048+0000","last_active":"2024-04-28T13:33:33.998769+0000","last_peered":"2024-04-28T13:33:33.998769+0000","last_clean":"2024-04-28T13:33:33.998769+0000","last_became_active":"2024-04-28T13:33:33.988758+0000","last_became_peered":"2024-04-28T13:33:33.988758+0000","last_unstale":"2024-04-28T13:33:33.998769+0000","last_undegraded":"2024-04-28T13:33:33.998769+0000","last_fullsized":"2024-04-28T13:33:33.998769+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:24:18.185935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999883+0000","last_change":"2024-04-28T13:33:32.999239+0000","last_active":"2024-04-28T13:33:34.999883+0000","last_peered":"2024-04-28T13:33:34.999883+0000","last_clean":"2024-04-28T13:33:34.999883+0000","last_became_active":"2024-04-28T13:33:32.998813+0000","last_became_peered":"2024-04-28T13:33:32.998813+0000","last_unstale":"2024-04-28T13:33:34.999883+0000","last_undegraded":"2024-04-28T13:33:34.999883+0000","last_fullsized":"2024-04-28T13:33:34.999883+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:22:34.444155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001579+0000","last_change":"2024-04-28T13:33:34.003176+0000","last_active":"2024-04-28T13:33:35.001579+0000","last_peered":"2024-04-28T13:33:35.001579+0000","last_clean":"2024-04-28T13:33:35.001579+0000","last_became_active":"2024-04-28T13:33:34.002904+0000","last_became_peered":"2024-04-28T13:33:34.002904+0000","last_unstale":"2024-04-28T13:33:35.001579+0000","last_undegraded":"2024-04-28T13:33:35.001579+0000","last_fullsized":"2024-04-28T13:33:35.001579+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.018798+0000","last_change":"2024-04-28T13:33:32.997729+0000","last_active":"2024-04-28T13:33:34.018798+0000","last_peered":"2024-04-28T13:33:34.018798+0000","last_clean":"2024-04-28T13:33:34.018798+0000","last_became_active":"2024-04-28T13:33:32.996881+0000","last_became_peered":"2024-04-28T13:33:32.996881+0000","last_unstale":"2024-04-28T13:33:34.018798+0000","last_undegraded":"2024-04-28T13:33:34.018798+0000","last_fullsized":"2024-04-28T13:33:34.018798+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.016605+0000","last_change":"2024-04-28T13:33:34.004339+0000","last_active":"2024-04-28T13:33:34.016605+0000","last_peered":"2024-04-28T13:33:34.016605+0000","last_clean":"2024-04-28T13:33:34.016605+0000","last_became_active":"2024-04-28T13:33:34.002361+0000","last_became_peered":"2024-04-28T13:33:34.002361+0000","last_unstale":"2024-04-28T13:33:34.016605+0000","last_undegraded":"2024-04-28T13:33:34.016605+0000","last_fullsized":"2024-04-28T13:33:34.016605+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:52:46.527475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.010939+0000","last_change":"2024-04-28T13:33:33.000754+0000","last_active":"2024-04-28T13:33:34.010939+0000","last_peered":"2024-04-28T13:33:34.010939+0000","last_clean":"2024-04-28T13:33:34.010939+0000","last_became_active":"2024-04-28T13:33:33.000387+0000","last_became_peered":"2024-04-28T13:33:33.000387+0000","last_unstale":"2024-04-28T13:33:34.010939+0000","last_undegraded":"2024-04-28T13:33:34.010939+0000","last_fullsized":"2024-04-28T13:33:34.010939+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:28.481114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999283+0000","last_change":"2024-04-28T13:33:33.994316+0000","last_active":"2024-04-28T13:33:34.999283+0000","last_peered":"2024-04-28T13:33:34.999283+0000","last_clean":"2024-04-28T13:33:34.999283+0000","last_became_active":"2024-04-28T13:33:33.994002+0000","last_became_peered":"2024-04-28T13:33:33.994002+0000","last_unstale":"2024-04-28T13:33:34.999283+0000","last_undegraded":"2024-04-28T13:33:34.999283+0000","last_fullsized":"2024-04-28T13:33:34.999283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:03.446736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.016168+0000","last_change":"2024-04-28T13:33:32.999540+0000","last_active":"2024-04-28T13:33:34.016168+0000","last_peered":"2024-04-28T13:33:34.016168+0000","last_clean":"2024-04-28T13:33:34.016168+0000","last_became_active":"2024-04-28T13:33:32.999147+0000","last_became_peered":"2024-04-28T13:33:32.999147+0000","last_unstale":"2024-04-28T13:33:34.016168+0000","last_undegraded":"2024-04-28T13:33:34.016168+0000","last_fullsized":"2024-04-28T13:33:34.016168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:45.657949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.005003+0000","last_change":"2024-04-28T13:33:34.000424+0000","last_active":"2024-04-28T13:33:34.005003+0000","last_peered":"2024-04-28T13:33:34.005003+0000","last_clean":"2024-04-28T13:33:34.005003+0000","last_became_active":"2024-04-28T13:33:33.999414+0000","last_became_peered":"2024-04-28T13:33:33.999414+0000","last_unstale":"2024-04-28T13:33:34.005003+0000","last_undegraded":"2024-04-28T13:33:34.005003+0000","last_fullsized":"2024-04-28T13:33:34.005003+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:22:35.458698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999411+0000","last_change":"2024-04-28T13:33:32.988943+0000","last_active":"2024-04-28T13:33:34.999411+0000","last_peered":"2024-04-28T13:33:34.999411+0000","last_clean":"2024-04-28T13:33:34.999411+0000","last_became_active":"2024-04-28T13:33:32.986596+0000","last_became_peered":"2024-04-28T13:33:32.986596+0000","last_unstale":"2024-04-28T13:33:34.999411+0000","last_undegraded":"2024-04-28T13:33:34.999411+0000","last_fullsized":"2024-04-28T13:33:34.999411+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.007696+0000","last_change":"2024-04-28T13:33:34.002579+0000","last_active":"2024-04-28T13:33:34.007696+0000","last_peered":"2024-04-28T13:33:34.007696+0000","last_clean":"2024-04-28T13:33:34.007696+0000","last_became_active":"2024-04-28T13:33:34.002254+0000","last_became_peered":"2024-04-28T13:33:34.002254+0000","last_unstale":"2024-04-28T13:33:34.007696+0000","last_undegraded":"2024-04-28T13:33:34.007696+0000","last_fullsized":"2024-04-28T13:33:34.007696+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:30.570296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001072+0000","last_change":"2024-04-28T13:33:32.996052+0000","last_active":"2024-04-28T13:33:35.001072+0000","last_peered":"2024-04-28T13:33:35.001072+0000","last_clean":"2024-04-28T13:33:35.001072+0000","last_became_active":"2024-04-28T13:33:32.990258+0000","last_became_peered":"2024-04-28T13:33:32.990258+0000","last_unstale":"2024-04-28T13:33:35.001072+0000","last_undegraded":"2024-04-28T13:33:35.001072+0000","last_fullsized":"2024-04-28T13:33:35.001072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.001508+0000","last_change":"2024-04-28T13:33:33.998180+0000","last_active":"2024-04-28T13:33:34.001508+0000","last_peered":"2024-04-28T13:33:34.001508+0000","last_clean":"2024-04-28T13:33:34.001508+0000","last_became_active":"2024-04-28T13:33:33.997730+0000","last_became_peered":"2024-04-28T13:33:33.997730+0000","last_unstale":"2024-04-28T13:33:34.001508+0000","last_undegraded":"2024-04-28T13:33:34.001508+0000","last_fullsized":"2024-04-28T13:33:34.001508+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:55:17.195691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.983250+0000","last_change":"2024-04-28T13:33:32.989759+0000","last_active":"2024-04-28T13:33:33.983250+0000","last_peered":"2024-04-28T13:33:33.983250+0000","last_clean":"2024-04-28T13:33:33.983250+0000","last_became_active":"2024-04-28T13:33:32.988565+0000","last_became_peered":"2024-04-28T13:33:32.988565+0000","last_unstale":"2024-04-28T13:33:33.983250+0000","last_undegraded":"2024-04-28T13:33:33.983250+0000","last_fullsized":"2024-04-28T13:33:33.983250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:27:52.790170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.000164+0000","last_change":"2024-04-28T13:33:34.007736+0000","last_active":"2024-04-28T13:33:35.000164+0000","last_peered":"2024-04-28T13:33:35.000164+0000","last_clean":"2024-04-28T13:33:35.000164+0000","last_became_active":"2024-04-28T13:33:34.007170+0000","last_became_peered":"2024-04-28T13:33:34.007170+0000","last_unstale":"2024-04-28T13:33:35.000164+0000","last_undegraded":"2024-04-28T13:33:35.000164+0000","last_fullsized":"2024-04-28T13:33:35.000164+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:01:33.789936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.000108+0000","last_change":"2024-04-28T13:33:32.992483+0000","last_active":"2024-04-28T13:33:35.000108+0000","last_peered":"2024-04-28T13:33:35.000108+0000","last_clean":"2024-04-28T13:33:35.000108+0000","last_became_active":"2024-04-28T13:33:32.992216+0000","last_became_peered":"2024-04-28T13:33:32.992216+0000","last_unstale":"2024-04-28T13:33:35.000108+0000","last_undegraded":"2024-04-28T13:33:35.000108+0000","last_fullsized":"2024-04-28T13:33:35.000108+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:11.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.011091+0000","last_change":"2024-04-28T13:33:34.009420+0000","last_active":"2024-04-28T13:33:34.011091+0000","last_peered":"2024-04-28T13:33:34.011091+0000","last_clean":"2024-04-28T13:33:34.011091+0000","last_became_active":"2024-04-28T13:33:34.009020+0000","last_became_peered":"2024-04-28T13:33:34.009020+0000","last_unstale":"2024-04-28T13:33:34.011091+0000","last_undegraded":"2024-04-28T13:33:34.011091+0000","last_fullsized":"2024-04-28T13:33:34.011091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.991627+0000","last_change":"2024-04-28T13:33:32.997498+0000","last_active":"2024-04-28T13:33:33.991627+0000","last_peered":"2024-04-28T13:33:33.991627+0000","last_clean":"2024-04-28T13:33:33.991627+0000","last_became_active":"2024-04-28T13:33:32.997237+0000","last_became_peered":"2024-04-28T13:33:32.997237+0000","last_unstale":"2024-04-28T13:33:33.991627+0000","last_undegraded":"2024-04-28T13:33:33.991627+0000","last_fullsized":"2024-04-28T13:33:33.991627+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:37:18.628510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.008405+0000","last_change":"2024-04-28T13:33:34.002865+0000","last_active":"2024-04-28T13:33:34.008405+0000","last_peered":"2024-04-28T13:33:34.008405+0000","last_clean":"2024-04-28T13:33:34.008405+0000","last_became_active":"2024-04-28T13:33:34.002421+0000","last_became_peered":"2024-04-28T13:33:34.002421+0000","last_unstale":"2024-04-28T13:33:34.008405+0000","last_undegraded":"2024-04-28T13:33:34.008405+0000","last_fullsized":"2024-04-28T13:33:34.008405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.007950+0000","last_change":"2024-04-28T13:33:33.004559+0000","last_active":"2024-04-28T13:33:34.007950+0000","last_peered":"2024-04-28T13:33:34.007950+0000","last_clean":"2024-04-28T13:33:34.007950+0000","last_became_active":"2024-04-28T13:33:33.004153+0000","last_became_peered":"2024-04-28T13:33:33.004153+0000","last_unstale":"2024-04-28T13:33:34.007950+0000","last_undegraded":"2024-04-28T13:33:34.007950+0000","last_fullsized":"2024-04-28T13:33:34.007950+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:02.432193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.012182+0000","last_change":"2024-04-28T13:33:34.006854+0000","last_active":"2024-04-28T13:33:34.012182+0000","last_peered":"2024-04-28T13:33:34.012182+0000","last_clean":"2024-04-28T13:33:34.012182+0000","last_became_active":"2024-04-28T13:33:34.006561+0000","last_became_peered":"2024-04-28T13:33:34.006561+0000","last_unstale":"2024-04-28T13:33:34.012182+0000","last_undegraded":"2024-04-28T13:33:34.012182+0000","last_fullsized":"2024-04-28T13:33:34.012182+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999932+0000","last_change":"2024-04-28T13:33:32.991367+0000","last_active":"2024-04-28T13:33:34.999932+0000","last_peered":"2024-04-28T13:33:34.999932+0000","last_clean":"2024-04-28T13:33:34.999932+0000","last_became_active":"2024-04-28T13:33:32.991060+0000","last_became_peered":"2024-04-28T13:33:32.991060+0000","last_unstale":"2024-04-28T13:33:34.999932+0000","last_undegraded":"2024-04-28T13:33:34.999932+0000","last_fullsized":"2024-04-28T13:33:34.999932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:00:16.144423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.001266+0000","last_change":"2024-04-28T13:33:33.996658+0000","last_active":"2024-04-28T13:33:34.001266+0000","last_peered":"2024-04-28T13:33:34.001266+0000","last_clean":"2024-04-28T13:33:34.001266+0000","last_became_active":"2024-04-28T13:33:33.996260+0000","last_became_peered":"2024-04-28T13:33:33.996260+0000","last_unstale":"2024-04-28T13:33:34.001266+0000","last_undegraded":"2024-04-28T13:33:34.001266+0000","last_fullsized":"2024-04-28T13:33:34.001266+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:54:02.364417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.021953+0000","last_change":"2024-04-28T13:33:32.990968+0000","last_active":"2024-04-28T13:33:34.021953+0000","last_peered":"2024-04-28T13:33:34.021953+0000","last_clean":"2024-04-28T13:33:34.021953+0000","last_became_active":"2024-04-28T13:33:32.990718+0000","last_became_peered":"2024-04-28T13:33:32.990718+0000","last_unstale":"2024-04-28T13:33:34.021953+0000","last_undegraded":"2024-04-28T13:33:34.021953+0000","last_fullsized":"2024-04-28T13:33:34.021953+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:03.358943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.006429+0000","last_change":"2024-04-28T13:33:34.003961+0000","last_active":"2024-04-28T13:33:34.006429+0000","last_peered":"2024-04-28T13:33:34.006429+0000","last_clean":"2024-04-28T13:33:34.006429+0000","last_became_active":"2024-04-28T13:33:34.003599+0000","last_became_peered":"2024-04-28T13:33:34.003599+0000","last_unstale":"2024-04-28T13:33:34.006429+0000","last_undegraded":"2024-04-28T13:33:34.006429+0000","last_fullsized":"2024-04-28T13:33:34.006429+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:12.546153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.989344+0000","last_change":"2024-04-28T13:33:33.000521+0000","last_active":"2024-04-28T13:33:33.989344+0000","last_peered":"2024-04-28T13:33:33.989344+0000","last_clean":"2024-04-28T13:33:33.989344+0000","last_became_active":"2024-04-28T13:33:32.998867+0000","last_became_peered":"2024-04-28T13:33:32.998867+0000","last_unstale":"2024-04-28T13:33:33.989344+0000","last_undegraded":"2024-04-28T13:33:33.989344+0000","last_fullsized":"2024-04-28T13:33:33.989344+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:43.562896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.015834+0000","last_change":"2024-04-28T13:33:34.006368+0000","last_active":"2024-04-28T13:33:34.015834+0000","last_peered":"2024-04-28T13:33:34.015834+0000","last_clean":"2024-04-28T13:33:34.015834+0000","last_became_active":"2024-04-28T13:33:34.002579+0000","last_became_peered":"2024-04-28T13:33:34.002579+0000","last_unstale":"2024-04-28T13:33:34.015834+0000","last_undegraded":"2024-04-28T13:33:34.015834+0000","last_fullsized":"2024-04-28T13:33:34.015834+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:01:13.143511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.986501+0000","last_change":"2024-04-28T13:33:32.991648+0000","last_active":"2024-04-28T13:33:33.986501+0000","last_peered":"2024-04-28T13:33:33.986501+0000","last_clean":"2024-04-28T13:33:33.986501+0000","last_became_active":"2024-04-28T13:33:32.991369+0000","last_became_peered":"2024-04-28T13:33:32.991369+0000","last_unstale":"2024-04-28T13:33:33.986501+0000","last_undegraded":"2024-04-28T13:33:33.986501+0000","last_fullsized":"2024-04-28T13:33:33.986501+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:43.562896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.013499+0000","last_change":"2024-04-28T13:33:34.010407+0000","last_active":"2024-04-28T13:33:34.013499+0000","last_peered":"2024-04-28T13:33:34.013499+0000","last_clean":"2024-04-28T13:33:34.013499+0000","last_became_active":"2024-04-28T13:33:34.010077+0000","last_became_peered":"2024-04-28T13:33:34.010077+0000","last_unstale":"2024-04-28T13:33:34.013499+0000","last_undegraded":"2024-04-28T13:33:34.013499+0000","last_fullsized":"2024-04-28T13:33:34.013499+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:01:33.789936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.018534+0000","last_change":"2024-04-28T13:33:32.989587+0000","last_active":"2024-04-28T13:33:34.018534+0000","last_peered":"2024-04-28T13:33:34.018534+0000","last_clean":"2024-04-28T13:33:34.018534+0000","last_became_active":"2024-04-28T13:33:32.988579+0000","last_became_peered":"2024-04-28T13:33:32.988579+0000","last_unstale":"2024-04-28T13:33:34.018534+0000","last_undegraded":"2024-04-28T13:33:34.018534+0000","last_fullsized":"2024-04-28T13:33:34.018534+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:35.233850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999875+0000","last_change":"2024-04-28T13:33:34.012322+0000","last_active":"2024-04-28T13:33:34.999875+0000","last_peered":"2024-04-28T13:33:34.999875+0000","last_clean":"2024-04-28T13:33:34.999875+0000","last_became_active":"2024-04-28T13:33:34.012112+0000","last_became_peered":"2024-04-28T13:33:34.012112+0000","last_unstale":"2024-04-28T13:33:34.999875+0000","last_undegraded":"2024-04-28T13:33:34.999875+0000","last_fullsized":"2024-04-28T13:33:34.999875+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:10.015862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.000018+0000","last_change":"2024-04-28T13:33:32.990958+0000","last_active":"2024-04-28T13:33:35.000018+0000","last_peered":"2024-04-28T13:33:35.000018+0000","last_clean":"2024-04-28T13:33:35.000018+0000","last_became_active":"2024-04-28T13:33:32.990659+0000","last_became_peered":"2024-04-28T13:33:32.990659+0000","last_unstale":"2024-04-28T13:33:35.000018+0000","last_undegraded":"2024-04-28T13:33:35.000018+0000","last_fullsized":"2024-04-28T13:33:35.000018+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:24:34.028457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014446+0000","last_change":"2024-04-28T13:33:34.005832+0000","last_active":"2024-04-28T13:33:34.014446+0000","last_peered":"2024-04-28T13:33:34.014446+0000","last_clean":"2024-04-28T13:33:34.014446+0000","last_became_active":"2024-04-28T13:33:34.005364+0000","last_became_peered":"2024-04-28T13:33:34.005364+0000","last_unstale":"2024-04-28T13:33:34.014446+0000","last_undegraded":"2024-04-28T13:33:34.014446+0000","last_fullsized":"2024-04-28T13:33:34.014446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:35.969568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.000007+0000","last_change":"2024-04-28T13:33:32.995338+0000","last_active":"2024-04-28T13:33:35.000007+0000","last_peered":"2024-04-28T13:33:35.000007+0000","last_clean":"2024-04-28T13:33:35.000007+0000","last_became_active":"2024-04-28T13:33:32.994798+0000","last_became_peered":"2024-04-28T13:33:32.994798+0000","last_unstale":"2024-04-28T13:33:35.000007+0000","last_undegraded":"2024-04-28T13:33:35.000007+0000","last_fullsized":"2024-04-28T13:33:35.000007+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:29.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.003779+0000","last_change":"2024-04-28T13:33:34.002242+0000","last_active":"2024-04-28T13:33:34.003779+0000","last_peered":"2024-04-28T13:33:34.003779+0000","last_clean":"2024-04-28T13:33:34.003779+0000","last_became_active":"2024-04-28T13:33:34.001893+0000","last_became_peered":"2024-04-28T13:33:34.001893+0000","last_unstale":"2024-04-28T13:33:34.003779+0000","last_undegraded":"2024-04-28T13:33:34.003779+0000","last_fullsized":"2024-04-28T13:33:34.003779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:03:14.657009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001343+0000","last_change":"2024-04-28T13:33:32.991441+0000","last_active":"2024-04-28T13:33:35.001343+0000","last_peered":"2024-04-28T13:33:35.001343+0000","last_clean":"2024-04-28T13:33:35.001343+0000","last_became_active":"2024-04-28T13:33:32.990745+0000","last_became_peered":"2024-04-28T13:33:32.990745+0000","last_unstale":"2024-04-28T13:33:35.001343+0000","last_undegraded":"2024-04-28T13:33:35.001343+0000","last_fullsized":"2024-04-28T13:33:35.001343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:30.656667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.005445+0000","last_change":"2024-04-28T13:33:34.003094+0000","last_active":"2024-04-28T13:33:34.005445+0000","last_peered":"2024-04-28T13:33:34.005445+0000","last_clean":"2024-04-28T13:33:34.005445+0000","last_became_active":"2024-04-28T13:33:34.002228+0000","last_became_peered":"2024-04-28T13:33:34.002228+0000","last_unstale":"2024-04-28T13:33:34.005445+0000","last_undegraded":"2024-04-28T13:33:34.005445+0000","last_fullsized":"2024-04-28T13:33:34.005445+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:27:53.804713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.988463+0000","last_change":"2024-04-28T13:33:33.003720+0000","last_active":"2024-04-28T13:33:33.988463+0000","last_peered":"2024-04-28T13:33:33.988463+0000","last_clean":"2024-04-28T13:33:33.988463+0000","last_became_active":"2024-04-28T13:33:33.003389+0000","last_became_peered":"2024-04-28T13:33:33.003389+0000","last_unstale":"2024-04-28T13:33:33.988463+0000","last_undegraded":"2024-04-28T13:33:33.988463+0000","last_fullsized":"2024-04-28T13:33:33.988463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:18.814381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.015707+0000","last_change":"2024-04-28T13:33:34.006579+0000","last_active":"2024-04-28T13:33:34.015707+0000","last_peered":"2024-04-28T13:33:34.015707+0000","last_clean":"2024-04-28T13:33:34.015707+0000","last_became_active":"2024-04-28T13:33:34.006282+0000","last_became_peered":"2024-04-28T13:33:34.006282+0000","last_unstale":"2024-04-28T13:33:34.015707+0000","last_undegraded":"2024-04-28T13:33:34.015707+0000","last_fullsized":"2024-04-28T13:33:34.015707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:09:39.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.016492+0000","last_change":"2024-04-28T13:33:32.999347+0000","last_active":"2024-04-28T13:33:34.016492+0000","last_peered":"2024-04-28T13:33:34.016492+0000","last_clean":"2024-04-28T13:33:34.016492+0000","last_became_active":"2024-04-28T13:33:32.998923+0000","last_became_peered":"2024-04-28T13:33:32.998923+0000","last_unstale":"2024-04-28T13:33:34.016492+0000","last_undegraded":"2024-04-28T13:33:34.016492+0000","last_fullsized":"2024-04-28T13:33:34.016492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:46.522691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.018494+0000","last_change":"2024-04-28T13:33:34.011875+0000","last_active":"2024-04-28T13:33:34.018494+0000","last_peered":"2024-04-28T13:33:34.018494+0000","last_clean":"2024-04-28T13:33:34.018494+0000","last_became_active":"2024-04-28T13:33:34.011271+0000","last_became_peered":"2024-04-28T13:33:34.011271+0000","last_unstale":"2024-04-28T13:33:34.018494+0000","last_undegraded":"2024-04-28T13:33:34.018494+0000","last_fullsized":"2024-04-28T13:33:34.018494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:42.082730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.000983+0000","last_change":"2024-04-28T13:33:33.002216+0000","last_active":"2024-04-28T13:33:35.000983+0000","last_peered":"2024-04-28T13:33:35.000983+0000","last_clean":"2024-04-28T13:33:35.000983+0000","last_became_active":"2024-04-28T13:33:33.000335+0000","last_became_peered":"2024-04-28T13:33:33.000335+0000","last_unstale":"2024-04-28T13:33:35.000983+0000","last_undegraded":"2024-04-28T13:33:35.000983+0000","last_fullsized":"2024-04-28T13:33:35.000983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:34.955026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999276+0000","last_change":"2024-04-28T13:33:34.000104+0000","last_active":"2024-04-28T13:33:34.999276+0000","last_peered":"2024-04-28T13:33:34.999276+0000","last_clean":"2024-04-28T13:33:34.999276+0000","last_became_active":"2024-04-28T13:33:33.999839+0000","last_became_peered":"2024-04-28T13:33:33.999839+0000","last_unstale":"2024-04-28T13:33:34.999276+0000","last_undegraded":"2024-04-28T13:33:34.999276+0000","last_fullsized":"2024-04-28T13:33:34.999276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:03:14.657009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.982243+0000","last_change":"2024-04-28T13:33:32.992149+0000","last_active":"2024-04-28T13:33:33.982243+0000","last_peered":"2024-04-28T13:33:33.982243+0000","last_clean":"2024-04-28T13:33:33.982243+0000","last_became_active":"2024-04-28T13:33:32.991800+0000","last_became_peered":"2024-04-28T13:33:32.991800+0000","last_unstale":"2024-04-28T13:33:33.982243+0000","last_undegraded":"2024-04-28T13:33:33.982243+0000","last_fullsized":"2024-04-28T13:33:33.982243+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:56:39.708282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.016566+0000","last_change":"2024-04-28T13:33:34.011133+0000","last_active":"2024-04-28T13:33:34.016566+0000","last_peered":"2024-04-28T13:33:34.016566+0000","last_clean":"2024-04-28T13:33:34.016566+0000","last_became_active":"2024-04-28T13:33:34.010896+0000","last_became_peered":"2024-04-28T13:33:34.010896+0000","last_unstale":"2024-04-28T13:33:34.016566+0000","last_undegraded":"2024-04-28T13:33:34.016566+0000","last_fullsized":"2024-04-28T13:33:34.016566+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:19.235757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.988279+0000","last_change":"2024-04-28T13:33:33.000859+0000","last_active":"2024-04-28T13:33:33.988279+0000","last_peered":"2024-04-28T13:33:33.988279+0000","last_clean":"2024-04-28T13:33:33.988279+0000","last_became_active":"2024-04-28T13:33:33.000095+0000","last_became_peered":"2024-04-28T13:33:33.000095+0000","last_unstale":"2024-04-28T13:33:33.988279+0000","last_undegraded":"2024-04-28T13:33:33.988279+0000","last_fullsized":"2024-04-28T13:33:33.988279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:18.221214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.010743+0000","last_change":"2024-04-28T13:33:34.008405+0000","last_active":"2024-04-28T13:33:34.010743+0000","last_peered":"2024-04-28T13:33:34.010743+0000","last_clean":"2024-04-28T13:33:34.010743+0000","last_became_active":"2024-04-28T13:33:34.007425+0000","last_became_peered":"2024-04-28T13:33:34.007425+0000","last_unstale":"2024-04-28T13:33:34.010743+0000","last_undegraded":"2024-04-28T13:33:34.010743+0000","last_fullsized":"2024-04-28T13:33:34.010743+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:10.015862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.985821+0000","last_change":"2024-04-28T13:33:32.998594+0000","last_active":"2024-04-28T13:33:33.985821+0000","last_peered":"2024-04-28T13:33:33.985821+0000","last_clean":"2024-04-28T13:33:33.985821+0000","last_became_active":"2024-04-28T13:33:32.998328+0000","last_became_peered":"2024-04-28T13:33:32.998328+0000","last_unstale":"2024-04-28T13:33:33.985821+0000","last_undegraded":"2024-04-28T13:33:33.985821+0000","last_fullsized":"2024-04-28T13:33:33.985821+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:18.221214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.006713+0000","last_change":"2024-04-28T13:33:33.998134+0000","last_active":"2024-04-28T13:33:34.006713+0000","last_peered":"2024-04-28T13:33:34.006713+0000","last_clean":"2024-04-28T13:33:34.006713+0000","last_became_active":"2024-04-28T13:33:33.997834+0000","last_became_peered":"2024-04-28T13:33:33.997834+0000","last_unstale":"2024-04-28T13:33:34.006713+0000","last_undegraded":"2024-04-28T13:33:34.006713+0000","last_fullsized":"2024-04-28T13:33:34.006713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:52:46.527475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.991330+0000","last_change":"2024-04-28T13:33:33.005737+0000","last_active":"2024-04-28T13:33:33.991330+0000","last_peered":"2024-04-28T13:33:33.991330+0000","last_clean":"2024-04-28T13:33:33.991330+0000","last_became_active":"2024-04-28T13:33:33.005179+0000","last_became_peered":"2024-04-28T13:33:33.005179+0000","last_unstale":"2024-04-28T13:33:33.991330+0000","last_undegraded":"2024-04-28T13:33:33.991330+0000","last_fullsized":"2024-04-28T13:33:33.991330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:45.657949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014333+0000","last_change":"2024-04-28T13:33:34.011725+0000","last_active":"2024-04-28T13:33:34.014333+0000","last_peered":"2024-04-28T13:33:34.014333+0000","last_clean":"2024-04-28T13:33:34.014333+0000","last_became_active":"2024-04-28T13:33:34.011422+0000","last_became_peered":"2024-04-28T13:33:34.011422+0000","last_unstale":"2024-04-28T13:33:34.014333+0000","last_undegraded":"2024-04-28T13:33:34.014333+0000","last_fullsized":"2024-04-28T13:33:34.014333+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:01:13.143511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.007375+0000","last_change":"2024-04-28T13:33:32.997338+0000","last_active":"2024-04-28T13:33:34.007375+0000","last_peered":"2024-04-28T13:33:34.007375+0000","last_clean":"2024-04-28T13:33:34.007375+0000","last_became_active":"2024-04-28T13:33:32.996934+0000","last_became_peered":"2024-04-28T13:33:32.996934+0000","last_unstale":"2024-04-28T13:33:34.007375+0000","last_undegraded":"2024-04-28T13:33:34.007375+0000","last_fullsized":"2024-04-28T13:33:34.007375+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:29.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.021671+0000","last_change":"2024-04-28T13:33:34.020082+0000","last_active":"2024-04-28T13:33:34.021671+0000","last_peered":"2024-04-28T13:33:34.021671+0000","last_clean":"2024-04-28T13:33:34.021671+0000","last_became_active":"2024-04-28T13:33:34.018922+0000","last_became_peered":"2024-04-28T13:33:34.018922+0000","last_unstale":"2024-04-28T13:33:34.021671+0000","last_undegraded":"2024-04-28T13:33:34.021671+0000","last_fullsized":"2024-04-28T13:33:34.021671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:28.548233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.989409+0000","last_change":"2024-04-28T13:33:33.000626+0000","last_active":"2024-04-28T13:33:33.989409+0000","last_peered":"2024-04-28T13:33:33.989409+0000","last_clean":"2024-04-28T13:33:33.989409+0000","last_became_active":"2024-04-28T13:33:33.000049+0000","last_became_peered":"2024-04-28T13:33:33.000049+0000","last_unstale":"2024-04-28T13:33:33.989409+0000","last_undegraded":"2024-04-28T13:33:33.989409+0000","last_fullsized":"2024-04-28T13:33:33.989409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:18.814381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.013632+0000","last_change":"2024-04-28T13:33:34.004230+0000","last_active":"2024-04-28T13:33:34.013632+0000","last_peered":"2024-04-28T13:33:34.013632+0000","last_clean":"2024-04-28T13:33:34.013632+0000","last_became_active":"2024-04-28T13:33:34.003940+0000","last_became_peered":"2024-04-28T13:33:34.003940+0000","last_unstale":"2024-04-28T13:33:34.013632+0000","last_undegraded":"2024-04-28T13:33:34.013632+0000","last_fullsized":"2024-04-28T13:33:34.013632+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:01:33.789936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999548+0000","last_change":"2024-04-28T13:33:32.991869+0000","last_active":"2024-04-28T13:33:34.999548+0000","last_peered":"2024-04-28T13:33:34.999548+0000","last_clean":"2024-04-28T13:33:34.999548+0000","last_became_active":"2024-04-28T13:33:32.991435+0000","last_became_peered":"2024-04-28T13:33:32.991435+0000","last_unstale":"2024-04-28T13:33:34.999548+0000","last_undegraded":"2024-04-28T13:33:34.999548+0000","last_fullsized":"2024-04-28T13:33:34.999548+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:41.068187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.000896+0000","last_change":"2024-04-28T13:33:34.004178+0000","last_active":"2024-04-28T13:33:35.000896+0000","last_peered":"2024-04-28T13:33:35.000896+0000","last_clean":"2024-04-28T13:33:35.000896+0000","last_became_active":"2024-04-28T13:33:34.002092+0000","last_became_peered":"2024-04-28T13:33:34.002092+0000","last_unstale":"2024-04-28T13:33:35.000896+0000","last_undegraded":"2024-04-28T13:33:35.000896+0000","last_fullsized":"2024-04-28T13:33:35.000896+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:03.446736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.008707+0000","last_change":"2024-04-28T13:33:33.003796+0000","last_active":"2024-04-28T13:33:34.008707+0000","last_peered":"2024-04-28T13:33:34.008707+0000","last_clean":"2024-04-28T13:33:34.008707+0000","last_became_active":"2024-04-28T13:33:33.003374+0000","last_became_peered":"2024-04-28T13:33:33.003374+0000","last_unstale":"2024-04-28T13:33:34.008707+0000","last_undegraded":"2024-04-28T13:33:34.008707+0000","last_fullsized":"2024-04-28T13:33:34.008707+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:11.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.999264+0000","last_change":"2024-04-28T13:33:34.001551+0000","last_active":"2024-04-28T13:33:34.999264+0000","last_peered":"2024-04-28T13:33:34.999264+0000","last_clean":"2024-04-28T13:33:34.999264+0000","last_became_active":"2024-04-28T13:33:33.999963+0000","last_became_peered":"2024-04-28T13:33:33.999963+0000","last_unstale":"2024-04-28T13:33:34.999264+0000","last_undegraded":"2024-04-28T13:33:34.999264+0000","last_fullsized":"2024-04-28T13:33:34.999264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:19.828924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.983634+0000","last_change":"2024-04-28T13:33:32.991033+0000","last_active":"2024-04-28T13:33:33.983634+0000","last_peered":"2024-04-28T13:33:33.983634+0000","last_clean":"2024-04-28T13:33:33.983634+0000","last_became_active":"2024-04-28T13:33:32.990726+0000","last_became_peered":"2024-04-28T13:33:32.990726+0000","last_unstale":"2024-04-28T13:33:33.983634+0000","last_undegraded":"2024-04-28T13:33:33.983634+0000","last_fullsized":"2024-04-28T13:33:33.983634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:09.001319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:34.014662+0000","last_change":"2024-04-28T13:33:34.005421+0000","last_active":"2024-04-28T13:33:34.014662+0000","last_peered":"2024-04-28T13:33:34.014662+0000","last_clean":"2024-04-28T13:33:34.014662+0000","last_became_active":"2024-04-28T13:33:34.004776+0000","last_became_peered":"2024-04-28T13:33:34.004776+0000","last_unstale":"2024-04-28T13:33:34.014662+0000","last_undegraded":"2024-04-28T13:33:34.014662+0000","last_fullsized":"2024-04-28T13:33:34.014662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:42.082730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-28T13:33:35.001892+0000","last_change":"2024-04-28T13:33:32.991462+0000","last_active":"2024-04-28T13:33:35.001892+0000","last_peered":"2024-04-28T13:33:35.001892+0000","last_clean":"2024-04-28T13:33:35.001892+0000","last_became_active":"2024-04-28T13:33:32.991149+0000","last_became_peered":"2024-04-28T13:33:32.991149+0000","last_unstale":"2024-04-28T13:33:35.001892+0000","last_undegraded":"2024-04-28T13:33:35.001892+0000","last_fullsized":"2024-04-28T13:33:35.001892+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:41.068187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":213,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-28T13:33:33.988364+0000","last_change":"2024-04-28T13:31:11.831538+0000","last_active":"2024-04-28T13:33:33.988364+0000","last_peered":"2024-04-28T13:33:33.988364+0000","last_clean":"2024-04-28T13:33:33.988364+0000","last_became_active":"2024-04-28T13:31:11.828959+0000","last_became_peered":"2024-04-28T13:31:11.828959+0000","last_unstale":"2024-04-28T13:33:33.988364+0000","last_undegraded":"2024-04-28T13:33:33.988364+0000","last_fullsized":"2024-04-28T13:33:33.988364+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_clean_scrub_stamp":"2024-04-28T13:31:10.444388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:25:27.078911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1114112,"data_stored":800469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":524288,"data_stored":210683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":210081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":217923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":210649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":217945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":210081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1114112,"data_stored":800413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T13:33:38.937 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-28T13:33:38.938 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-28T13:33:39.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.192+0000 7ff8fa000640 1 Processor -- start 2024-04-28T13:33:39.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.192+0000 7ff8fa000640 1 -- start start 2024-04-28T13:33:39.193 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.194+0000 7ff8fa000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff8f4146690 0x7ff8f406ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:39.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.194+0000 7ff8fa000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f406f1c0 0x7ff8f4073130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:39.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.194+0000 7ff8fa000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f4073f90 con 0x7ff8f4073670 2024-04-28T13:33:39.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.194+0000 7ff8fa000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f406f620 con 0x7ff8f4146690 2024-04-28T13:33:39.194 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.194+0000 7ff8fa000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f4073ac0 con 0x7ff8f406f1c0 2024-04-28T13:33:39.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.194+0000 7ff8f8c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff8f4146690 0x7ff8f406ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:39.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.194+0000 7ff8f8c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff8f4146690 0x7ff8f406ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59782/0 (socket says 172.21.15.196:59782) 2024-04-28T13:33:39.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.194+0000 7ff8f8c00640 1 -- 172.21.15.196:0/2632229710 learned_addr learned my addr 172.21.15.196:0/2632229710 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:39.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.195+0000 7ff8f8c00640 1 -- 172.21.15.196:0/2632229710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f406f1c0 msgr2=0x7ff8f4073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:39.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.195+0000 7ff8f3e00640 1 --2- 172.21.15.196:0/2632229710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f406f1c0 0x7ff8f4073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:39.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.195+0000 7ff8f8c00640 1 --2- 172.21.15.196:0/2632229710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f406f1c0 0x7ff8f4073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.195+0000 7ff8f8c00640 1 -- 172.21.15.196:0/2632229710 >> v1:172.21.15.203:6789/0 conn(0x7ff8f4073670 legacy=0x7ff8f40ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:39.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.195+0000 7ff8f8c00640 1 -- 172.21.15.196:0/2632229710 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f406f840 con 0x7ff8f4146690 2024-04-28T13:33:39.195 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.195+0000 7ff8f3e00640 1 --2- 172.21.15.196:0/2632229710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f406f1c0 0x7ff8f4073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:39.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.196+0000 7ff8f8c00640 1 --2- 172.21.15.196:0/2632229710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f4146690 0x7ff8f406ec80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40098a0 tx=0x7ff8e402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9586ed9044940231 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:39.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.196+0000 7ff8f3400640 1 -- 172.21.15.196:0/2632229710 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4037070 con 0x7ff8f4146690 2024-04-28T13:33:39.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.196+0000 7ff8f3400640 1 -- 172.21.15.196:0/2632229710 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e4032440 con 0x7ff8f4146690 2024-04-28T13:33:39.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.196+0000 7ff8f3400640 1 -- 172.21.15.196:0/2632229710 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4032810 con 0x7ff8f4146690 2024-04-28T13:33:39.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.197+0000 7ff8fa000640 1 -- 172.21.15.196:0/2632229710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f4146690 msgr2=0x7ff8f406ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:39.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.197+0000 7ff8fa000640 1 --2- 172.21.15.196:0/2632229710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f4146690 0x7ff8f406ec80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40098a0 tx=0x7ff8e402a2a0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.197+0000 7ff8fa000640 1 -- 172.21.15.196:0/2632229710 shutdown_connections 2024-04-28T13:33:39.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.197+0000 7ff8fa000640 1 --2- 172.21.15.196:0/2632229710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f406f1c0 0x7ff8f4073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.197+0000 7ff8fa000640 1 --2- 172.21.15.196:0/2632229710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f4146690 0x7ff8f406ec80 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.197+0000 7ff8fa000640 1 -- 172.21.15.196:0/2632229710 >> 172.21.15.196:0/2632229710 conn(0x7ff8f406c2a0 msgr2=0x7ff8f406c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:39.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.197+0000 7ff8fa000640 1 -- 172.21.15.196:0/2632229710 shutdown_connections 2024-04-28T13:33:39.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.197+0000 7ff8fa000640 1 -- 172.21.15.196:0/2632229710 wait complete. 2024-04-28T13:33:39.199 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.199+0000 7ff8fa000640 1 Processor -- start 2024-04-28T13:33:39.199 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.199+0000 7ff8fa000640 1 -- start start 2024-04-28T13:33:39.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.200+0000 7ff8fa000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8f406f1c0 0x7ff8f41421f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:39.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.200+0000 7ff8fa000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f4073670 0x7ff8f41e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:39.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.200+0000 7ff8fa000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f4146690 0x7ff8f41e6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:39.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.200+0000 7ff8fa000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f415d990 con 0x7ff8f4146690 2024-04-28T13:33:39.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.200+0000 7ff8fa000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f415d6b0 con 0x7ff8f406f1c0 2024-04-28T13:33:39.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.200+0000 7ff8fa000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f415dc70 con 0x7ff8f4073670 2024-04-28T13:33:39.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.200+0000 7ff8f9600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f4146690 0x7ff8f41e6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:39.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.200+0000 7ff8f9600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f4146690 0x7ff8f41e6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59786/0 (socket says 172.21.15.196:59786) 2024-04-28T13:33:39.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.200+0000 7ff8f9600640 1 -- 172.21.15.196:0/1727071491 learned_addr learned my addr 172.21.15.196:0/1727071491 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:39.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.201+0000 7ff8f8c00640 1 --2- 172.21.15.196:0/1727071491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8f406f1c0 0x7ff8f41421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:39.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.201+0000 7ff8f3e00640 1 --2- 172.21.15.196:0/1727071491 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f4073670 0x7ff8f41e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:39.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.201+0000 7ff8f9600640 1 -- 172.21.15.196:0/1727071491 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f4073670 msgr2=0x7ff8f41e5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:39.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.201+0000 7ff8f9600640 1 --2- 172.21.15.196:0/1727071491 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f4073670 0x7ff8f41e5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.201+0000 7ff8f9600640 1 -- 172.21.15.196:0/1727071491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8f406f1c0 msgr2=0x7ff8f41421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:39.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.201+0000 7ff8f9600640 1 --2- 172.21.15.196:0/1727071491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8f406f1c0 0x7ff8f41421f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.201+0000 7ff8f9600640 1 -- 172.21.15.196:0/1727071491 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f41e7a60 con 0x7ff8f4146690 2024-04-28T13:33:39.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.201+0000 7ff8f9600640 1 --2- 172.21.15.196:0/1727071491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f4146690 0x7ff8f41e6390 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00c7a0 tx=0x7ff8ec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:39.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.201+0000 7ff8f0c00640 1 -- 172.21.15.196:0/1727071491 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec003b70 con 0x7ff8f4146690 2024-04-28T13:33:39.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.201+0000 7ff8f0c00640 1 -- 172.21.15.196:0/1727071491 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8ec003d30 con 0x7ff8f4146690 2024-04-28T13:33:39.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.201+0000 7ff8f0c00640 1 -- 172.21.15.196:0/1727071491 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec00aa90 con 0x7ff8f4146690 2024-04-28T13:33:39.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.202+0000 7ff8fa000640 1 -- 172.21.15.196:0/1727071491 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f4142730 con 0x7ff8f4146690 2024-04-28T13:33:39.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.202+0000 7ff8fa000640 1 -- 172.21.15.196:0/1727071491 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f41e6ca0 con 0x7ff8f4146690 2024-04-28T13:33:39.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.203+0000 7ff8fa000640 1 -- 172.21.15.196:0/1727071491 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f406a290 con 0x7ff8f4146690 2024-04-28T13:33:39.203 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.204+0000 7ff8f0c00640 1 -- 172.21.15.196:0/1727071491 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff8ec00ac30 con 0x7ff8f4146690 2024-04-28T13:33:39.204 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.205+0000 7ff8f0c00640 1 --2- 172.21.15.196:0/1727071491 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff8d806ca00 0x7ff8d806eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:39.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.205+0000 7ff8f0c00640 1 -- 172.21.15.196:0/1727071491 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7ff8ec08ba50 con 0x7ff8f4146690 2024-04-28T13:33:39.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.205+0000 7ff8f8c00640 1 --2- 172.21.15.196:0/1727071491 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff8d806ca00 0x7ff8d806eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:39.206 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.206+0000 7ff8f8c00640 1 --2- 172.21.15.196:0/1727071491 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff8d806ca00 0x7ff8d806eec0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4034000 tx=0x7ff8e402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:39.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.209+0000 7ff8f0c00640 1 -- 172.21.15.196:0/1727071491 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff8ec059e10 con 0x7ff8f4146690 2024-04-28T13:33:39.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.478+0000 7ff8fa000640 1 -- 172.21.15.196:0/1727071491 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff8f4142c40 con 0x7ff8f4146690 2024-04-28T13:33:39.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.482+0000 7ff8f0c00640 1 -- 172.21.15.196:0/1727071491 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7ff8ec059710 con 0x7ff8f4146690 2024-04-28T13:33:39.483 INFO:teuthology.orchestra.run.smithi196.stdout:[client.0] 2024-04-28T13:33:39.483 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBlTy5m8movMBAAnKfzHF0pAkX1F/dZiB0qBg== 2024-04-28T13:33:39.483 INFO:teuthology.orchestra.run.smithi196.stdout: caps mds = "allow" 2024-04-28T13:33:39.483 INFO:teuthology.orchestra.run.smithi196.stdout: caps mon = "allow r" 2024-04-28T13:33:39.483 INFO:teuthology.orchestra.run.smithi196.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-28T13:33:39.483 INFO:teuthology.orchestra.run.smithi196.stderr:updated caps for client.0 2024-04-28T13:33:39.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.486+0000 7ff8fa000640 1 -- 172.21.15.196:0/1727071491 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff8d806ca00 msgr2=0x7ff8d806eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:39.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.486+0000 7ff8fa000640 1 --2- 172.21.15.196:0/1727071491 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff8d806ca00 0x7ff8d806eec0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4034000 tx=0x7ff8e402e000 comp rx=0 tx=0).stop 2024-04-28T13:33:39.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.486+0000 7ff8fa000640 1 -- 172.21.15.196:0/1727071491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f4146690 msgr2=0x7ff8f41e6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:39.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.486+0000 7ff8fa000640 1 --2- 172.21.15.196:0/1727071491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f4146690 0x7ff8f41e6390 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00c7a0 tx=0x7ff8ec00cc60 comp rx=0 tx=0).stop 2024-04-28T13:33:39.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.486+0000 7ff8fa000640 1 -- 172.21.15.196:0/1727071491 shutdown_connections 2024-04-28T13:33:39.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.486+0000 7ff8fa000640 1 --2- 172.21.15.196:0/1727071491 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff8d806ca00 0x7ff8d806eec0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.486+0000 7ff8fa000640 1 --2- 172.21.15.196:0/1727071491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f4146690 0x7ff8f41e6390 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.486+0000 7ff8fa000640 1 --2- 172.21.15.196:0/1727071491 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f4073670 0x7ff8f41e5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.486+0000 7ff8fa000640 1 --2- 172.21.15.196:0/1727071491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8f406f1c0 0x7ff8f41421f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.486+0000 7ff8fa000640 1 -- 172.21.15.196:0/1727071491 >> 172.21.15.196:0/1727071491 conn(0x7ff8f406c2a0 msgr2=0x7ff8f4072690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:39.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.486+0000 7ff8fa000640 1 -- 172.21.15.196:0/1727071491 shutdown_connections 2024-04-28T13:33:39.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.486+0000 7ff8fa000640 1 -- 172.21.15.196:0/1727071491 wait complete. 2024-04-28T13:33:39.504 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:33:39.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.695+0000 7fe8dfe00640 1 Processor -- start 2024-04-28T13:33:39.695 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.695+0000 7fe8dfe00640 1 -- start start 2024-04-28T13:33:39.697 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.696+0000 7fe8dfe00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe8e00b5ee0 0x7fe8e00b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.696+0000 7fe8dfe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe8e00d1760 0x7fe8e006df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.697+0000 7fe8dfe00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e015e380 con 0x7fe8e009ec80 2024-04-28T13:33:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.697+0000 7fe8dfe00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e00b4b20 con 0x7fe8e00d1760 2024-04-28T13:33:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.697+0000 7fe8dfe00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e00b3f30 con 0x7fe8e00b5ee0 2024-04-28T13:33:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.697+0000 7fe8df400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe8e00d1760 0x7fe8e006df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.697+0000 7fe8df400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe8e00d1760 0x7fe8e006df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59794/0 (socket says 172.21.15.196:59794) 2024-04-28T13:33:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.697+0000 7fe8df400640 1 -- 172.21.15.196:0/1869891619 learned_addr learned my addr 172.21.15.196:0/1869891619 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.697+0000 7fe8d6000640 1 --1- 172.21.15.196:0/1869891619 >> v1:172.21.15.196:6789/0 conn(0x7fe8e009ec80 0x7fe8e00d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:38242/0 (socket says 172.21.15.196:38242) 2024-04-28T13:33:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.697+0000 7fe8dea00640 1 --2- 172.21.15.196:0/1869891619 >> v2:172.21.15.203:3300/0 conn(0x7fe8e00b5ee0 0x7fe8e00b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.697+0000 7fe8df400640 1 -- 172.21.15.196:0/1869891619 >> v2:172.21.15.203:3300/0 conn(0x7fe8e00b5ee0 msgr2=0x7fe8e00b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.697+0000 7fe8df400640 1 --2- 172.21.15.196:0/1869891619 >> v2:172.21.15.203:3300/0 conn(0x7fe8e00b5ee0 0x7fe8e00b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.697+0000 7fe8df400640 1 -- 172.21.15.196:0/1869891619 >> v1:172.21.15.196:6789/0 conn(0x7fe8e009ec80 legacy=0x7fe8e00d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:39.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.698+0000 7fe8df400640 1 -- 172.21.15.196:0/1869891619 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e00b41f0 con 0x7fe8e00d1760 2024-04-28T13:33:39.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.698+0000 7fe8de000640 1 -- 172.21.15.196:0/1869891619 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1154719004 0 0) 0x7fe8e015e380 con 0x7fe8e009ec80 2024-04-28T13:33:39.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.699+0000 7fe8df400640 1 --2- 172.21.15.196:0/1869891619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8e00d1760 0x7fe8e006df50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe8e006eda0 tx=0x7fe8d802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ccf8ee52a56929 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:39.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.699+0000 7fe8de000640 1 -- 172.21.15.196:0/1869891619 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8036070 con 0x7fe8e00d1760 2024-04-28T13:33:39.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.699+0000 7fe8de000640 1 -- 172.21.15.196:0/1869891619 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d8031600 con 0x7fe8e00d1760 2024-04-28T13:33:39.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.699+0000 7fe8de000640 1 -- 172.21.15.196:0/1869891619 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80319d0 con 0x7fe8e00d1760 2024-04-28T13:33:39.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.699+0000 7fe8dfe00640 1 -- 172.21.15.196:0/1869891619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8e00d1760 msgr2=0x7fe8e006df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:39.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.699+0000 7fe8dfe00640 1 --2- 172.21.15.196:0/1869891619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8e00d1760 0x7fe8e006df50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe8e006eda0 tx=0x7fe8d802a730 comp rx=0 tx=0).stop 2024-04-28T13:33:39.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.700+0000 7fe8dfe00640 1 -- 172.21.15.196:0/1869891619 shutdown_connections 2024-04-28T13:33:39.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.700+0000 7fe8dfe00640 1 --2- 172.21.15.196:0/1869891619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8e00d1760 0x7fe8e006df50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.700+0000 7fe8dfe00640 1 --2- 172.21.15.196:0/1869891619 >> v2:172.21.15.203:3300/0 conn(0x7fe8e00b5ee0 0x7fe8e00b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.700+0000 7fe8dfe00640 1 -- 172.21.15.196:0/1869891619 >> 172.21.15.196:0/1869891619 conn(0x7fe8e006c6c0 msgr2=0x7fe8e006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:39.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.700+0000 7fe8dfe00640 1 -- 172.21.15.196:0/1869891619 shutdown_connections 2024-04-28T13:33:39.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.700+0000 7fe8dfe00640 1 -- 172.21.15.196:0/1869891619 wait complete. 2024-04-28T13:33:39.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.701+0000 7fe8dfe00640 1 Processor -- start 2024-04-28T13:33:39.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.702+0000 7fe8dfe00640 1 -- start start 2024-04-28T13:33:39.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.702+0000 7fe8dfe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8e009ec80 0x7fe8e0157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:39.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.702+0000 7fe8dfe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8e00b5ee0 0x7fe8e01f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:39.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.703+0000 7fe8dfe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8e00d1760 0x7fe8e00b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:39.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.703+0000 7fe8dfe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e00b3c20 con 0x7fe8e00d1760 2024-04-28T13:33:39.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.703+0000 7fe8dfe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e00b47b0 con 0x7fe8e00b5ee0 2024-04-28T13:33:39.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.703+0000 7fe8dfe00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e00add70 con 0x7fe8e009ec80 2024-04-28T13:33:39.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.703+0000 7fe8df400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8e00d1760 0x7fe8e00b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:39.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.703+0000 7fe8df400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8e00d1760 0x7fe8e00b1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59800/0 (socket says 172.21.15.196:59800) 2024-04-28T13:33:39.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.703+0000 7fe8df400640 1 -- 172.21.15.196:0/1477971470 learned_addr learned my addr 172.21.15.196:0/1477971470 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:39.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.703+0000 7fe8df400640 1 -- 172.21.15.196:0/1477971470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8e009ec80 msgr2=0x7fe8e0157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:39.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.703+0000 7fe8df400640 1 --2- 172.21.15.196:0/1477971470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8e009ec80 0x7fe8e0157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.703+0000 7fe8d6000640 1 --2- 172.21.15.196:0/1477971470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8e00b5ee0 0x7fe8e01f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:39.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.703+0000 7fe8df400640 1 -- 172.21.15.196:0/1477971470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8e00b5ee0 msgr2=0x7fe8e01f31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:39.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.703+0000 7fe8df400640 1 --2- 172.21.15.196:0/1477971470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8e00b5ee0 0x7fe8e01f31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.703+0000 7fe8df400640 1 -- 172.21.15.196:0/1477971470 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e00b20c0 con 0x7fe8e00d1760 2024-04-28T13:33:39.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.704+0000 7fe8df400640 1 --2- 172.21.15.196:0/1477971470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8e00d1760 0x7fe8e00b1b80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe8d802ad20 tx=0x7fe8d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:39.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.704+0000 7fe8d7400640 1 -- 172.21.15.196:0/1477971470 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8036050 con 0x7fe8e00d1760 2024-04-28T13:33:39.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.704+0000 7fe8d7400640 1 -- 172.21.15.196:0/1477971470 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d8042070 con 0x7fe8e00d1760 2024-04-28T13:33:39.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.704+0000 7fe8d7400640 1 -- 172.21.15.196:0/1477971470 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8009510 con 0x7fe8e00d1760 2024-04-28T13:33:39.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.704+0000 7fe8dfe00640 1 -- 172.21.15.196:0/1477971470 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e0157a00 con 0x7fe8e00d1760 2024-04-28T13:33:39.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.704+0000 7fe8dfe00640 1 -- 172.21.15.196:0/1477971470 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e0157720 con 0x7fe8e00d1760 2024-04-28T13:33:39.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.705+0000 7fe8dfe00640 1 -- 172.21.15.196:0/1477971470 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e006a290 con 0x7fe8e00d1760 2024-04-28T13:33:39.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.706+0000 7fe8d7400640 1 -- 172.21.15.196:0/1477971470 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe8d8007bf0 con 0x7fe8e00d1760 2024-04-28T13:33:39.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.707+0000 7fe8d7400640 1 --2- 172.21.15.196:0/1477971470 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe8bc06c980 0x7fe8bc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:39.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.707+0000 7fe8d7400640 1 -- 172.21.15.196:0/1477971470 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fe8d80acba0 con 0x7fe8e00d1760 2024-04-28T13:33:39.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.708+0000 7fe8dea00640 1 --2- 172.21.15.196:0/1477971470 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe8bc06c980 0x7fe8bc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:39.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.709+0000 7fe8dea00640 1 --2- 172.21.15.196:0/1477971470 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe8bc06c980 0x7fe8bc06ee40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe8e015a840 tx=0x7fe8d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:39.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.711+0000 7fe8d7400640 1 -- 172.21.15.196:0/1477971470 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe8d807ae60 con 0x7fe8e00d1760 2024-04-28T13:33:39.931 INFO:tasks.ceph.mds.b.smithi203.stderr:2024-04-28T13:33:39.931+0000 7f81d8c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-28T13:33:39.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.948+0000 7fe8dfe00640 1 -- 172.21.15.196:0/1477971470 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8e006ee60 con 0x7fe8e00d1760 2024-04-28T13:33:39.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.949+0000 7fe8d7400640 1 -- 172.21.15.196:0/1477971470 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5193 (secure 0 0 0) 0x7fe8d807a760 con 0x7fe8e00d1760 2024-04-28T13:33:39.949 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:33:39.950 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2957566398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2957566398},{"type":"v1","addr":"172.21.15.203:6836","nonce":2957566398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5766,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1101263363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1101263363},{"type":"v1","addr":"172.21.15.196:6834","nonce":1101263363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1157781293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1157781293},{"type":"v1","addr":"172.21.15.196:6835","nonce":1157781293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:33:33.440220+0000","modified":"2024-04-28T13:33:36.491827+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5781},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5781":{"gid":5781,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/1525791340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1525791340},{"type":"v1","addr":"172.21.15.203:6837","nonce":1525791340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5781,"qdb_cluster":[5781]},"id":3}]} 2024-04-28T13:33:39.950 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 31 2024-04-28T13:33:39.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.952+0000 7fe8dfe00640 1 -- 172.21.15.196:0/1477971470 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe8bc06c980 msgr2=0x7fe8bc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:39.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.952+0000 7fe8dfe00640 1 --2- 172.21.15.196:0/1477971470 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe8bc06c980 0x7fe8bc06ee40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe8e015a840 tx=0x7fe8d0008040 comp rx=0 tx=0).stop 2024-04-28T13:33:39.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.952+0000 7fe8dfe00640 1 -- 172.21.15.196:0/1477971470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8e00d1760 msgr2=0x7fe8e00b1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:39.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.952+0000 7fe8dfe00640 1 --2- 172.21.15.196:0/1477971470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8e00d1760 0x7fe8e00b1b80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe8d802ad20 tx=0x7fe8d8004a40 comp rx=0 tx=0).stop 2024-04-28T13:33:39.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.952+0000 7fe8dfe00640 1 -- 172.21.15.196:0/1477971470 shutdown_connections 2024-04-28T13:33:39.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.952+0000 7fe8dfe00640 1 --2- 172.21.15.196:0/1477971470 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe8bc06c980 0x7fe8bc06ee40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.952+0000 7fe8dfe00640 1 --2- 172.21.15.196:0/1477971470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8e00d1760 0x7fe8e00b1b80 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.952+0000 7fe8dfe00640 1 --2- 172.21.15.196:0/1477971470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8e00b5ee0 0x7fe8e01f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.952+0000 7fe8dfe00640 1 --2- 172.21.15.196:0/1477971470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8e009ec80 0x7fe8e0157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:39.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.952+0000 7fe8dfe00640 1 -- 172.21.15.196:0/1477971470 >> 172.21.15.196:0/1477971470 conn(0x7fe8e006c6c0 msgr2=0x7fe8e00cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:39.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.953+0000 7fe8dfe00640 1 -- 172.21.15.196:0/1477971470 shutdown_connections 2024-04-28T13:33:39.953 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:39.953+0000 7fe8dfe00640 1 -- 172.21.15.196:0/1477971470 wait complete. 2024-04-28T13:33:40.096 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:33:40.104 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:33:40.374 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-28T13:33:33.440220+0000', 'modified': '2024-04-28T13:33:36.491827+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5781}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5781': {'gid': 5781, 'name': 'b', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6837/1525791340', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6835', 'nonce': 1525791340}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 1525791340}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5781, 'qdb_cluster': [5781]} 2024-04-28T13:33:40.374 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-28T13:33:40.374 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-28T13:33:40.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.586+0000 7f7df6000640 1 Processor -- start 2024-04-28T13:33:40.587 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.587+0000 7f7df6000640 1 -- start start 2024-04-28T13:33:40.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.588+0000 7f7df6000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7df009ba40 0x7f7df015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.588+0000 7f7df6000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7df0070570 0x7f7df00b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.588+0000 7f7df6000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7df015fce0 con 0x7f7df009be20 2024-04-28T13:33:40.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.588+0000 7f7df6000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f7df0160610 con 0x7f7df0070570 2024-04-28T13:33:40.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.588+0000 7f7df6000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7df00709b0 con 0x7f7df009ba40 2024-04-28T13:33:40.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.589+0000 7f7deea00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7df0070570 0x7f7df00b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.589+0000 7f7def400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7df009ba40 0x7f7df015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.589+0000 7f7defe00640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f7df009be20 0x7f7df0162f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.196:55302/0 (socket says 172.21.15.196:55302) 2024-04-28T13:33:40.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.589+0000 7f7defe00640 1 -- 172.21.15.196:0/3993075114 learned_addr learned my addr 172.21.15.196:0/3993075114 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:40.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.589+0000 7f7deea00640 1 --2- 172.21.15.196:0/3993075114 >> v2:172.21.15.203:3300/0 conn(0x7f7df0070570 0x7f7df00b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:40930/0 (socket says 172.21.15.196:40930) 2024-04-28T13:33:40.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.589+0000 7f7deea00640 1 -- 172.21.15.196:0/3993075114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7df009ba40 msgr2=0x7f7df015f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:40.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.589+0000 7f7deea00640 1 --2- 172.21.15.196:0/3993075114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7df009ba40 0x7f7df015f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.590+0000 7f7deea00640 1 -- 172.21.15.196:0/3993075114 >> v1:172.21.15.203:6789/0 conn(0x7f7df009be20 legacy=0x7f7df0162f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:40.590 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.590+0000 7f7deea00640 1 -- 172.21.15.196:0/3993075114 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df015ffc0 con 0x7f7df0070570 2024-04-28T13:33:40.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.591+0000 7f7deea00640 1 --2- 172.21.15.196:0/3993075114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7df0070570 0x7f7df00b5530 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8009d90 tx=0x7f7dd802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a47df46200cfab server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:40.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.592+0000 7f7dee000640 1 -- 172.21.15.196:0/3993075114 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8007740 con 0x7f7df0070570 2024-04-28T13:33:40.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.592+0000 7f7dee000640 1 -- 172.21.15.196:0/3993075114 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd80078e0 con 0x7f7df0070570 2024-04-28T13:33:40.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.592+0000 7f7dee000640 1 -- 172.21.15.196:0/3993075114 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8007cb0 con 0x7f7df0070570 2024-04-28T13:33:40.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.592+0000 7f7df6000640 1 -- 172.21.15.196:0/3993075114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7df0070570 msgr2=0x7f7df00b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:40.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.592+0000 7f7df6000640 1 --2- 172.21.15.196:0/3993075114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7df0070570 0x7f7df00b5530 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8009d90 tx=0x7f7dd802a660 comp rx=0 tx=0).stop 2024-04-28T13:33:40.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.593+0000 7f7df6000640 1 -- 172.21.15.196:0/3993075114 shutdown_connections 2024-04-28T13:33:40.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.593+0000 7f7df6000640 1 --2- 172.21.15.196:0/3993075114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7df0070570 0x7f7df00b5530 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.593+0000 7f7df6000640 1 --2- 172.21.15.196:0/3993075114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7df009ba40 0x7f7df015f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.593+0000 7f7df6000640 1 -- 172.21.15.196:0/3993075114 >> 172.21.15.196:0/3993075114 conn(0x7f7df00b4a10 msgr2=0x7f7df00b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:40.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.593+0000 7f7df6000640 1 -- 172.21.15.196:0/3993075114 shutdown_connections 2024-04-28T13:33:40.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.593+0000 7f7df6000640 1 -- 172.21.15.196:0/3993075114 wait complete. 2024-04-28T13:33:40.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.594+0000 7f7df6000640 1 Processor -- start 2024-04-28T13:33:40.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.595+0000 7f7df6000640 1 -- start start 2024-04-28T13:33:40.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.595+0000 7f7df6000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7df0070570 0x7f7df00ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7df6000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7df009ba40 0x7f7df0152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7df6000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7df009be20 0x7f7df00b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7df6000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7df00d2050 con 0x7f7df009ba40 2024-04-28T13:33:40.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7df6000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7df009c2b0 con 0x7f7df009be20 2024-04-28T13:33:40.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7df6000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7df00d21d0 con 0x7f7df0070570 2024-04-28T13:33:40.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7deea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7df009ba40 0x7f7df0152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7deea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7df009ba40 0x7f7df0152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59804/0 (socket says 172.21.15.196:59804) 2024-04-28T13:33:40.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7deea00640 1 -- 172.21.15.196:0/1613362572 learned_addr learned my addr 172.21.15.196:0/1613362572 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:40.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7def400640 1 --2- 172.21.15.196:0/1613362572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7df0070570 0x7f7df00ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7defe00640 1 --2- 172.21.15.196:0/1613362572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7df009be20 0x7f7df00b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7deea00640 1 -- 172.21.15.196:0/1613362572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7df0070570 msgr2=0x7f7df00ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:40.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7deea00640 1 --2- 172.21.15.196:0/1613362572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7df0070570 0x7f7df00ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7deea00640 1 -- 172.21.15.196:0/1613362572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7df009be20 msgr2=0x7f7df00b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:40.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7deea00640 1 --2- 172.21.15.196:0/1613362572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7df009be20 0x7f7df00b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7deea00640 1 -- 172.21.15.196:0/1613362572 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df00cf2c0 con 0x7f7df009ba40 2024-04-28T13:33:40.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.596+0000 7f7defe00640 1 --2- 172.21.15.196:0/1613362572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7df009be20 0x7f7df00b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:40.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.597+0000 7f7deea00640 1 --2- 172.21.15.196:0/1613362572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7df009ba40 0x7f7df0152520 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8009eb0 tx=0x7f7dd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:40.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.597+0000 7f7de7400640 1 -- 172.21.15.196:0/1613362572 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8040070 con 0x7f7df009ba40 2024-04-28T13:33:40.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.597+0000 7f7de7400640 1 -- 172.21.15.196:0/1613362572 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd8031bc0 con 0x7f7df009ba40 2024-04-28T13:33:40.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.597+0000 7f7de7400640 1 -- 172.21.15.196:0/1613362572 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8045460 con 0x7f7df009ba40 2024-04-28T13:33:40.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.597+0000 7f7df6000640 1 -- 172.21.15.196:0/1613362572 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df00cf550 con 0x7f7df009ba40 2024-04-28T13:33:40.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.597+0000 7f7df6000640 1 -- 172.21.15.196:0/1613362572 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df00cfb10 con 0x7f7df009ba40 2024-04-28T13:33:40.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.599+0000 7f7de7400640 1 -- 172.21.15.196:0/1613362572 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7dd8039040 con 0x7f7df009ba40 2024-04-28T13:33:40.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.600+0000 7f7de7400640 1 --2- 172.21.15.196:0/1613362572 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7dcc06c980 0x7f7dcc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.600+0000 7f7de7400640 1 -- 172.21.15.196:0/1613362572 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f7dd80ad5a0 con 0x7f7df009ba40 2024-04-28T13:33:40.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.601+0000 7f7def400640 1 --2- 172.21.15.196:0/1613362572 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7dcc06c980 0x7f7dcc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.601+0000 7f7df8dc5740 1 Processor -- start 2024-04-28T13:33:40.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.601+0000 7f7df8dc5740 1 -- start start 2024-04-28T13:33:40.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.602+0000 7f7df8dc5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec245e0 0x56303ec26740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.602+0000 7f7df8dc5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec60cb0 0x56303ec249c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.602+0000 7f7df8dc5740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec4f0a0 0x56303ec4f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.602+0000 7f7df8dc5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56303eb88600 con 0x56303ec60cb0 2024-04-28T13:33:40.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.602+0000 7f7df8dc5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56303eb25100 con 0x56303ec245e0 2024-04-28T13:33:40.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.602+0000 7f7df8dc5740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x56303eb845c0 con 0x56303ec4f0a0 2024-04-28T13:33:40.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.602+0000 7f7def400640 1 --2- 172.21.15.196:0/1613362572 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7dcc06c980 0x7f7dcc06ee40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009a10 tx=0x7f7de0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:40.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.602+0000 7f7deea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec60cb0 0x56303ec249c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.602+0000 7f7deea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec60cb0 0x56303ec249c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59806/0 (socket says 172.21.15.196:59806) 2024-04-28T13:33:40.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.602+0000 7f7deea00640 1 -- 172.21.15.196:0/1364317898 learned_addr learned my addr 172.21.15.196:0/1364317898 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:40.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.603+0000 7f7def400640 1 --2- 172.21.15.196:0/1364317898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec245e0 0x56303ec26740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.603+0000 7f7defe00640 1 --2- 172.21.15.196:0/1364317898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec4f0a0 0x56303ec4f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.603+0000 7f7deea00640 1 -- 172.21.15.196:0/1364317898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec4f0a0 msgr2=0x56303ec4f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:40.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.603+0000 7f7deea00640 1 --2- 172.21.15.196:0/1364317898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec4f0a0 0x56303ec4f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.603+0000 7f7deea00640 1 -- 172.21.15.196:0/1364317898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec245e0 msgr2=0x56303ec26740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:40.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.603+0000 7f7deea00640 1 --2- 172.21.15.196:0/1364317898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec245e0 0x56303ec26740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.603+0000 7f7deea00640 1 -- 172.21.15.196:0/1364317898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56303e89a2d0 con 0x56303ec60cb0 2024-04-28T13:33:40.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.603+0000 7f7defe00640 1 --2- 172.21.15.196:0/1364317898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec4f0a0 0x56303ec4f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:40.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.603+0000 7f7def400640 1 --2- 172.21.15.196:0/1364317898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec245e0 0x56303ec26740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:40.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.603+0000 7f7deea00640 1 --2- 172.21.15.196:0/1364317898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec60cb0 0x56303ec249c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7dd807b0f0 tx=0x7f7dd807b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ecfbd49b60ee41 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:40.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.603+0000 7f7ddf400640 1 -- 172.21.15.196:0/1364317898 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd804a040 con 0x56303ec60cb0 2024-04-28T13:33:40.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.603+0000 7f7ddf400640 1 -- 172.21.15.196:0/1364317898 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd807f510 con 0x56303ec60cb0 2024-04-28T13:33:40.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.603+0000 7f7ddf400640 1 -- 172.21.15.196:0/1364317898 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8083d40 con 0x56303ec60cb0 2024-04-28T13:33:40.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.604+0000 7f7df8dc5740 1 -- 172.21.15.196:0/1364317898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec60cb0 msgr2=0x56303ec249c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:40.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.604+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/1364317898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec60cb0 0x56303ec249c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7dd807b0f0 tx=0x7f7dd807b860 comp rx=0 tx=0).stop 2024-04-28T13:33:40.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.604+0000 7f7df8dc5740 1 -- 172.21.15.196:0/1364317898 shutdown_connections 2024-04-28T13:33:40.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.604+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/1364317898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec4f0a0 0x56303ec4f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.604+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/1364317898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec60cb0 0x56303ec249c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.604+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/1364317898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec245e0 0x56303ec26740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.604+0000 7f7df8dc5740 1 -- 172.21.15.196:0/1364317898 >> 172.21.15.196:0/1364317898 conn(0x56303ebb8d10 msgr2=0x56303ec6a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:40.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.604+0000 7f7df8dc5740 1 -- 172.21.15.196:0/1364317898 shutdown_connections 2024-04-28T13:33:40.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.604+0000 7f7df8dc5740 1 -- 172.21.15.196:0/1364317898 wait complete. 2024-04-28T13:33:40.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.605+0000 7f7df8dc5740 1 -- start start 2024-04-28T13:33:40.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.605+0000 7f7df8dc5740 1 Processor -- start 2024-04-28T13:33:40.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.606+0000 7f7df8dc5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec245e0 0x56303ec3f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.606+0000 7f7df8dc5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec4f0a0 0x56303ec62cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.606+0000 7f7df8dc5740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec60cb0 0x56303ec63210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.606+0000 7f7df8dc5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56303eb25100 con 0x56303ec4f0a0 2024-04-28T13:33:40.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.606+0000 7f7df8dc5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56303eb0ff70 con 0x56303ec245e0 2024-04-28T13:33:40.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.606+0000 7f7df8dc5740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x56303eb88600 con 0x56303ec60cb0 2024-04-28T13:33:40.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.606+0000 7f7df8dc5740 10 client.0 _ll_register_callbacks cb 0x56303eb7e480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T13:33:40.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.606+0000 7f7deea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec4f0a0 0x56303ec62cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.606+0000 7f7deea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec4f0a0 0x56303ec62cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59818/0 (socket says 172.21.15.196:59818) 2024-04-28T13:33:40.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.606+0000 7f7deea00640 1 -- 172.21.15.196:0/3447791921 learned_addr learned my addr 172.21.15.196:0/3447791921 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:40.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7def400640 1 --2- 172.21.15.196:0/3447791921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec245e0 0x56303ec3f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7defe00640 1 --2- 172.21.15.196:0/3447791921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec60cb0 0x56303ec63210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7deea00640 1 -- 172.21.15.196:0/3447791921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec60cb0 msgr2=0x56303ec63210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:40.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7deea00640 1 --2- 172.21.15.196:0/3447791921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec60cb0 0x56303ec63210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7deea00640 1 -- 172.21.15.196:0/3447791921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec245e0 msgr2=0x56303ec3f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:40.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7deea00640 1 --2- 172.21.15.196:0/3447791921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec245e0 0x56303ec3f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7deea00640 1 -- 172.21.15.196:0/3447791921 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56303e9646b0 con 0x56303ec4f0a0 2024-04-28T13:33:40.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7defe00640 1 --2- 172.21.15.196:0/3447791921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec60cb0 0x56303ec63210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:40.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7deea00640 1 --2- 172.21.15.196:0/3447791921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec4f0a0 0x56303ec62cd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7dd807b320 tx=0x7f7dd803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77e8f2e8ddd9c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:40.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7dd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-04-28T13:33:40.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7dd7e00640 1 -- 172.21.15.196:0/3447791921 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd809a020 con 0x56303ec4f0a0 2024-04-28T13:33:40.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7dd7e00640 1 -- 172.21.15.196:0/3447791921 <== mon.0 v2:172.21.15.196:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f7dd807b3c0 con 0x56303ec4f0a0 2024-04-28T13:33:40.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7df8dc5740 1 -- 172.21.15.196:0/3447791921 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dd807f510 con 0x56303ec4f0a0 2024-04-28T13:33:40.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7dd7e00640 1 -- 172.21.15.196:0/3447791921 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7dd80a0050 con 0x56303ec4f0a0 2024-04-28T13:33:40.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7df8dc5740 10 client.5925 fetch_fsmap learned FSMap version 31 2024-04-28T13:33:40.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.607+0000 7f7df8dc5740 1 -- 172.21.15.196:0/3447791921 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x56303ec61750 con 0x56303ec4f0a0 2024-04-28T13:33:40.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.608+0000 7f7dd7e00640 1 -- 172.21.15.196:0/3447791921 <== mon.0 v2:172.21.15.196:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd807f510 con 0x56303ec4f0a0 2024-04-28T13:33:40.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.608+0000 7f7dd7e00640 1 -- 172.21.15.196:0/3447791921 <== mon.0 v2:172.21.15.196:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7f7dd80838b0 con 0x56303ec4f0a0 2024-04-28T13:33:40.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.608+0000 7f7df8dc5740 10 client.5925 fetch_fsmap finished waiting for FSMap version 31 2024-04-28T13:33:40.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.608+0000 7f7df8dc5740 10 client.5925 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T13:33:40.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.608+0000 7f7df8dc5740 20 client.5925 populate_metadata read hostname 'smithi196' 2024-04-28T13:33:40.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.608+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/3447791921 >> [v2:172.21.15.203:6835/1525791340,v1:172.21.15.203:6837/1525791340] conn(0x56303ec619e0 0x56303ec86fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.608+0000 7f7df8dc5740 4 client.5925 mds_command: new command op to 5781 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T13:33:40.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.608+0000 7f7df8dc5740 1 -- 172.21.15.196:0/3447791921 --> [v2:172.21.15.203:6835/1525791340,v1:172.21.15.203:6837/1525791340] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56303eb96210 con 0x56303ec619e0 2024-04-28T13:33:40.612 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.609+0000 7f7def400640 1 --2- 172.21.15.196:0/3447791921 >> [v2:172.21.15.203:6835/1525791340,v1:172.21.15.203:6837/1525791340] conn(0x56303ec619e0 0x56303ec86fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.610+0000 7f7def400640 1 --2- 172.21.15.196:0/3447791921 >> [v2:172.21.15.203:6835/1525791340,v1:172.21.15.203:6837/1525791340] conn(0x56303ec619e0 0x56303ec86fa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=985e37b5c1c0a472 server_cookie=e7bec5d79c6636a4 in_seq=0 out_seq=0 2024-04-28T13:33:40.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.610+0000 7f7dd7e00640 10 client.5925 ms_handle_connect on v2:172.21.15.203:6835/1525791340 2024-04-28T13:33:40.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.612+0000 7f7dd7e00640 1 -- 172.21.15.196:0/3447791921 <== mds.0 v2:172.21.15.203:6835/1525791340 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56303eb96210 con 0x56303ec619e0 2024-04-28T13:33:40.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.612+0000 7f7dd7e00640 10 client.5925 handle_command_reply: tid=0 2024-04-28T13:33:40.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.612+0000 7f7df8dc5740 1 client.5925 shutdown 2024-04-28T13:33:40.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.612+0000 7f7df8dc5740 1 -- 172.21.15.196:0/3447791921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec4f0a0 msgr2=0x56303ec62cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:40.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.612+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/3447791921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec4f0a0 0x56303ec62cd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7dd807b320 tx=0x7f7dd803f000 comp rx=0 tx=0).stop 2024-04-28T13:33:40.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.612+0000 7f7df8dc5740 1 -- 172.21.15.196:0/3447791921 shutdown_connections 2024-04-28T13:33:40.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.612+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/3447791921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec60cb0 0x56303ec63210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.612+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/3447791921 >> [v2:172.21.15.203:6835/1525791340,v1:172.21.15.203:6837/1525791340] conn(0x56303ec619e0 0x56303ec86fa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.612+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/3447791921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec4f0a0 0x56303ec62cd0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.612+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/3447791921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec245e0 0x56303ec3f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.612+0000 7f7dd7e00640 0 client.5925 ms_handle_reset on v2:172.21.15.203:6835/1525791340 2024-04-28T13:33:40.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.612+0000 7f7df8dc5740 1 -- 172.21.15.196:0/3447791921 >> 172.21.15.196:0/3447791921 conn(0x56303ebb8d10 msgr2=0x56303ec6d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:40.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.612+0000 7f7df8dc5740 1 -- 172.21.15.196:0/3447791921 shutdown_connections 2024-04-28T13:33:40.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.612+0000 7f7df8dc5740 1 -- 172.21.15.196:0/3447791921 wait complete. 2024-04-28T13:33:40.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.613+0000 7f7df8dc5740 20 client.5925 trim_cache size 0 max 16384 2024-04-28T13:33:40.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.631+0000 7f7df8dc5740 1 Processor -- start 2024-04-28T13:33:40.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.631+0000 7f7df8dc5740 1 -- start start 2024-04-28T13:33:40.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.632+0000 7f7df8dc5740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec245e0 0x56303ec62c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.632+0000 7f7df8dc5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec4f0a0 0x56303ec52d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.632+0000 7f7df8dc5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec60cb0 0x56303ec4a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.632+0000 7f7df8dc5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7df0200a80 con 0x56303ec60cb0 2024-04-28T13:33:40.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.632+0000 7f7df8dc5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7df00cbe40 con 0x56303ec4f0a0 2024-04-28T13:33:40.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.632+0000 7f7df8dc5740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7df00c7fa0 con 0x56303ec245e0 2024-04-28T13:33:40.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.633+0000 7f7defe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec60cb0 0x56303ec4a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.633+0000 7f7defe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec60cb0 0x56303ec4a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59830/0 (socket says 172.21.15.196:59830) 2024-04-28T13:33:40.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.633+0000 7f7defe00640 1 -- 172.21.15.196:0/2805078060 learned_addr learned my addr 172.21.15.196:0/2805078060 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:40.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.633+0000 7f7deea00640 1 --2- 172.21.15.196:0/2805078060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec4f0a0 0x56303ec52d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.633+0000 7f7def400640 1 --2- 172.21.15.196:0/2805078060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec245e0 0x56303ec62c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.633+0000 7f7defe00640 1 -- 172.21.15.196:0/2805078060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec245e0 msgr2=0x56303ec62c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:40.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.633+0000 7f7defe00640 1 --2- 172.21.15.196:0/2805078060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec245e0 0x56303ec62c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.633+0000 7f7defe00640 1 -- 172.21.15.196:0/2805078060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec4f0a0 msgr2=0x56303ec52d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:40.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.633+0000 7f7defe00640 1 --2- 172.21.15.196:0/2805078060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec4f0a0 0x56303ec52d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.633+0000 7f7defe00640 1 -- 172.21.15.196:0/2805078060 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56303ec4ecc0 con 0x56303ec60cb0 2024-04-28T13:33:40.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.633+0000 7f7deea00640 1 --2- 172.21.15.196:0/2805078060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec4f0a0 0x56303ec52d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:40.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.634+0000 7f7defe00640 1 --2- 172.21.15.196:0/2805078060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec60cb0 0x56303ec4a240 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7de800f6a0 tx=0x7f7de800fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4185f6907f86b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:40.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.634+0000 7f7ddea00640 1 -- 172.21.15.196:0/2805078060 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8005ab0 con 0x56303ec60cb0 2024-04-28T13:33:40.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.634+0000 7f7ddea00640 1 -- 172.21.15.196:0/2805078060 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de8005c70 con 0x56303ec60cb0 2024-04-28T13:33:40.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.634+0000 7f7ddea00640 1 -- 172.21.15.196:0/2805078060 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de800da30 con 0x56303ec60cb0 2024-04-28T13:33:40.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.634+0000 7f7df8dc5740 1 -- 172.21.15.196:0/2805078060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec60cb0 msgr2=0x56303ec4a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:40.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.634+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/2805078060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec60cb0 0x56303ec4a240 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7de800f6a0 tx=0x7f7de800fb60 comp rx=0 tx=0).stop 2024-04-28T13:33:40.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.634+0000 7f7df8dc5740 1 -- 172.21.15.196:0/2805078060 shutdown_connections 2024-04-28T13:33:40.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.634+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/2805078060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec60cb0 0x56303ec4a240 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.634+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/2805078060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec4f0a0 0x56303ec52d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.634+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/2805078060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec245e0 0x56303ec62c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.634+0000 7f7df8dc5740 1 -- 172.21.15.196:0/2805078060 >> 172.21.15.196:0/2805078060 conn(0x56303ebb8d10 msgr2=0x56303ec525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:40.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.634+0000 7f7df8dc5740 1 -- 172.21.15.196:0/2805078060 shutdown_connections 2024-04-28T13:33:40.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.634+0000 7f7df8dc5740 1 -- 172.21.15.196:0/2805078060 wait complete. 2024-04-28T13:33:40.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.635+0000 7f7df8dc5740 1 -- start start 2024-04-28T13:33:40.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.635+0000 7f7df8dc5740 1 Processor -- start 2024-04-28T13:33:40.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.636+0000 7f7df8dc5740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec245e0 0x56303ec82840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.636+0000 7f7df8dc5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec4f0a0 0x56303ec82d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.636+0000 7f7df8dc5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec60cb0 0x56303ece6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.636+0000 7f7df8dc5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56303eb25100 con 0x56303ec4f0a0 2024-04-28T13:33:40.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.636+0000 7f7df8dc5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56303ec51000 con 0x56303ec60cb0 2024-04-28T13:33:40.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.636+0000 7f7df8dc5740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x56303eb0ff70 con 0x56303ec245e0 2024-04-28T13:33:40.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.636+0000 7f7df8dc5740 10 client.0 _ll_register_callbacks cb 0x7f7dd807ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T13:33:40.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.636+0000 7f7deea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec4f0a0 0x56303ec82d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.636+0000 7f7deea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec4f0a0 0x56303ec82d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59836/0 (socket says 172.21.15.196:59836) 2024-04-28T13:33:40.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.636+0000 7f7deea00640 1 -- 172.21.15.196:0/1664056774 learned_addr learned my addr 172.21.15.196:0/1664056774 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:40.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.637+0000 7f7deea00640 1 -- 172.21.15.196:0/1664056774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec245e0 msgr2=0x56303ec82840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:40.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.637+0000 7f7deea00640 1 --2- 172.21.15.196:0/1664056774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec245e0 0x56303ec82840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.637+0000 7f7deea00640 1 -- 172.21.15.196:0/1664056774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec60cb0 msgr2=0x56303ece6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.637+0000 7f7deea00640 1 --2- 172.21.15.196:0/1664056774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec60cb0 0x56303ece6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.637+0000 7f7deea00640 1 -- 172.21.15.196:0/1664056774 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56303ec78890 con 0x56303ec4f0a0 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.637+0000 7f7deea00640 1 --2- 172.21.15.196:0/1664056774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec4f0a0 0x56303ec82d80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80093e0 tx=0x7f7dd8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd35f4cbede089f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.637+0000 7f7dd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.637+0000 7f7dd7e00640 1 -- 172.21.15.196:0/1664056774 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd804a040 con 0x56303ec4f0a0 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.637+0000 7f7dd7e00640 1 -- 172.21.15.196:0/1664056774 <== mon.0 v2:172.21.15.196:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f7dd807b3c0 con 0x56303ec4f0a0 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.637+0000 7f7df8dc5740 1 -- 172.21.15.196:0/1664056774 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de8005c70 con 0x56303ec4f0a0 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.637+0000 7f7dd7e00640 1 -- 172.21.15.196:0/1664056774 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7dd807f510 con 0x56303ec4f0a0 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.637+0000 7f7df8dc5740 10 client.5931 fetch_fsmap learned FSMap version 31 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.637+0000 7f7df8dc5740 1 -- 172.21.15.196:0/1664056774 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x56303ec78890 con 0x56303ec4f0a0 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.638+0000 7f7dd7e00640 1 -- 172.21.15.196:0/1664056774 <== mon.0 v2:172.21.15.196:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de8005c70 con 0x56303ec4f0a0 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.638+0000 7f7dd7e00640 1 -- 172.21.15.196:0/1664056774 <== mon.0 v2:172.21.15.196:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7f7dd808fb70 con 0x56303ec4f0a0 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.638+0000 7f7df8dc5740 10 client.5931 fetch_fsmap finished waiting for FSMap version 31 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.638+0000 7f7df8dc5740 10 client.5931 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T13:33:40.640 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.638+0000 7f7df8dc5740 20 client.5931 populate_metadata read hostname 'smithi196' 2024-04-28T13:33:40.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.638+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/1664056774 >> [v2:172.21.15.203:6835/1525791340,v1:172.21.15.203:6837/1525791340] conn(0x56303ec69250 0x56303ec69630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.638+0000 7f7df8dc5740 4 client.5931 mds_command: new command op to 5781 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T13:33:40.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.638+0000 7f7df8dc5740 1 -- 172.21.15.196:0/1664056774 --> [v2:172.21.15.203:6835/1525791340,v1:172.21.15.203:6837/1525791340] -- command(tid 0: {"prefix": "status"}) -- 0x56303eb90080 con 0x56303ec69250 2024-04-28T13:33:40.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.638+0000 7f7def400640 1 --2- 172.21.15.196:0/1664056774 >> [v2:172.21.15.203:6835/1525791340,v1:172.21.15.203:6837/1525791340] conn(0x56303ec69250 0x56303ec69630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.639+0000 7f7def400640 1 --2- 172.21.15.196:0/1664056774 >> [v2:172.21.15.203:6835/1525791340,v1:172.21.15.203:6837/1525791340] conn(0x56303ec69250 0x56303ec69630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e09ac7d821064d4 server_cookie=5d15d19b12928f88 in_seq=0 out_seq=0 2024-04-28T13:33:40.641 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.639+0000 7f7dd7e00640 10 client.5931 ms_handle_connect on v2:172.21.15.203:6835/1525791340 2024-04-28T13:33:40.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.640+0000 7f7dd7e00640 1 -- 172.21.15.196:0/1664056774 <== mds.0 v2:172.21.15.203:6835/1525791340 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56303eb90080 con 0x56303ec69250 2024-04-28T13:33:40.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.640+0000 7f7dd7e00640 10 client.5931 handle_command_reply: tid=0 2024-04-28T13:33:40.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.640+0000 7f7df8dc5740 1 client.5931 shutdown 2024-04-28T13:33:40.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.640+0000 7f7df8dc5740 1 -- 172.21.15.196:0/1664056774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec4f0a0 msgr2=0x56303ec82d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:40.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.640+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/1664056774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec4f0a0 0x56303ec82d80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80093e0 tx=0x7f7dd8009750 comp rx=0 tx=0).stop 2024-04-28T13:33:40.642 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.640+0000 7f7df8dc5740 1 -- 172.21.15.196:0/1664056774 shutdown_connections 2024-04-28T13:33:40.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.640+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/1664056774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56303ec60cb0 0x56303ece6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.640+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/1664056774 >> [v2:172.21.15.203:6835/1525791340,v1:172.21.15.203:6837/1525791340] conn(0x56303ec69250 0x56303ec69630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.640+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/1664056774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56303ec4f0a0 0x56303ec82d80 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.640+0000 7f7df8dc5740 1 --2- 172.21.15.196:0/1664056774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56303ec245e0 0x56303ec82840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.640+0000 7f7dd7e00640 0 client.5931 ms_handle_reset on v2:172.21.15.203:6835/1525791340 2024-04-28T13:33:40.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.640+0000 7f7df8dc5740 1 -- 172.21.15.196:0/1664056774 >> 172.21.15.196:0/1664056774 conn(0x56303ebb8d10 msgr2=0x56303ec60340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:40.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.640+0000 7f7df8dc5740 1 -- 172.21.15.196:0/1664056774 shutdown_connections 2024-04-28T13:33:40.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.641+0000 7f7df8dc5740 1 -- 172.21.15.196:0/1664056774 wait complete. 2024-04-28T13:33:40.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.641+0000 7f7df8dc5740 20 client.5931 trim_cache size 0 max 16384 2024-04-28T13:33:40.643 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:33:40.643 INFO:teuthology.orchestra.run.smithi196.stdout: "cluster_fsid": "270aea32-2e16-4a97-85f7-aeae39bc8c76", 2024-04-28T13:33:40.644 INFO:teuthology.orchestra.run.smithi196.stdout: "whoami": 0, 2024-04-28T13:33:40.644 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 5781, 2024-04-28T13:33:40.644 INFO:teuthology.orchestra.run.smithi196.stdout: "want_state": "up:active", 2024-04-28T13:33:40.644 INFO:teuthology.orchestra.run.smithi196.stdout: "state": "up:active", 2024-04-28T13:33:40.644 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs", 2024-04-28T13:33:40.644 INFO:teuthology.orchestra.run.smithi196.stdout: "rank_uptime": 6.6719213120000003, 2024-04-28T13:33:40.644 INFO:teuthology.orchestra.run.smithi196.stdout: "mdsmap_epoch": 31, 2024-04-28T13:33:40.644 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_epoch": 45, 2024-04-28T13:33:40.644 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_epoch_barrier": 45, 2024-04-28T13:33:40.644 INFO:teuthology.orchestra.run.smithi196.stdout: "uptime": 12.074371398 2024-04-28T13:33:40.644 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:33:40.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.641+0000 7f7df6000640 1 -- 172.21.15.196:0/1613362572 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7dcc06c980 msgr2=0x7f7dcc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:40.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.641+0000 7f7df6000640 1 --2- 172.21.15.196:0/1613362572 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7dcc06c980 0x7f7dcc06ee40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009a10 tx=0x7f7de0008040 comp rx=0 tx=0).stop 2024-04-28T13:33:40.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.641+0000 7f7df6000640 1 -- 172.21.15.196:0/1613362572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7df009ba40 msgr2=0x7f7df0152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:40.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.641+0000 7f7df6000640 1 --2- 172.21.15.196:0/1613362572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7df009ba40 0x7f7df0152520 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8009eb0 tx=0x7f7dd8004a40 comp rx=0 tx=0).stop 2024-04-28T13:33:40.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.641+0000 7f7df6000640 1 -- 172.21.15.196:0/1613362572 shutdown_connections 2024-04-28T13:33:40.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.641+0000 7f7df6000640 1 --2- 172.21.15.196:0/1613362572 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7dcc06c980 0x7f7dcc06ee40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.641+0000 7f7df6000640 1 --2- 172.21.15.196:0/1613362572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7df009be20 0x7f7df00b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.641+0000 7f7df6000640 1 --2- 172.21.15.196:0/1613362572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7df009ba40 0x7f7df0152520 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.641+0000 7f7df6000640 1 --2- 172.21.15.196:0/1613362572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7df0070570 0x7f7df00ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.641+0000 7f7df6000640 1 -- 172.21.15.196:0/1613362572 >> 172.21.15.196:0/1613362572 conn(0x7f7df00b4a10 msgr2=0x7f7df006ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:40.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.641+0000 7f7df6000640 1 -- 172.21.15.196:0/1613362572 shutdown_connections 2024-04-28T13:33:40.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.641+0000 7f7df6000640 1 -- 172.21.15.196:0/1613362572 wait complete. 2024-04-28T13:33:40.657 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T13:33:40.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.839+0000 7f9e6b000640 1 Processor -- start 2024-04-28T13:33:40.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.839+0000 7f9e6b000640 1 -- start start 2024-04-28T13:33:40.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.840+0000 7f9e6b000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9e6415b4e0 0x7f9e64071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.840+0000 7f9e6b000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9e6415b8c0 0x7f9e640cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.840+0000 7f9e6b000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9e64073820 0x7f9e6406ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.840+0000 7f9e6b000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9e640ce230 con 0x7f9e6415b8c0 2024-04-28T13:33:40.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.841+0000 7f9e6b000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9e640733e0 con 0x7f9e6415b4e0 2024-04-28T13:33:40.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.841+0000 7f9e6b000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9e64072f00 con 0x7f9e64073820 2024-04-28T13:33:40.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.841+0000 7f9e5b400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9e6415b8c0 0x7f9e640cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.841+0000 7f9e5b400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9e6415b8c0 0x7f9e640cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59838/0 (socket says 172.21.15.196:59838) 2024-04-28T13:33:40.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.841+0000 7f9e5b400640 1 -- 172.21.15.196:0/2948409572 learned_addr learned my addr 172.21.15.196:0/2948409572 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:40.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.841+0000 7f9e69000640 1 --2- 172.21.15.196:0/2948409572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9e64073820 0x7f9e6406ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.841+0000 7f9e63e00640 1 --2- 172.21.15.196:0/2948409572 >> v2:172.21.15.203:3300/0 conn(0x7f9e6415b4e0 0x7f9e64071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.841+0000 7f9e5b400640 1 -- 172.21.15.196:0/2948409572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9e64073820 msgr2=0x7f9e6406ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:40.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.841+0000 7f9e5b400640 1 --2- 172.21.15.196:0/2948409572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9e64073820 0x7f9e6406ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.841+0000 7f9e5b400640 1 -- 172.21.15.196:0/2948409572 >> v2:172.21.15.203:3300/0 conn(0x7f9e6415b4e0 msgr2=0x7f9e64071fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:40.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.841+0000 7f9e5b400640 1 --2- 172.21.15.196:0/2948409572 >> v2:172.21.15.203:3300/0 conn(0x7f9e6415b4e0 0x7f9e64071fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.841+0000 7f9e5b400640 1 -- 172.21.15.196:0/2948409572 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e64072660 con 0x7f9e6415b8c0 2024-04-28T13:33:40.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.841+0000 7f9e63e00640 1 --2- 172.21.15.196:0/2948409572 >> v2:172.21.15.203:3300/0 conn(0x7f9e6415b4e0 0x7f9e64071fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:40.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.843+0000 7f9e5b400640 1 --2- 172.21.15.196:0/2948409572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6415b8c0 0x7f9e640cdcf0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9e640b3ba0 tx=0x7f9e4c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da955f919f194a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:40.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.843+0000 7f9e63400640 1 -- 172.21.15.196:0/2948409572 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c037070 con 0x7f9e6415b8c0 2024-04-28T13:33:40.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.843+0000 7f9e63400640 1 -- 172.21.15.196:0/2948409572 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4c032440 con 0x7f9e6415b8c0 2024-04-28T13:33:40.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.843+0000 7f9e63400640 1 -- 172.21.15.196:0/2948409572 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c032810 con 0x7f9e6415b8c0 2024-04-28T13:33:40.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.843+0000 7f9e6b000640 1 -- 172.21.15.196:0/2948409572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6415b8c0 msgr2=0x7f9e640cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:40.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.843+0000 7f9e6b000640 1 --2- 172.21.15.196:0/2948409572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6415b8c0 0x7f9e640cdcf0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9e640b3ba0 tx=0x7f9e4c02a1e0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.844+0000 7f9e6b000640 1 -- 172.21.15.196:0/2948409572 shutdown_connections 2024-04-28T13:33:40.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.844+0000 7f9e6b000640 1 --2- 172.21.15.196:0/2948409572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6415b8c0 0x7f9e640cdcf0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.844+0000 7f9e6b000640 1 --2- 172.21.15.196:0/2948409572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9e64073820 0x7f9e6406ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.844+0000 7f9e6b000640 1 --2- 172.21.15.196:0/2948409572 >> v2:172.21.15.203:3300/0 conn(0x7f9e6415b4e0 0x7f9e64071fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.844+0000 7f9e6b000640 1 -- 172.21.15.196:0/2948409572 >> 172.21.15.196:0/2948409572 conn(0x7f9e640d0d30 msgr2=0x7f9e640d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:40.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.844+0000 7f9e6b000640 1 -- 172.21.15.196:0/2948409572 shutdown_connections 2024-04-28T13:33:40.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.844+0000 7f9e6b000640 1 -- 172.21.15.196:0/2948409572 wait complete. 2024-04-28T13:33:40.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.845+0000 7f9e6b000640 1 Processor -- start 2024-04-28T13:33:40.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.846+0000 7f9e6b000640 1 -- start start 2024-04-28T13:33:40.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.846+0000 7f9e6b000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9e64073820 0x7f9e6414abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.846+0000 7f9e6b000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e6415b4e0 0x7f9e64156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.846+0000 7f9e6b000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6415b8c0 0x7f9e64158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.846+0000 7f9e6b000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e640b3e80 con 0x7f9e6415b8c0 2024-04-28T13:33:40.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.846+0000 7f9e6b000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9e64072c20 con 0x7f9e6415b4e0 2024-04-28T13:33:40.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.846+0000 7f9e6b000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6406f0f0 con 0x7f9e64073820 2024-04-28T13:33:40.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.847+0000 7f9e69000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6415b8c0 0x7f9e64158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.847+0000 7f9e69000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6415b8c0 0x7f9e64158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59850/0 (socket says 172.21.15.196:59850) 2024-04-28T13:33:40.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.847+0000 7f9e69000640 1 -- 172.21.15.196:0/3321547657 learned_addr learned my addr 172.21.15.196:0/3321547657 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:40.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.847+0000 7f9e5b400640 1 --2- 172.21.15.196:0/3321547657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e6415b4e0 0x7f9e64156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.847+0000 7f9e63e00640 1 --2- 172.21.15.196:0/3321547657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9e64073820 0x7f9e6414abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.847+0000 7f9e69000640 1 -- 172.21.15.196:0/3321547657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9e64073820 msgr2=0x7f9e6414abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:40.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.847+0000 7f9e69000640 1 --2- 172.21.15.196:0/3321547657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9e64073820 0x7f9e6414abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.847+0000 7f9e69000640 1 -- 172.21.15.196:0/3321547657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e6415b4e0 msgr2=0x7f9e64156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:40.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.847+0000 7f9e69000640 1 --2- 172.21.15.196:0/3321547657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e6415b4e0 0x7f9e64156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:40.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.847+0000 7f9e69000640 1 -- 172.21.15.196:0/3321547657 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e64157560 con 0x7f9e6415b8c0 2024-04-28T13:33:40.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.847+0000 7f9e5b400640 1 --2- 172.21.15.196:0/3321547657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e6415b4e0 0x7f9e64156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:40.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.848+0000 7f9e69000640 1 --2- 172.21.15.196:0/3321547657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6415b8c0 0x7f9e64158ad0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00d800 tx=0x7f9e5c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:40.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.848+0000 7f9e60c00640 1 -- 172.21.15.196:0/3321547657 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c018020 con 0x7f9e6415b8c0 2024-04-28T13:33:40.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.848+0000 7f9e60c00640 1 -- 172.21.15.196:0/3321547657 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e5c0096c0 con 0x7f9e6415b8c0 2024-04-28T13:33:40.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.848+0000 7f9e60c00640 1 -- 172.21.15.196:0/3321547657 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c009a90 con 0x7f9e6415b8c0 2024-04-28T13:33:40.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.848+0000 7f9e6b000640 1 -- 172.21.15.196:0/3321547657 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6414fae0 con 0x7f9e6415b8c0 2024-04-28T13:33:40.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.848+0000 7f9e6b000640 1 -- 172.21.15.196:0/3321547657 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e641500a0 con 0x7f9e6415b8c0 2024-04-28T13:33:40.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.849+0000 7f9e6b000640 1 -- 172.21.15.196:0/3321547657 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e6406a290 con 0x7f9e6415b8c0 2024-04-28T13:33:40.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.850+0000 7f9e60c00640 1 -- 172.21.15.196:0/3321547657 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f9e5c01b020 con 0x7f9e6415b8c0 2024-04-28T13:33:40.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.851+0000 7f9e60c00640 1 --2- 172.21.15.196:0/3321547657 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9e4006c980 0x7f9e4006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:40.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.851+0000 7f9e60c00640 1 -- 172.21.15.196:0/3321547657 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f9e5c08c1f0 con 0x7f9e6415b8c0 2024-04-28T13:33:40.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.852+0000 7f9e63e00640 1 --2- 172.21.15.196:0/3321547657 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9e4006c980 0x7f9e4006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:40.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.853+0000 7f9e63e00640 1 --2- 172.21.15.196:0/3321547657 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9e4006c980 0x7f9e4006ee40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9e54008f80 tx=0x7f9e540084d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:40.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:40.855+0000 7f9e60c00640 1 -- 172.21.15.196:0/3321547657 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9e5c022020 con 0x7f9e6415b8c0 2024-04-28T13:33:41.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41.097+0000 7f9e6b000640 1 -- 172.21.15.196:0/3321547657 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9e6414eb20 con 0x7f9e6415b8c0 2024-04-28T13:33:41.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41.098+0000 7f9e60c00640 1 -- 172.21.15.196:0/3321547657 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) ==== 53+0+73 (secure 0 0 0) 0x7f9e5c05a530 con 0x7f9e6415b8c0 2024-04-28T13:33:41.097 INFO:teuthology.orchestra.run.smithi196.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-28T13:33:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41.100+0000 7f9e6b000640 1 -- 172.21.15.196:0/3321547657 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9e4006c980 msgr2=0x7f9e4006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41.100+0000 7f9e6b000640 1 --2- 172.21.15.196:0/3321547657 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9e4006c980 0x7f9e4006ee40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9e54008f80 tx=0x7f9e540084d0 comp rx=0 tx=0).stop 2024-04-28T13:33:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41.100+0000 7f9e6b000640 1 -- 172.21.15.196:0/3321547657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6415b8c0 msgr2=0x7f9e64158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41.100+0000 7f9e6b000640 1 --2- 172.21.15.196:0/3321547657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6415b8c0 0x7f9e64158ad0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00d800 tx=0x7f9e5c00dcc0 comp rx=0 tx=0).stop 2024-04-28T13:33:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41.100+0000 7f9e6b000640 1 -- 172.21.15.196:0/3321547657 shutdown_connections 2024-04-28T13:33:41.100 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41.100+0000 7f9e6b000640 1 --2- 172.21.15.196:0/3321547657 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9e4006c980 0x7f9e4006ee40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41.100+0000 7f9e6b000640 1 --2- 172.21.15.196:0/3321547657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e6415b8c0 0x7f9e64158ad0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41.100+0000 7f9e6b000640 1 --2- 172.21.15.196:0/3321547657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e6415b4e0 0x7f9e64156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41.100+0000 7f9e6b000640 1 --2- 172.21.15.196:0/3321547657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9e64073820 0x7f9e6414abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41.100+0000 7f9e6b000640 1 -- 172.21.15.196:0/3321547657 >> 172.21.15.196:0/3321547657 conn(0x7f9e640d0d30 msgr2=0x7f9e640b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41.100+0000 7f9e6b000640 1 -- 172.21.15.196:0/3321547657 shutdown_connections 2024-04-28T13:33:41.101 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41.101+0000 7f9e6b000640 1 -- 172.21.15.196:0/3321547657 wait complete. 2024-04-28T13:33:41.118 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-28T13:33:41.118 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T13:33:41.118 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-04-28T13:33:41.118 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-28T13:33:41.118 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-28T13:33:41.118 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-28T13:33:41.118 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-28T13:33:41.118 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T13:33:41.118 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-28T13:33:41.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:41.119 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-04-28T13:33:41.167 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:33:41.167 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:33:41.167 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:33:41.167 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:33:41.167 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-28T13:33:41.167 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:33:41.167 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:33:41.167 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-04-28T13:33:41.167 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42529sec preferred_lft 42529sec 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 1e:4d:24:54:f5:c3 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:33:41.168 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::3cb7:a8ff:fe55:18c0/64 scope link 2024-04-28T13:33:41.169 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:33:41.169 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:33:41.169 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether c6:b9:8c:18:7a:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T13:33:41.169 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::c4b9:8cff:fe18:7aec/64 scope link 2024-04-28T13:33:41.169 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:33:41.169 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:33:41.169 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 1e:4d:24:54:f5:c3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T13:33:41.169 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::1c4d:24ff:fe54:f5c3/64 scope link 2024-04-28T13:33:41.169 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:33:41.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:41.169 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T13:33:41.286 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-04-28T13:33:41.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:41.290 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-04-28T13:33:41.345 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-04-28T13:33:41.345 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T13:33:41.345 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T13:33:41.345 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T13:33:41.345 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T13:33:41.345 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:33:41.346 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-28T13:33:41.346 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T13:33:41.346 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T13:33:41.346 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T13:33:41.346 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-28T13:33:41.450 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:33:41.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:33:41.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:41.555 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-28T13:33:41.605 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-28T13:33:41.605 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-28T13:33:41.606 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-28T13:33:41.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:41.606 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-04-28T13:33:41.687 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:33:41.687 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-28T13:33:41.738 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-28T13:33:41.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:33:41.738 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-28T13:33:41.790 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-28T13:33:41.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:41.791 DEBUG:teuthology.orchestra.run.smithi196:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:33:41.914 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:33:41.914 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-28T13:33:41.914 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:33:41.935 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:33:41.936 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-28T13:33:41.936 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:33:41.936 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:33:41.936 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:33:41.967 INFO:teuthology.orchestra.run.smithi196.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-28T13:33:41.967 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-28T13:33:41.967 INFO:teuthology.orchestra.run.smithi196.stdout:invalid new device string format 2024-04-28T13:33:41.967 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:33:41.967 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:33:41.967 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: resolved to: "172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-28T13:33:41.967 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-28T13:33:41.967 INFO:teuthology.orchestra.run.smithi196.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=270aea32-2e16-4a97-85f7-aeae39bc8c76" will pass to kernel 2024-04-28T13:33:41.969 INFO:teuthology.orchestra.run.smithi196.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-28T13:33:41.969 INFO:teuthology.orchestra.run.smithi196.stdout: You just mounted a file system that supports labels which does not 2024-04-28T13:33:41.969 INFO:teuthology.orchestra.run.smithi196.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-28T13:33:41.969 INFO:teuthology.orchestra.run.smithi196.stdout: applications will generate AVC messages and not be allowed access to 2024-04-28T13:33:41.969 INFO:teuthology.orchestra.run.smithi196.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-28T13:33:41.974 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-28T13:33:41.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:41.974 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-28T13:33:42.050 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:33:42.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:33:42.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:33:42.187 INFO:teuthology.orchestra.run.smithi196.stdout:client5107 2024-04-28T13:33:42.188 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:33:42.188 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:33:42.236 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T13:33:42.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.470+0000 7f1e77e00640 1 Processor -- start 2024-04-28T13:33:42.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.470+0000 7f1e77e00640 1 -- start start 2024-04-28T13:33:42.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.472+0000 7f1e77e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1e7013ff70 0x7f1e70146080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:42.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.472+0000 7f1e77e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1e70140350 0x7f1e701465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:42.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.472+0000 7f1e77e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e700d05b0 con 0x7f1e700cf810 2024-04-28T13:33:42.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.472+0000 7f1e77e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1e700d0890 con 0x7f1e70140350 2024-04-28T13:33:42.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.472+0000 7f1e77e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f1e7006e450 con 0x7f1e7013ff70 2024-04-28T13:33:42.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.472+0000 7f1e74a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1e70140350 0x7f1e701465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:42.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.472+0000 7f1e75e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f1e700cf810 0x7f1e700cfcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:42878/0 (socket says 172.21.15.196:42878) 2024-04-28T13:33:42.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.472+0000 7f1e75e00640 1 -- 172.21.15.196:0/4036602259 learned_addr learned my addr 172.21.15.196:0/4036602259 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:42.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.473+0000 7f1e74a00640 1 -- 172.21.15.196:0/4036602259 >> v2:172.21.15.203:3300/0 conn(0x7f1e7013ff70 msgr2=0x7f1e70146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:42.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.473+0000 7f1e74a00640 1 --2- 172.21.15.196:0/4036602259 >> v2:172.21.15.203:3300/0 conn(0x7f1e7013ff70 0x7f1e70146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:42.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.473+0000 7f1e74a00640 1 -- 172.21.15.196:0/4036602259 >> v1:172.21.15.196:6789/0 conn(0x7f1e700cf810 legacy=0x7f1e700cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:42.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.473+0000 7f1e74a00640 1 -- 172.21.15.196:0/4036602259 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7006e730 con 0x7f1e70140350 2024-04-28T13:33:42.473 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.473+0000 7f1e6fe00640 1 -- 172.21.15.196:0/4036602259 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 735793269 0 0) 0x7f1e700d05b0 con 0x7f1e700cf810 2024-04-28T13:33:42.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.474+0000 7f1e74a00640 1 --2- 172.21.15.196:0/4036602259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e70140350 0x7f1e701465c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1e70074200 tx=0x7f1e5c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d706e4eeaf73d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:42.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.474+0000 7f1e6fe00640 1 -- 172.21.15.196:0/4036602259 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c007740 con 0x7f1e70140350 2024-04-28T13:33:42.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.474+0000 7f1e6fe00640 1 -- 172.21.15.196:0/4036602259 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e5c0078e0 con 0x7f1e70140350 2024-04-28T13:33:42.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.474+0000 7f1e6fe00640 1 -- 172.21.15.196:0/4036602259 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c007cb0 con 0x7f1e70140350 2024-04-28T13:33:42.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.475+0000 7f1e77e00640 1 -- 172.21.15.196:0/4036602259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e70140350 msgr2=0x7f1e701465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:42.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.475+0000 7f1e77e00640 1 --2- 172.21.15.196:0/4036602259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e70140350 0x7f1e701465c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1e70074200 tx=0x7f1e5c02a5f0 comp rx=0 tx=0).stop 2024-04-28T13:33:42.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.475+0000 7f1e77e00640 1 -- 172.21.15.196:0/4036602259 shutdown_connections 2024-04-28T13:33:42.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.475+0000 7f1e77e00640 1 --2- 172.21.15.196:0/4036602259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e70140350 0x7f1e701465c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:42.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.475+0000 7f1e77e00640 1 --2- 172.21.15.196:0/4036602259 >> v2:172.21.15.203:3300/0 conn(0x7f1e7013ff70 0x7f1e70146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:42.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.475+0000 7f1e77e00640 1 -- 172.21.15.196:0/4036602259 >> 172.21.15.196:0/4036602259 conn(0x7f1e700d0d30 msgr2=0x7f1e700d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:42.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.475+0000 7f1e77e00640 1 -- 172.21.15.196:0/4036602259 shutdown_connections 2024-04-28T13:33:42.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.475+0000 7f1e77e00640 1 -- 172.21.15.196:0/4036602259 wait complete. 2024-04-28T13:33:42.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.477+0000 7f1e77e00640 1 Processor -- start 2024-04-28T13:33:42.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.477+0000 7f1e77e00640 1 -- start start 2024-04-28T13:33:42.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.478+0000 7f1e77e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e700cf810 0x7f1e700b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:42.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.478+0000 7f1e77e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e7013ff70 0x7f1e700b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:42.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.478+0000 7f1e77e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e70140350 0x7f1e700b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:42.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.478+0000 7f1e77e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7006efe0 con 0x7f1e700cf810 2024-04-28T13:33:42.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.478+0000 7f1e77e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7006ee60 con 0x7f1e7013ff70 2024-04-28T13:33:42.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.478+0000 7f1e77e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1e7006f160 con 0x7f1e70140350 2024-04-28T13:33:42.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.478+0000 7f1e75400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e700cf810 0x7f1e700b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:42.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.478+0000 7f1e75400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e700cf810 0x7f1e700b5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:51312/0 (socket says 172.21.15.196:51312) 2024-04-28T13:33:42.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.478+0000 7f1e75400640 1 -- 172.21.15.196:0/468318938 learned_addr learned my addr 172.21.15.196:0/468318938 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:42.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.478+0000 7f1e75400640 1 -- 172.21.15.196:0/468318938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e70140350 msgr2=0x7f1e700b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:42.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.478+0000 7f1e75400640 1 --2- 172.21.15.196:0/468318938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e70140350 0x7f1e700b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:42.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.478+0000 7f1e75400640 1 -- 172.21.15.196:0/468318938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e7013ff70 msgr2=0x7f1e700b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:42.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.478+0000 7f1e75400640 1 --2- 172.21.15.196:0/468318938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e7013ff70 0x7f1e700b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:42.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.478+0000 7f1e75400640 1 -- 172.21.15.196:0/468318938 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e70072bc0 con 0x7f1e700cf810 2024-04-28T13:33:42.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.479+0000 7f1e75400640 1 --2- 172.21.15.196:0/468318938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e700cf810 0x7f1e700b5ac0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000b4a0 tx=0x7f1e6000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:42.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.479+0000 7f1e6d600640 1 -- 172.21.15.196:0/468318938 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60014070 con 0x7f1e700cf810 2024-04-28T13:33:42.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.479+0000 7f1e6d600640 1 -- 172.21.15.196:0/468318938 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e60010610 con 0x7f1e700cf810 2024-04-28T13:33:42.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.479+0000 7f1e77e00640 1 -- 172.21.15.196:0/468318938 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e70072e40 con 0x7f1e700cf810 2024-04-28T13:33:42.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.479+0000 7f1e6d600640 1 -- 172.21.15.196:0/468318938 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60010a00 con 0x7f1e700cf810 2024-04-28T13:33:42.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.479+0000 7f1e77e00640 1 -- 172.21.15.196:0/468318938 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e700736e0 con 0x7f1e700cf810 2024-04-28T13:33:42.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.480+0000 7f1e77e00640 1 -- 172.21.15.196:0/468318938 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e70073230 con 0x7f1e700cf810 2024-04-28T13:33:42.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.481+0000 7f1e6d600640 1 -- 172.21.15.196:0/468318938 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f1e60017070 con 0x7f1e700cf810 2024-04-28T13:33:42.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.482+0000 7f1e6d600640 1 --2- 172.21.15.196:0/468318938 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1e5006c980 0x7f1e5006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:42.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.483+0000 7f1e6d600640 1 -- 172.21.15.196:0/468318938 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f1e6008b6f0 con 0x7f1e700cf810 2024-04-28T13:33:42.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.483+0000 7f1e74a00640 1 --2- 172.21.15.196:0/468318938 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1e5006c980 0x7f1e5006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:42.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.484+0000 7f1e74a00640 1 --2- 172.21.15.196:0/468318938 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1e5006c980 0x7f1e5006ee40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c034000 tx=0x7f1e5c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:42.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.487+0000 7f1e6d600640 1 -- 172.21.15.196:0/468318938 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e70072bc0 con 0x7f1e700cf810 2024-04-28T13:33:42.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.727+0000 7f1e77e00640 1 -- 172.21.15.196:0/468318938 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1e700b45e0 con 0x7f1e700cf810 2024-04-28T13:33:42.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.728+0000 7f1e6d600640 1 -- 172.21.15.196:0/468318938 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) ==== 53+0+73 (secure 0 0 0) 0x7f1e60059a30 con 0x7f1e700cf810 2024-04-28T13:33:42.728 INFO:teuthology.orchestra.run.smithi196.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-28T13:33:42.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.731+0000 7f1e77e00640 1 -- 172.21.15.196:0/468318938 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1e5006c980 msgr2=0x7f1e5006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:42.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.731+0000 7f1e77e00640 1 --2- 172.21.15.196:0/468318938 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1e5006c980 0x7f1e5006ee40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c034000 tx=0x7f1e5c02e000 comp rx=0 tx=0).stop 2024-04-28T13:33:42.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.731+0000 7f1e77e00640 1 -- 172.21.15.196:0/468318938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e700cf810 msgr2=0x7f1e700b5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:42.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.731+0000 7f1e77e00640 1 --2- 172.21.15.196:0/468318938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e700cf810 0x7f1e700b5ac0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000b4a0 tx=0x7f1e6000b960 comp rx=0 tx=0).stop 2024-04-28T13:33:42.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.731+0000 7f1e77e00640 1 -- 172.21.15.196:0/468318938 shutdown_connections 2024-04-28T13:33:42.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.731+0000 7f1e77e00640 1 --2- 172.21.15.196:0/468318938 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f1e5006c980 0x7f1e5006ee40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:42.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.731+0000 7f1e77e00640 1 --2- 172.21.15.196:0/468318938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e70140350 0x7f1e700b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:42.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.731+0000 7f1e77e00640 1 --2- 172.21.15.196:0/468318938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e7013ff70 0x7f1e700b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:42.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.731+0000 7f1e77e00640 1 --2- 172.21.15.196:0/468318938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e700cf810 0x7f1e700b5ac0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:42.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.731+0000 7f1e77e00640 1 -- 172.21.15.196:0/468318938 >> 172.21.15.196:0/468318938 conn(0x7f1e700d0d30 msgr2=0x7f1e700b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:42.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.731+0000 7f1e77e00640 1 -- 172.21.15.196:0/468318938 shutdown_connections 2024-04-28T13:33:42.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:42.731+0000 7f1e77e00640 1 -- 172.21.15.196:0/468318938 wait complete. 2024-04-28T13:33:42.750 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-28T13:33:42.750 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T13:33:42.750 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2024-04-28T13:33:42.750 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-28T13:33:42.750 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-28T13:33:42.750 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-28T13:33:42.750 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-28T13:33:42.750 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T13:33:42.750 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-28T13:33:42.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:42.751 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2024-04-28T13:33:42.810 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T13:33:42.810 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T13:33:42.810 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T13:33:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:33:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-28T13:33:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:33:42.811 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:33:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:33:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f0 2024-04-28T13:33:42.811 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:33:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:33:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f1 2024-04-28T13:33:42.811 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T13:33:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:33:42.811 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T13:33:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:33:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T13:33:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42496sec preferred_lft 42496sec 2024-04-28T13:33:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ec4:7aff:fe8f:4a23/64 scope link 2024-04-28T13:33:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:33:42.812 INFO:teuthology.orchestra.run.smithi203.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T13:33:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 1e:04:67:22:4e:82 brd ff:ff:ff:ff:ff:ff 2024-04-28T13:33:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-28T13:33:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:33:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::4e3:c8ff:fea0:8dce/64 scope link 2024-04-28T13:33:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:33:42.812 INFO:teuthology.orchestra.run.smithi203.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:33:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 1e:04:67:22:4e:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T13:33:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::1c04:67ff:fe22:4e82/64 scope link 2024-04-28T13:33:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:33:42.812 INFO:teuthology.orchestra.run.smithi203.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T13:33:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ae:7c:61:1f:29:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-28T13:33:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ac7c:61ff:fe1f:297b/64 scope link 2024-04-28T13:33:42.813 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-28T13:33:42.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:42.814 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T13:33:42.896 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-28T13:33:42.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:42.901 DEBUG:teuthology.orchestra.run.smithi203:> route 2024-04-28T13:33:42.954 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2024-04-28T13:33:42.954 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T13:33:42.954 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T13:33:42.954 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T13:33:42.954 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T13:33:42.955 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T13:33:42.955 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-28T13:33:42.955 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T13:33:42.955 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T13:33:42.955 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T13:33:42.956 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-28T13:33:43.061 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:33:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T13:33:43.155 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:33:43.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:43.165 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-28T13:33:43.211 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-28T13:33:43.211 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-28T13:33:43.212 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-28T13:33:43.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:43.212 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2024-04-28T13:33:43.295 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:33:43.295 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-28T13:33:43.348 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-28T13:33:43.349 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T13:33:43.349 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-28T13:33:43.401 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-28T13:33:43.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:43.401 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:33:43.528 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:33:43.528 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-28T13:33:43.528 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:33:43.551 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:33:43.551 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-28T13:33:43.551 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:33:43.551 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:33:43.551 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:33:43.579 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-28T13:33:43.579 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-28T13:33:43.580 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-28T13:33:43.580 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-28T13:33:43.580 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-28T13:33:43.580 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-28T13:33:43.580 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.196:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-28T13:33:43.580 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=270aea32-2e16-4a97-85f7-aeae39bc8c76" will pass to kernel 2024-04-28T13:33:43.583 INFO:teuthology.orchestra.run.smithi203.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-28T13:33:43.583 INFO:teuthology.orchestra.run.smithi203.stdout: You just mounted a file system that supports labels which does not 2024-04-28T13:33:43.583 INFO:teuthology.orchestra.run.smithi203.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-28T13:33:43.583 INFO:teuthology.orchestra.run.smithi203.stdout: applications will generate AVC messages and not be allowed access to 2024-04-28T13:33:43.583 INFO:teuthology.orchestra.run.smithi203.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-28T13:33:43.586 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-28T13:33:43.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:33:43.587 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-28T13:33:43.654 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T13:33:43.753 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:33:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T13:33:43.758 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-28T13:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T13:33:43.767 INFO:teuthology.orchestra.run.smithi203.stdout:client4820 2024-04-28T13:33:43.767 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:33:43.767 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:33:43.816 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds debug_mds 25 2024-04-28T13:33:44.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.029+0000 7fa0ba400640 1 Processor -- start 2024-04-28T13:33:44.029 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.029+0000 7fa0ba400640 1 -- start start 2024-04-28T13:33:44.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.031+0000 7fa0ba400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b40d3c10 0x7fa0b40b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.031+0000 7fa0ba400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa0b40b8370 0x7fa0b40d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.031+0000 7fa0ba400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b40cfb80 con 0x7fa0b40d3830 2024-04-28T13:33:44.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.031+0000 7fa0ba400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b40d2ee0 con 0x7fa0b40b8370 2024-04-28T13:33:44.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.031+0000 7fa0ba400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b40d31c0 con 0x7fa0b40d3c10 2024-04-28T13:33:44.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.031+0000 7fa0b9000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa0b40d3830 0x7fa0b40d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:42892/0 (socket says 172.21.15.196:42892) 2024-04-28T13:33:44.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.031+0000 7fa0b9000640 1 -- 172.21.15.196:0/176628362 learned_addr learned my addr 172.21.15.196:0/176628362 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:44.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3e00640 1 --2- 172.21.15.196:0/176628362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b40d3c10 0x7fa0b40b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b9a00640 1 --2- 172.21.15.196:0/176628362 >> v2:172.21.15.203:3300/0 conn(0x7fa0b40b8370 0x7fa0b40d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 -- 172.21.15.196:0/176628362 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 14937659 0 0) 0x7fa0a4001110 con 0x7fa0b40d3830 2024-04-28T13:33:44.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 -- 172.21.15.196:0/176628362 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa098003520 con 0x7fa0b40d3830 2024-04-28T13:33:44.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 -- 172.21.15.196:0/176628362 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3725397084 0 0) 0x7fa098003520 con 0x7fa0b40d3830 2024-04-28T13:33:44.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 -- 172.21.15.196:0/176628362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b40d3c10 msgr2=0x7fa0b40b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:44.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 --2- 172.21.15.196:0/176628362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b40d3c10 0x7fa0b40b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 -- 172.21.15.196:0/176628362 >> v2:172.21.15.203:3300/0 conn(0x7fa0b40b8370 msgr2=0x7fa0b40d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:44.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 --2- 172.21.15.196:0/176628362 >> v2:172.21.15.203:3300/0 conn(0x7fa0b40b8370 0x7fa0b40d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 -- 172.21.15.196:0/176628362 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b40cfe00 con 0x7fa0b40d3830 2024-04-28T13:33:44.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 -- 172.21.15.196:0/176628362 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fa0a4001840 con 0x7fa0b40d3830 2024-04-28T13:33:44.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 -- 172.21.15.196:0/176628362 >> v1:172.21.15.196:6789/0 conn(0x7fa0b40d3830 legacy=0x7fa0b40d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:44.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 --2- 172.21.15.196:0/176628362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b40d3c10 0x7fa0980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 --2- 172.21.15.196:0/176628362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa098003fe0 0x7fa098004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 --2- 172.21.15.196:0/176628362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa098004d40 0x7fa0980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 -- 172.21.15.196:0/176628362 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b40d2ee0 con 0x7fa098004d40 2024-04-28T13:33:44.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 -- 172.21.15.196:0/176628362 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b40d31c0 con 0x7fa098003fe0 2024-04-28T13:33:44.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.032+0000 7fa0b3400640 1 -- 172.21.15.196:0/176628362 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa09800bac0 con 0x7fa0b40d3c10 2024-04-28T13:33:44.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.033+0000 7fa0b9a00640 1 --2- 172.21.15.196:0/176628362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa098004d40 0x7fa0980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.033+0000 7fa0b3e00640 1 --2- 172.21.15.196:0/176628362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa098003fe0 0x7fa098004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.033+0000 7fa0b9000640 1 --2- 172.21.15.196:0/176628362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b40d3c10 0x7fa0980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.033+0000 7fa0b9a00640 1 -- 172.21.15.196:0/176628362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b40d3c10 msgr2=0x7fa0980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:44.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.033+0000 7fa0b9a00640 1 --2- 172.21.15.196:0/176628362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b40d3c10 0x7fa0980039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.033+0000 7fa0b9a00640 1 -- 172.21.15.196:0/176628362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa098003fe0 msgr2=0x7fa098004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:44.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.033+0000 7fa0b9a00640 1 --2- 172.21.15.196:0/176628362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa098003fe0 0x7fa098004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.033+0000 7fa0b9a00640 1 -- 172.21.15.196:0/176628362 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa098003520 con 0x7fa098004d40 2024-04-28T13:33:44.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.034+0000 7fa0b9a00640 1 --2- 172.21.15.196:0/176628362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa098004d40 0x7fa0980051c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac007330 tx=0x7fa098005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f960f814b4fe4702 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:44.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.034+0000 7fa0b3400640 1 -- 172.21.15.196:0/176628362 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac009be0 con 0x7fa098004d40 2024-04-28T13:33:44.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.034+0000 7fa0b3400640 1 -- 172.21.15.196:0/176628362 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ac036600 con 0x7fa098004d40 2024-04-28T13:33:44.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.034+0000 7fa0b3400640 1 -- 172.21.15.196:0/176628362 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac0369d0 con 0x7fa098004d40 2024-04-28T13:33:44.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.035+0000 7fa0ba400640 1 -- 172.21.15.196:0/176628362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa098004d40 msgr2=0x7fa0980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:44.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.035+0000 7fa0ba400640 1 --2- 172.21.15.196:0/176628362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa098004d40 0x7fa0980051c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac007330 tx=0x7fa098005830 comp rx=0 tx=0).stop 2024-04-28T13:33:44.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.035+0000 7fa0b9a00640 1 -- 172.21.15.196:0/176628362 reap_dead start 2024-04-28T13:33:44.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.035+0000 7fa0ba400640 1 -- 172.21.15.196:0/176628362 shutdown_connections 2024-04-28T13:33:44.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.035+0000 7fa0ba400640 1 -- 172.21.15.196:0/176628362 >> 172.21.15.196:0/176628362 conn(0x7fa0b40d4fe0 msgr2=0x7fa0b40d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:44.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.035+0000 7fa0ba400640 1 -- 172.21.15.196:0/176628362 shutdown_connections 2024-04-28T13:33:44.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.035+0000 7fa0ba400640 1 -- 172.21.15.196:0/176628362 wait complete. 2024-04-28T13:33:44.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.037+0000 7fa0ba400640 1 Processor -- start 2024-04-28T13:33:44.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.037+0000 7fa0ba400640 1 -- start start 2024-04-28T13:33:44.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.038+0000 7fa0ba400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa098004d40 0x7fa0b40ba030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.038+0000 7fa0ba400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b40cc0b0 0x7fa0b4163120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.038+0000 7fa0ba400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0b41492e0 0x7fa0b4149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.038+0000 7fa0ba400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b40d2bb0 con 0x7fa098004d40 2024-04-28T13:33:44.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.038+0000 7fa0ba400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b40d5d90 con 0x7fa0b41492e0 2024-04-28T13:33:44.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.038+0000 7fa0ba400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b40b7e30 con 0x7fa0b40cc0b0 2024-04-28T13:33:44.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.038+0000 7fa0b9000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa098004d40 0x7fa0b40ba030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.038+0000 7fa0b9000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa098004d40 0x7fa0b40ba030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:51332/0 (socket says 172.21.15.196:51332) 2024-04-28T13:33:44.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.038+0000 7fa0b9000640 1 -- 172.21.15.196:0/1239834794 learned_addr learned my addr 172.21.15.196:0/1239834794 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:44.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.039+0000 7fa0b9a00640 1 --2- 172.21.15.196:0/1239834794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0b41492e0 0x7fa0b4149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.039+0000 7fa0b9000640 1 -- 172.21.15.196:0/1239834794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b40cc0b0 msgr2=0x7fa0b4163120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:44.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.039+0000 7fa0b3e00640 1 --2- 172.21.15.196:0/1239834794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b40cc0b0 0x7fa0b4163120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.039+0000 7fa0b9000640 1 --2- 172.21.15.196:0/1239834794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b40cc0b0 0x7fa0b4163120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.039+0000 7fa0b9000640 1 -- 172.21.15.196:0/1239834794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0b41492e0 msgr2=0x7fa0b4149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:44.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.039+0000 7fa0b9000640 1 --2- 172.21.15.196:0/1239834794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0b41492e0 0x7fa0b4149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.039+0000 7fa0b9000640 1 -- 172.21.15.196:0/1239834794 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b40ba570 con 0x7fa098004d40 2024-04-28T13:33:44.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.039+0000 7fa0b9a00640 1 --2- 172.21.15.196:0/1239834794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0b41492e0 0x7fa0b4149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:44.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.039+0000 7fa0b9000640 1 --2- 172.21.15.196:0/1239834794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa098004d40 0x7fa0b40ba030 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400d8f0 tx=0x7fa0a400ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:44.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.039+0000 7fa0b0c00640 1 -- 172.21.15.196:0/1239834794 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4019070 con 0x7fa098004d40 2024-04-28T13:33:44.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.039+0000 7fa0b0c00640 1 -- 172.21.15.196:0/1239834794 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a40074e0 con 0x7fa098004d40 2024-04-28T13:33:44.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.039+0000 7fa0b0c00640 1 -- 172.21.15.196:0/1239834794 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4004a20 con 0x7fa098004d40 2024-04-28T13:33:44.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.039+0000 7fa0ba400640 1 -- 172.21.15.196:0/1239834794 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b40cc620 con 0x7fa098004d40 2024-04-28T13:33:44.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.040+0000 7fa0ba400640 1 -- 172.21.15.196:0/1239834794 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b40ccbe0 con 0x7fa098004d40 2024-04-28T13:33:44.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.041+0000 7fa0ba400640 1 -- 172.21.15.196:0/1239834794 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b4050730 con 0x7fa098004d40 2024-04-28T13:33:44.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.042+0000 7fa0b0c00640 1 -- 172.21.15.196:0/1239834794 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa0a401c070 con 0x7fa098004d40 2024-04-28T13:33:44.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.042+0000 7fa0b0c00640 1 --2- 172.21.15.196:0/1239834794 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa09406c980 0x7fa09406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.043+0000 7fa0b0c00640 1 -- 172.21.15.196:0/1239834794 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fa0a408b910 con 0x7fa098004d40 2024-04-28T13:33:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.043+0000 7fa0b3e00640 1 --2- 172.21.15.196:0/1239834794 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa09406c980 0x7fa09406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.045+0000 7fa0b3e00640 1 --2- 172.21.15.196:0/1239834794 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa09406c980 0x7fa09406ee40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa0b415fd40 tx=0x7fa0a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:44.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.047+0000 7fa0b0c00640 1 -- 172.21.15.196:0/1239834794 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa0b40cfb80 con 0x7fa098004d40 2024-04-28T13:33:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.250+0000 7fa0ba400640 1 -- 172.21.15.196:0/1239834794 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fa0b40d17d0 con 0x7fa098004d40 2024-04-28T13:33:44.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.257+0000 7fa0b0c00640 1 -- 172.21.15.196:0/1239834794 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v17)=0 v17) ==== 108+0+0 (secure 0 0 0) 0x7fa0a4059bd0 con 0x7fa098004d40 2024-04-28T13:33:44.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.260+0000 7fa0ba400640 1 -- 172.21.15.196:0/1239834794 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa09406c980 msgr2=0x7fa09406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:44.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.260+0000 7fa0ba400640 1 --2- 172.21.15.196:0/1239834794 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa09406c980 0x7fa09406ee40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa0b415fd40 tx=0x7fa0a0008040 comp rx=0 tx=0).stop 2024-04-28T13:33:44.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.260+0000 7fa0ba400640 1 -- 172.21.15.196:0/1239834794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa098004d40 msgr2=0x7fa0b40ba030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:44.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.260+0000 7fa0ba400640 1 --2- 172.21.15.196:0/1239834794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa098004d40 0x7fa0b40ba030 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400d8f0 tx=0x7fa0a400ddb0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.260+0000 7fa0ba400640 1 -- 172.21.15.196:0/1239834794 shutdown_connections 2024-04-28T13:33:44.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.260+0000 7fa0ba400640 1 --2- 172.21.15.196:0/1239834794 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa09406c980 0x7fa09406ee40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.260+0000 7fa0ba400640 1 --2- 172.21.15.196:0/1239834794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0b41492e0 0x7fa0b4149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.260+0000 7fa0ba400640 1 --2- 172.21.15.196:0/1239834794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b40cc0b0 0x7fa0b4163120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.260+0000 7fa0ba400640 1 --2- 172.21.15.196:0/1239834794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa098004d40 0x7fa0b40ba030 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.260+0000 7fa0ba400640 1 -- 172.21.15.196:0/1239834794 >> 172.21.15.196:0/1239834794 conn(0x7fa0b40d4fe0 msgr2=0x7fa0b406e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:44.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.260+0000 7fa0ba400640 1 -- 172.21.15.196:0/1239834794 shutdown_connections 2024-04-28T13:33:44.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.260+0000 7fa0ba400640 1 -- 172.21.15.196:0/1239834794 wait complete. 2024-04-28T13:33:44.279 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_quiesce_splitauth true 2024-04-28T13:33:44.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.480+0000 7fec0f400640 1 Processor -- start 2024-04-28T13:33:44.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.480+0000 7fec0f400640 1 -- start start 2024-04-28T13:33:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.482+0000 7fec0f400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec080b65f0 0x7fec080b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.482+0000 7fec0f400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fec08163b40 0x7fec0806f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.482+0000 7fec0f400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fec0806f880 0x7fec08167450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.482+0000 7fec0f400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fec08073440 con 0x7fec0806f880 2024-04-28T13:33:44.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.482+0000 7fec0f400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fec080b6f10 con 0x7fec08163b40 2024-04-28T13:33:44.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.482+0000 7fec0f400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fec080723a0 con 0x7fec080b65f0 2024-04-28T13:33:44.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.482+0000 7fec0d400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fec0806f880 0x7fec08167450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.482+0000 7fec0d400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fec0806f880 0x7fec08167450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:51346/0 (socket says 172.21.15.196:51346) 2024-04-28T13:33:44.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.482+0000 7fec0d400640 1 -- 172.21.15.196:0/4210079781 learned_addr learned my addr 172.21.15.196:0/4210079781 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:44.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.482+0000 7fec0ca00640 1 --2- 172.21.15.196:0/4210079781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec080b65f0 0x7fec080b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.483+0000 7fec07e00640 1 --2- 172.21.15.196:0/4210079781 >> v2:172.21.15.203:3300/0 conn(0x7fec08163b40 0x7fec0806f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.483+0000 7fec0d400640 1 -- 172.21.15.196:0/4210079781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec080b65f0 msgr2=0x7fec080b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:44.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.483+0000 7fec0d400640 1 --2- 172.21.15.196:0/4210079781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec080b65f0 0x7fec080b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.483+0000 7fec0d400640 1 -- 172.21.15.196:0/4210079781 >> v2:172.21.15.203:3300/0 conn(0x7fec08163b40 msgr2=0x7fec0806f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:44.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.483+0000 7fec0d400640 1 --2- 172.21.15.196:0/4210079781 >> v2:172.21.15.203:3300/0 conn(0x7fec08163b40 0x7fec0806f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.483+0000 7fec0d400640 1 -- 172.21.15.196:0/4210079781 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec08072680 con 0x7fec0806f880 2024-04-28T13:33:44.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.483+0000 7fec07e00640 1 --2- 172.21.15.196:0/4210079781 >> v2:172.21.15.203:3300/0 conn(0x7fec08163b40 0x7fec0806f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:44.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.484+0000 7fec0d400640 1 --2- 172.21.15.196:0/4210079781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0806f880 0x7fec08167450 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fec080b3c10 tx=0x7fec0002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa34646802c371b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:44.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.484+0000 7fec07400640 1 -- 172.21.15.196:0/4210079781 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00036070 con 0x7fec0806f880 2024-04-28T13:33:44.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.484+0000 7fec07400640 1 -- 172.21.15.196:0/4210079781 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec00031600 con 0x7fec0806f880 2024-04-28T13:33:44.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.484+0000 7fec07400640 1 -- 172.21.15.196:0/4210079781 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec000319d0 con 0x7fec0806f880 2024-04-28T13:33:44.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.485+0000 7fec0f400640 1 -- 172.21.15.196:0/4210079781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0806f880 msgr2=0x7fec08167450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:44.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.485+0000 7fec0f400640 1 --2- 172.21.15.196:0/4210079781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0806f880 0x7fec08167450 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fec080b3c10 tx=0x7fec0002a610 comp rx=0 tx=0).stop 2024-04-28T13:33:44.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.485+0000 7fec0f400640 1 -- 172.21.15.196:0/4210079781 shutdown_connections 2024-04-28T13:33:44.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.485+0000 7fec0f400640 1 --2- 172.21.15.196:0/4210079781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0806f880 0x7fec08167450 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.485+0000 7fec0f400640 1 --2- 172.21.15.196:0/4210079781 >> v2:172.21.15.203:3300/0 conn(0x7fec08163b40 0x7fec0806f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.485+0000 7fec0f400640 1 --2- 172.21.15.196:0/4210079781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec080b65f0 0x7fec080b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.485+0000 7fec0f400640 1 -- 172.21.15.196:0/4210079781 >> 172.21.15.196:0/4210079781 conn(0x7fec080cd8a0 msgr2=0x7fec080cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:44.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.485+0000 7fec0f400640 1 -- 172.21.15.196:0/4210079781 shutdown_connections 2024-04-28T13:33:44.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.485+0000 7fec0f400640 1 -- 172.21.15.196:0/4210079781 wait complete. 2024-04-28T13:33:44.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.487+0000 7fec0f400640 1 Processor -- start 2024-04-28T13:33:44.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.487+0000 7fec0f400640 1 -- start start 2024-04-28T13:33:44.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.488+0000 7fec0f400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec0806f880 0x7fec08166790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.488+0000 7fec0f400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec080b65f0 0x7fec0815e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.488+0000 7fec0f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec08163b40 0x7fec0815b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.488+0000 7fec0f400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec0806fce0 con 0x7fec080b65f0 2024-04-28T13:33:44.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.488+0000 7fec0f400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fec08072c40 con 0x7fec08163b40 2024-04-28T13:33:44.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.488+0000 7fec0f400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fec08167990 con 0x7fec0806f880 2024-04-28T13:33:44.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.488+0000 7fec07e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec080b65f0 0x7fec0815e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.488+0000 7fec07e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec080b65f0 0x7fec0815e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:51348/0 (socket says 172.21.15.196:51348) 2024-04-28T13:33:44.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.488+0000 7fec07e00640 1 -- 172.21.15.196:0/1028279040 learned_addr learned my addr 172.21.15.196:0/1028279040 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:44.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.489+0000 7fec0ca00640 1 --2- 172.21.15.196:0/1028279040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec0806f880 0x7fec08166790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.489+0000 7fec07e00640 1 -- 172.21.15.196:0/1028279040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec0806f880 msgr2=0x7fec08166790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:44.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.489+0000 7fec07e00640 1 --2- 172.21.15.196:0/1028279040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec0806f880 0x7fec08166790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.489+0000 7fec07e00640 1 -- 172.21.15.196:0/1028279040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec08163b40 msgr2=0x7fec0815b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:44.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.489+0000 7fec07e00640 1 --2- 172.21.15.196:0/1028279040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec08163b40 0x7fec0815b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.489+0000 7fec07e00640 1 -- 172.21.15.196:0/1028279040 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0815eec0 con 0x7fec080b65f0 2024-04-28T13:33:44.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.489+0000 7fec0ca00640 1 --2- 172.21.15.196:0/1028279040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec0806f880 0x7fec08166790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:33:44.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.489+0000 7fec07e00640 1 --2- 172.21.15.196:0/1028279040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec080b65f0 0x7fec0815e980 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7febf400c7c0 tx=0x7febf400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:44.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.489+0000 7fec04c00640 1 -- 172.21.15.196:0/1028279040 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4015070 con 0x7fec080b65f0 2024-04-28T13:33:44.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.489+0000 7fec04c00640 1 -- 172.21.15.196:0/1028279040 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf4007820 con 0x7fec080b65f0 2024-04-28T13:33:44.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.489+0000 7fec0f400640 1 -- 172.21.15.196:0/1028279040 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec08166d70 con 0x7fec080b65f0 2024-04-28T13:33:44.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.489+0000 7fec04c00640 1 -- 172.21.15.196:0/1028279040 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4007bf0 con 0x7fec080b65f0 2024-04-28T13:33:44.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.489+0000 7fec0f400640 1 -- 172.21.15.196:0/1028279040 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec0815d930 con 0x7fec080b65f0 2024-04-28T13:33:44.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.491+0000 7fec0f400640 1 -- 172.21.15.196:0/1028279040 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0806a290 con 0x7fec080b65f0 2024-04-28T13:33:44.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.491+0000 7fec04c00640 1 -- 172.21.15.196:0/1028279040 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7febf4018070 con 0x7fec080b65f0 2024-04-28T13:33:44.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.492+0000 7fec04c00640 1 --2- 172.21.15.196:0/1028279040 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7febe806c980 0x7febe806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.493+0000 7fec04c00640 1 -- 172.21.15.196:0/1028279040 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7febf408bca0 con 0x7fec080b65f0 2024-04-28T13:33:44.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.493+0000 7fec0ca00640 1 --2- 172.21.15.196:0/1028279040 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7febe806c980 0x7febe806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.494+0000 7fec0ca00640 1 --2- 172.21.15.196:0/1028279040 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7febe806c980 0x7febe806ee40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7febfc006fd0 tx=0x7febfc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:44.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.497+0000 7fec04c00640 1 -- 172.21.15.196:0/1028279040 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7febf4059e40 con 0x7fec080b65f0 2024-04-28T13:33:44.697 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.697+0000 7fec0f400640 1 -- 172.21.15.196:0/1028279040 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command([{prefix=config set, name=mds_cache_quiesce_splitauth}] v 0) -- 0x7fec080b4fc0 con 0x7fec080b65f0 2024-04-28T13:33:44.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.703+0000 7fec04c00640 1 -- 172.21.15.196:0/1028279040 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_quiesce_splitauth}]=0 v18)=0 v18) ==== 128+0+0 (secure 0 0 0) 0x7febf4059740 con 0x7fec080b65f0 2024-04-28T13:33:44.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.706+0000 7fec0f400640 1 -- 172.21.15.196:0/1028279040 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7febe806c980 msgr2=0x7febe806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:44.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.706+0000 7fec0f400640 1 --2- 172.21.15.196:0/1028279040 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7febe806c980 0x7febe806ee40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7febfc006fd0 tx=0x7febfc008040 comp rx=0 tx=0).stop 2024-04-28T13:33:44.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.706+0000 7fec0f400640 1 -- 172.21.15.196:0/1028279040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec080b65f0 msgr2=0x7fec0815e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:44.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.706+0000 7fec0f400640 1 --2- 172.21.15.196:0/1028279040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec080b65f0 0x7fec0815e980 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7febf400c7c0 tx=0x7febf400cc80 comp rx=0 tx=0).stop 2024-04-28T13:33:44.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.706+0000 7fec0f400640 1 -- 172.21.15.196:0/1028279040 shutdown_connections 2024-04-28T13:33:44.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.706+0000 7fec0f400640 1 --2- 172.21.15.196:0/1028279040 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7febe806c980 0x7febe806ee40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.706+0000 7fec0f400640 1 --2- 172.21.15.196:0/1028279040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec08163b40 0x7fec0815b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.706+0000 7fec0f400640 1 --2- 172.21.15.196:0/1028279040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec080b65f0 0x7fec0815e980 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.706+0000 7fec0f400640 1 --2- 172.21.15.196:0/1028279040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec0806f880 0x7fec08166790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.706+0000 7fec0f400640 1 -- 172.21.15.196:0/1028279040 >> 172.21.15.196:0/1028279040 conn(0x7fec080cd8a0 msgr2=0x7fec080b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:44.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.706+0000 7fec0f400640 1 -- 172.21.15.196:0/1028279040 shutdown_connections 2024-04-28T13:33:44.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.706+0000 7fec0f400640 1 -- 172.21.15.196:0/1028279040 wait complete. 2024-04-28T13:33:44.723 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol_quiesce --mode=777 2024-04-28T13:33:44.922 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.922+0000 7fa6d2e00640 1 Processor -- start 2024-04-28T13:33:44.922 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.922+0000 7fa6d2e00640 1 -- start start 2024-04-28T13:33:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.924+0000 7fa6d2e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6cc0b50c0 0x7fa6cc0d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.924+0000 7fa6d2e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa6cc0cfb40 0x7fa6cc0b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.924+0000 7fa6d2e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6cc0d3900 con 0x7fa6cc1629c0 2024-04-28T13:33:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.924+0000 7fa6d2e00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa6cc0d3be0 con 0x7fa6cc0cfb40 2024-04-28T13:33:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.924+0000 7fa6d2e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa6cc0b5580 con 0x7fa6cc0b50c0 2024-04-28T13:33:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.924+0000 7fa6d1a00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa6cc1629c0 0x7fa6cc0b49b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:42906/0 (socket says 172.21.15.196:42906) 2024-04-28T13:33:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.924+0000 7fa6d1a00640 1 -- 172.21.15.196:0/3627050455 learned_addr learned my addr 172.21.15.196:0/3627050455 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:33:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.924+0000 7fa6d2400640 1 --2- 172.21.15.196:0/3627050455 >> v2:172.21.15.196:3300/0 conn(0x7fa6cc0cfb40 0x7fa6cc0b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.924+0000 7fa6cbe00640 1 -- 172.21.15.196:0/3627050455 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4094303733 0 0) 0x7fa6cc0d3900 con 0x7fa6cc1629c0 2024-04-28T13:33:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.925+0000 7fa6d2400640 1 -- 172.21.15.196:0/3627050455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6cc0b50c0 msgr2=0x7fa6cc0d5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:33:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.925+0000 7fa6d2400640 1 --2- 172.21.15.196:0/3627050455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6cc0b50c0 0x7fa6cc0d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.925+0000 7fa6d2400640 1 -- 172.21.15.196:0/3627050455 >> v1:172.21.15.196:6789/0 conn(0x7fa6cc1629c0 legacy=0x7fa6cc0b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.925+0000 7fa6d2400640 1 -- 172.21.15.196:0/3627050455 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6cc0b4140 con 0x7fa6cc0cfb40 2024-04-28T13:33:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.926+0000 7fa6d2400640 1 --2- 172.21.15.196:0/3627050455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6cc0cfb40 0x7fa6cc0b5aa0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009d90 tx=0x7fa6c402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e611b0538e5b70c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:44.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.926+0000 7fa6cbe00640 1 -- 172.21.15.196:0/3627050455 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4007740 con 0x7fa6cc0cfb40 2024-04-28T13:33:44.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.926+0000 7fa6cbe00640 1 -- 172.21.15.196:0/3627050455 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c40078e0 con 0x7fa6cc0cfb40 2024-04-28T13:33:44.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.926+0000 7fa6cbe00640 1 -- 172.21.15.196:0/3627050455 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4007cb0 con 0x7fa6cc0cfb40 2024-04-28T13:33:44.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.926+0000 7fa6d2e00640 1 -- 172.21.15.196:0/3627050455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6cc0cfb40 msgr2=0x7fa6cc0b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:44.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.926+0000 7fa6d2e00640 1 --2- 172.21.15.196:0/3627050455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6cc0cfb40 0x7fa6cc0b5aa0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009d90 tx=0x7fa6c402a660 comp rx=0 tx=0).stop 2024-04-28T13:33:44.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.926+0000 7fa6d2e00640 1 -- 172.21.15.196:0/3627050455 shutdown_connections 2024-04-28T13:33:44.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.926+0000 7fa6d2e00640 1 --2- 172.21.15.196:0/3627050455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6cc0cfb40 0x7fa6cc0b5aa0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.926+0000 7fa6d2e00640 1 --2- 172.21.15.196:0/3627050455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6cc0b50c0 0x7fa6cc0d5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.926+0000 7fa6d2e00640 1 -- 172.21.15.196:0/3627050455 >> 172.21.15.196:0/3627050455 conn(0x7fa6cc09e570 msgr2=0x7fa6cc0d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:33:44.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.927+0000 7fa6d2e00640 1 -- 172.21.15.196:0/3627050455 shutdown_connections 2024-04-28T13:33:44.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.927+0000 7fa6d2e00640 1 -- 172.21.15.196:0/3627050455 wait complete. 2024-04-28T13:33:44.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.928+0000 7fa6d2e00640 1 Processor -- start 2024-04-28T13:33:44.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.928+0000 7fa6d2e00640 1 -- start start 2024-04-28T13:33:44.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.929+0000 7fa6d2e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6cc0b50c0 0x7fa6cc156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.929+0000 7fa6d2e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6cc0cfb40 0x7fa6cc0d1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.929+0000 7fa6d2e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6cc1629c0 0x7fa6cc0d1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.929+0000 7fa6d2e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc0d4040 con 0x7fa6cc1629c0 2024-04-28T13:33:44.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.929+0000 7fa6d2e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc0d3ec0 con 0x7fa6cc0b50c0 2024-04-28T13:33:44.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.929+0000 7fa6d2e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa6cc0b3880 con 0x7fa6cc0cfb40 2024-04-28T13:33:44.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.930+0000 7fa6d2400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6cc1629c0 0x7fa6cc0d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.930+0000 7fa6d2400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6cc1629c0 0x7fa6cc0d1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:51366/0 (socket says 172.21.15.196:51366) 2024-04-28T13:33:44.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.930+0000 7fa6d2400640 1 -- 172.21.15.196:0/336524048 learned_addr learned my addr 172.21.15.196:0/336524048 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:33:44.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.930+0000 7fa6d1a00640 1 --2- 172.21.15.196:0/336524048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6cc0b50c0 0x7fa6cc156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.930+0000 7fa6d1000640 1 --2- 172.21.15.196:0/336524048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6cc0cfb40 0x7fa6cc0d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.930+0000 7fa6d2400640 1 -- 172.21.15.196:0/336524048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6cc0cfb40 msgr2=0x7fa6cc0d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:44.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.931+0000 7fa6d2400640 1 --2- 172.21.15.196:0/336524048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6cc0cfb40 0x7fa6cc0d1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.931+0000 7fa6d2400640 1 -- 172.21.15.196:0/336524048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6cc0b50c0 msgr2=0x7fa6cc156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:33:44.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.931+0000 7fa6d2400640 1 --2- 172.21.15.196:0/336524048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6cc0b50c0 0x7fa6cc156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:33:44.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.931+0000 7fa6d2400640 1 -- 172.21.15.196:0/336524048 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6cc0cbb30 con 0x7fa6cc1629c0 2024-04-28T13:33:44.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.931+0000 7fa6d2400640 1 --2- 172.21.15.196:0/336524048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6cc1629c0 0x7fa6cc0d1990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009eb0 tx=0x7fa6c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:44.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.931+0000 7fa6c9600640 1 -- 172.21.15.196:0/336524048 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4040070 con 0x7fa6cc1629c0 2024-04-28T13:33:44.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.931+0000 7fa6c9600640 1 -- 172.21.15.196:0/336524048 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c4031c90 con 0x7fa6cc1629c0 2024-04-28T13:33:44.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.931+0000 7fa6d2e00640 1 -- 172.21.15.196:0/336524048 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6cc156db0 con 0x7fa6cc1629c0 2024-04-28T13:33:44.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.931+0000 7fa6c9600640 1 -- 172.21.15.196:0/336524048 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4045680 con 0x7fa6cc1629c0 2024-04-28T13:33:44.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.931+0000 7fa6d2e00640 1 -- 172.21.15.196:0/336524048 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6cc0cb050 con 0x7fa6cc1629c0 2024-04-28T13:33:44.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.932+0000 7fa6d2e00640 1 -- 172.21.15.196:0/336524048 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6cc069ae0 con 0x7fa6cc1629c0 2024-04-28T13:33:44.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.934+0000 7fa6c9600640 1 -- 172.21.15.196:0/336524048 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa6c4039040 con 0x7fa6cc1629c0 2024-04-28T13:33:44.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.935+0000 7fa6c9600640 1 --2- 172.21.15.196:0/336524048 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa6ac06c980 0x7fa6ac06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:33:44.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.935+0000 7fa6c9600640 1 -- 172.21.15.196:0/336524048 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fa6c40ad7e0 con 0x7fa6cc1629c0 2024-04-28T13:33:44.935 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.936+0000 7fa6d1a00640 1 --2- 172.21.15.196:0/336524048 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa6ac06c980 0x7fa6ac06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:33:44.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.937+0000 7fa6d1a00640 1 --2- 172.21.15.196:0/336524048 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa6ac06c980 0x7fa6ac06ee40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0000f60 tx=0x7fa6c0007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:33:44.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:44.939+0000 7fa6c9600640 1 -- 172.21.15.196:0/336524048 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa6cc069ae0 con 0x7fa6cc1629c0 2024-04-28T13:33:45.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:33:45.169+0000 7fa6d2e00640 1 -- 172.21.15.196:0/336524048 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol_quiesce", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fa6cc050730 con 0x7fa6ac06c980 2024-04-28T13:34:10.231 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:34:10.239 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:34:40.362 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:34:40.371 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:35:10.502 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:35:10.511 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:35:40.630 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:35:40.638 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:35:44.819 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-28T13:35:44.821 INFO:tasks.cephfs_test_runner:That a `find` can be executed on a quiesced path. ... ERROR 2024-04-28T13:35:44.822 INFO:tasks.cephfs_test_runner: 2024-04-28T13:35:44.822 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-28T13:35:44.822 INFO:tasks.cephfs_test_runner:ERROR: test_quiesce_find (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-28T13:35:44.822 INFO:tasks.cephfs_test_runner:That a `find` can be executed on a quiesced path. 2024-04-28T13:35:44.822 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-28T13:35:44.822 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-28T13:35:44.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/test_quiesce.py", line 50, in setUp 2024-04-28T13:35:44.822 INFO:tasks.cephfs_test_runner: self.run_ceph_cmd(f'fs subvolume create {self.fs.name} {self.QUIESCE_SUBVOLUME} --mode=777') 2024-04-28T13:35:44.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-04-28T13:35:44.822 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-04-28T13:35:44.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi196 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol_quiesce --mode=777' 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner: 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner:Ran 2 tests in 223.887s 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner: 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-28T13:35:44.823 INFO:tasks.cephfs_test_runner: 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner:ERROR: test_quiesce_find (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner:That a `find` can be executed on a quiesced path. 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/test_quiesce.py", line 50, in setUp 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner: self.run_ceph_cmd(f'fs subvolume create {self.fs.name} {self.QUIESCE_SUBVOLUME} --mode=777') 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait 2024-04-28T13:35:44.824 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-28T13:35:44.825 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-28T13:35:44.825 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-28T13:35:44.825 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi196 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol_quiesce --mode=777' 2024-04-28T13:35:44.825 INFO:tasks.cephfs_test_runner: 2024-04-28T13:35:44.825 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_quiesce_find (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-28T13:35:45.042 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=50b72d1600b845d0924df4aa241bdae9 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_quiesce_find (tasks.cephfs.test_quiesce.TestQuiesce) 2024-04-28T13:35:45.044 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-28T13:35:45.053 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-28T13:35:45.070 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-28T13:35:45.071 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:35:45.071 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:35:45.131 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:35:45.132 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:35:45.185 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:35:45.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.391+0000 7f9861800640 1 Processor -- start 2024-04-28T13:35:45.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.391+0000 7f9861800640 1 -- start start 2024-04-28T13:35:45.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.392+0000 7f9861800640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f985c0d1760 0x7f985c0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:45.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.393+0000 7f9861800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f985c15e380 con 0x7f985c09ec80 2024-04-28T13:35:45.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.393+0000 7f9861800640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f985c0b4b20 con 0x7f985c0b5ee0 2024-04-28T13:35:45.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.393+0000 7f9861800640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f985c0b3f30 con 0x7f985c0d1760 2024-04-28T13:35:45.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.393+0000 7f9860e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f985c09ec80 0x7f985c06df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:55616/0 (socket says 172.21.15.196:55616) 2024-04-28T13:35:45.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.393+0000 7f9860e00640 1 -- 172.21.15.196:0/2014375783 learned_addr learned my addr 172.21.15.196:0/2014375783 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:45.396 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.393+0000 7f985b400640 1 --2- 172.21.15.196:0/2014375783 >> v2:172.21.15.203:3300/0 conn(0x7f985c0d1760 0x7f985c0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:45.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.393+0000 7f985aa00640 1 -- 172.21.15.196:0/2014375783 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4021158500 0 0) 0x7f985c15e380 con 0x7f985c09ec80 2024-04-28T13:35:45.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.393+0000 7f985aa00640 1 -- 172.21.15.196:0/2014375783 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9840003520 con 0x7f985c09ec80 2024-04-28T13:35:45.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.394+0000 7f985aa00640 1 -- 172.21.15.196:0/2014375783 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4258808044 0 0) 0x7f9840003520 con 0x7f985c09ec80 2024-04-28T13:35:45.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.394+0000 7f985aa00640 1 -- 172.21.15.196:0/2014375783 >> v2:172.21.15.203:3300/0 conn(0x7f985c0d1760 msgr2=0x7f985c0b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:45.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.394+0000 7f985aa00640 1 --2- 172.21.15.196:0/2014375783 >> v2:172.21.15.203:3300/0 conn(0x7f985c0d1760 0x7f985c0b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:45.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.394+0000 7f985aa00640 1 -- 172.21.15.196:0/2014375783 >> v1:172.21.15.203:6789/0 conn(0x7f985c0b5ee0 legacy=0x7f985c0d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:45.397 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.394+0000 7f985aa00640 1 -- 172.21.15.196:0/2014375783 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985c0b41f0 con 0x7f985c09ec80 2024-04-28T13:35:45.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.394+0000 7f985aa00640 1 -- 172.21.15.196:0/2014375783 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f98540019f0 con 0x7f985c09ec80 2024-04-28T13:35:45.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.394+0000 7f985b400640 1 --2- 172.21.15.196:0/2014375783 >> v2:172.21.15.203:3300/0 conn(0x7f985c0d1760 0x7f985c0b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:35:45.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.394+0000 7f985aa00640 1 -- 172.21.15.196:0/2014375783 >> v1:172.21.15.196:6789/0 conn(0x7f985c09ec80 legacy=0x7f985c06df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:45.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.394+0000 7f985aa00640 1 --2- 172.21.15.196:0/2014375783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9840003a00 0x7f9840003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:45.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.394+0000 7f985aa00640 1 --2- 172.21.15.196:0/2014375783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9840004800 0x7f9840004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:45.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.394+0000 7f985aa00640 1 --2- 172.21.15.196:0/2014375783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9840005210 0x7f984000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:45.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.394+0000 7f985aa00640 1 -- 172.21.15.196:0/2014375783 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f985c0b3f30 con 0x7f9840005210 2024-04-28T13:35:45.398 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.394+0000 7f985aa00640 1 -- 172.21.15.196:0/2014375783 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f984000c160 con 0x7f9840003a00 2024-04-28T13:35:45.399 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.394+0000 7f985aa00640 1 -- 172.21.15.196:0/2014375783 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f984000c360 con 0x7f9840004800 2024-04-28T13:35:45.399 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.394+0000 7f9860e00640 1 --2- 172.21.15.196:0/2014375783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9840005210 0x7f984000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:45.399 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.395+0000 7f985be00640 1 --2- 172.21.15.196:0/2014375783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9840003a00 0x7f9840003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:45.399 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.395+0000 7f9860e00640 1 -- 172.21.15.196:0/2014375783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9840004800 msgr2=0x7f9840004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:45.399 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.395+0000 7f9860e00640 1 --2- 172.21.15.196:0/2014375783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9840004800 0x7f9840004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:45.399 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.395+0000 7f9860e00640 1 -- 172.21.15.196:0/2014375783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9840003a00 msgr2=0x7f9840003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:45.399 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.395+0000 7f9860e00640 1 --2- 172.21.15.196:0/2014375783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9840003a00 0x7f9840003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:45.400 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.395+0000 7f9860e00640 1 -- 172.21.15.196:0/2014375783 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9840003520 con 0x7f9840005210 2024-04-28T13:35:45.400 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.396+0000 7f9860e00640 1 -- 172.21.15.196:0/2014375783 reap_dead start 2024-04-28T13:35:45.400 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.396+0000 7f9860e00640 1 --2- 172.21.15.196:0/2014375783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9840005210 0x7f984000b9c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f985400a790 tx=0x7f9840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e3465840b20799 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:45.400 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.396+0000 7f985aa00640 1 -- 172.21.15.196:0/2014375783 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985400a3b0 con 0x7f9840005210 2024-04-28T13:35:45.400 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.396+0000 7f985aa00640 1 -- 172.21.15.196:0/2014375783 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f985402f070 con 0x7f9840005210 2024-04-28T13:35:45.400 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.396+0000 7f985aa00640 1 -- 172.21.15.196:0/2014375783 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985402bc20 con 0x7f9840005210 2024-04-28T13:35:45.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.396+0000 7f9861800640 1 -- 172.21.15.196:0/2014375783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9840005210 msgr2=0x7f984000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:45.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.396+0000 7f9861800640 1 --2- 172.21.15.196:0/2014375783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9840005210 0x7f984000b9c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f985400a790 tx=0x7f9840005830 comp rx=0 tx=0).stop 2024-04-28T13:35:45.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.397+0000 7f9861800640 1 -- 172.21.15.196:0/2014375783 shutdown_connections 2024-04-28T13:35:45.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.397+0000 7f9861800640 1 --2- 172.21.15.196:0/2014375783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9840005210 0x7f984000b9c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:45.401 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.397+0000 7f9861800640 1 -- 172.21.15.196:0/2014375783 >> 172.21.15.196:0/2014375783 conn(0x7f985c06c6c0 msgr2=0x7f985c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:45.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.397+0000 7f9861800640 1 -- 172.21.15.196:0/2014375783 shutdown_connections 2024-04-28T13:35:45.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.397+0000 7f9861800640 1 -- 172.21.15.196:0/2014375783 wait complete. 2024-04-28T13:35:45.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.398+0000 7f9861800640 1 Processor -- start 2024-04-28T13:35:45.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.398+0000 7f9861800640 1 -- start start 2024-04-28T13:35:45.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.399+0000 7f9861800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9840005210 0x7f985c157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:45.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.399+0000 7f9861800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f985c149740 0x7f985c0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:45.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.399+0000 7f9861800640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f985c158b10 0x7f985c158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:45.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.399+0000 7f9861800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f985c0b3c20 con 0x7f9840005210 2024-04-28T13:35:45.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.399+0000 7f9861800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f985c0b47b0 con 0x7f985c149740 2024-04-28T13:35:45.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.399+0000 7f9861800640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f985c0add70 con 0x7f985c158b10 2024-04-28T13:35:45.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.400+0000 7f985be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9840005210 0x7f985c157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:45.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.400+0000 7f985be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9840005210 0x7f985c157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58676/0 (socket says 172.21.15.196:58676) 2024-04-28T13:35:45.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.400+0000 7f985be00640 1 -- 172.21.15.196:0/2055384142 learned_addr learned my addr 172.21.15.196:0/2055384142 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:45.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.400+0000 7f985be00640 1 -- 172.21.15.196:0/2055384142 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f985c158b10 msgr2=0x7f985c158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:45.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.400+0000 7f985be00640 1 --2- 172.21.15.196:0/2055384142 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f985c158b10 0x7f985c158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:45.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.400+0000 7f985be00640 1 -- 172.21.15.196:0/2055384142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f985c149740 msgr2=0x7f985c0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:45.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.400+0000 7f985b400640 1 --2- 172.21.15.196:0/2055384142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f985c149740 0x7f985c0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:45.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.400+0000 7f985be00640 1 --2- 172.21.15.196:0/2055384142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f985c149740 0x7f985c0cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:45.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.400+0000 7f985be00640 1 -- 172.21.15.196:0/2055384142 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985c149bc0 con 0x7f9840005210 2024-04-28T13:35:45.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.400+0000 7f985be00640 1 --2- 172.21.15.196:0/2055384142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9840005210 0x7f985c157ba0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f984c00e500 tx=0x7f984c00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:45.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.401+0000 7f9853e00640 1 -- 172.21.15.196:0/2055384142 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c0052e0 con 0x7f9840005210 2024-04-28T13:35:45.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.401+0000 7f9853e00640 1 -- 172.21.15.196:0/2055384142 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984c018070 con 0x7f9840005210 2024-04-28T13:35:45.407 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.401+0000 7f9853e00640 1 -- 172.21.15.196:0/2055384142 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c003810 con 0x7f9840005210 2024-04-28T13:35:45.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.401+0000 7f9861800640 1 -- 172.21.15.196:0/2055384142 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f985c15f430 con 0x7f9840005210 2024-04-28T13:35:45.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.401+0000 7f9861800640 1 -- 172.21.15.196:0/2055384142 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f985c15f9f0 con 0x7f9840005210 2024-04-28T13:35:45.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.402+0000 7f9861800640 1 -- 172.21.15.196:0/2055384142 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f985c06a290 con 0x7f9840005210 2024-04-28T13:35:45.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.403+0000 7f9853e00640 1 -- 172.21.15.196:0/2055384142 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f984c0039b0 con 0x7f9840005210 2024-04-28T13:35:45.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.404+0000 7f9853e00640 1 --2- 172.21.15.196:0/2055384142 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f983c06c980 0x7f983c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:45.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.405+0000 7f9853e00640 1 -- 172.21.15.196:0/2055384142 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f984c08b920 con 0x7f9840005210 2024-04-28T13:35:45.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.405+0000 7f985b400640 1 --2- 172.21.15.196:0/2055384142 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f983c06c980 0x7f983c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:45.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.406+0000 7f985b400640 1 --2- 172.21.15.196:0/2055384142 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f983c06c980 0x7f983c06ee40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9848006fd0 tx=0x7f9848008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:45.409 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.409+0000 7f9853e00640 1 -- 172.21.15.196:0/2055384142 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f984c059c30 con 0x7f9840005210 2024-04-28T13:35:45.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.610+0000 7f9861800640 1 -- 172.21.15.196:0/2055384142 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f985c0d0690 con 0x7f9840005210 2024-04-28T13:35:45.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.611+0000 7f9853e00640 1 -- 172.21.15.196:0/2055384142 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16741 (secure 0 0 0) 0x7f984c059530 con 0x7f9840005210 2024-04-28T13:35:45.611 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:35:45.612 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":47,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:34:04.329925+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:33:31.193265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-28T13:33:32.415392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.32","mappings":[{"from":0,"to":5}]},{"pgid":"7.38","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:35:45.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.614+0000 7f9861800640 1 -- 172.21.15.196:0/2055384142 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f983c06c980 msgr2=0x7f983c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:45.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.614+0000 7f9861800640 1 --2- 172.21.15.196:0/2055384142 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f983c06c980 0x7f983c06ee40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9848006fd0 tx=0x7f9848008040 comp rx=0 tx=0).stop 2024-04-28T13:35:45.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.614+0000 7f9861800640 1 -- 172.21.15.196:0/2055384142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9840005210 msgr2=0x7f985c157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:45.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.614+0000 7f9861800640 1 --2- 172.21.15.196:0/2055384142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9840005210 0x7f985c157ba0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f984c00e500 tx=0x7f984c00e9c0 comp rx=0 tx=0).stop 2024-04-28T13:35:45.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.614+0000 7f9861800640 1 -- 172.21.15.196:0/2055384142 shutdown_connections 2024-04-28T13:35:45.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.614+0000 7f9861800640 1 --2- 172.21.15.196:0/2055384142 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f983c06c980 0x7f983c06ee40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:45.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.614+0000 7f9861800640 1 --2- 172.21.15.196:0/2055384142 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f985c158b10 0x7f985c158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:45.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.614+0000 7f9861800640 1 --2- 172.21.15.196:0/2055384142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f985c149740 0x7f985c0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:45.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.614+0000 7f9861800640 1 --2- 172.21.15.196:0/2055384142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9840005210 0x7f985c157ba0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:45.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.614+0000 7f9861800640 1 -- 172.21.15.196:0/2055384142 >> 172.21.15.196:0/2055384142 conn(0x7f985c06c6c0 msgr2=0x7f985c15c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:45.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.614+0000 7f9861800640 1 -- 172.21.15.196:0/2055384142 shutdown_connections 2024-04-28T13:35:45.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.614+0000 7f9861800640 1 -- 172.21.15.196:0/2055384142 wait complete. 2024-04-28T13:35:45.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:33:31.193265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:33:32.415392+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:35:45.635 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:35:45.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.831+0000 7fbd1cc00640 1 Processor -- start 2024-04-28T13:35:45.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.831+0000 7fbd1cc00640 1 -- start start 2024-04-28T13:35:45.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.833+0000 7fbd1cc00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbd180b04c0 0x7fbd180af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:45.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.833+0000 7fbd1cc00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd181607d0 con 0x7fbd18145960 2024-04-28T13:35:45.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.833+0000 7fbd1cc00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd180aea50 con 0x7fbd1815fa10 2024-04-28T13:35:45.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.833+0000 7fbd1cc00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbd18160570 con 0x7fbd180b04c0 2024-04-28T13:35:45.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.833+0000 7fbd15400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbd180b04c0 0x7fbd180af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:45.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.833+0000 7fbd15400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbd180b04c0 0x7fbd180af530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58678/0 (socket says 172.21.15.196:58678) 2024-04-28T13:35:45.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.833+0000 7fbd15400640 1 -- 172.21.15.196:0/2010206189 learned_addr learned my addr 172.21.15.196:0/2010206189 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:45.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.833+0000 7fbd15e00640 1 --1- 172.21.15.196:0/2010206189 >> v1:172.21.15.196:6789/0 conn(0x7fbd18145960 0x7fbd180afdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:55626/0 (socket says 172.21.15.196:55626) 2024-04-28T13:35:45.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.834+0000 7fbd15400640 1 -- 172.21.15.196:0/2010206189 >> v1:172.21.15.203:6789/0 conn(0x7fbd1815fa10 legacy=0x7fbd1815fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:45.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.834+0000 7fbd15400640 1 -- 172.21.15.196:0/2010206189 >> v1:172.21.15.196:6789/0 conn(0x7fbd18145960 legacy=0x7fbd180afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:45.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.834+0000 7fbd14a00640 1 -- 172.21.15.196:0/2010206189 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4287073807 0 0) 0x7fbd181607d0 con 0x7fbd18145960 2024-04-28T13:35:45.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.834+0000 7fbd15400640 1 -- 172.21.15.196:0/2010206189 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd180aed30 con 0x7fbd180b04c0 2024-04-28T13:35:45.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.835+0000 7fbd15400640 1 --2- 172.21.15.196:0/2010206189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd180b04c0 0x7fbd180af530 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009870 tx=0x7fbd0402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cade690ef4106beb server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:45.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.835+0000 7fbd14a00640 1 -- 172.21.15.196:0/2010206189 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04037070 con 0x7fbd180b04c0 2024-04-28T13:35:45.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.835+0000 7fbd14a00640 1 -- 172.21.15.196:0/2010206189 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd04032440 con 0x7fbd180b04c0 2024-04-28T13:35:45.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.835+0000 7fbd14a00640 1 -- 172.21.15.196:0/2010206189 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04032810 con 0x7fbd180b04c0 2024-04-28T13:35:45.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.835+0000 7fbd1cc00640 1 -- 172.21.15.196:0/2010206189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd180b04c0 msgr2=0x7fbd180af530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:45.836 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.835+0000 7fbd1cc00640 1 --2- 172.21.15.196:0/2010206189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd180b04c0 0x7fbd180af530 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009870 tx=0x7fbd0402a2a0 comp rx=0 tx=0).stop 2024-04-28T13:35:45.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.836+0000 7fbd1cc00640 1 -- 172.21.15.196:0/2010206189 shutdown_connections 2024-04-28T13:35:45.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.836+0000 7fbd1cc00640 1 --2- 172.21.15.196:0/2010206189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd180b04c0 0x7fbd180af530 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:45.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.836+0000 7fbd1cc00640 1 -- 172.21.15.196:0/2010206189 >> 172.21.15.196:0/2010206189 conn(0x7fbd180cd8a0 msgr2=0x7fbd180cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:45.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.836+0000 7fbd1cc00640 1 -- 172.21.15.196:0/2010206189 shutdown_connections 2024-04-28T13:35:45.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.836+0000 7fbd1cc00640 1 -- 172.21.15.196:0/2010206189 wait complete. 2024-04-28T13:35:45.839 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.837+0000 7fbd1cc00640 1 Processor -- start 2024-04-28T13:35:45.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.838+0000 7fbd1cc00640 1 -- start start 2024-04-28T13:35:45.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.838+0000 7fbd1cc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd180b04c0 0x7fbd181f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:45.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.838+0000 7fbd1cc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd18145960 0x7fbd18151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:45.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.838+0000 7fbd1cc00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd1815fa10 0x7fbd180c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:45.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.839+0000 7fbd1cc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd180734f0 con 0x7fbd180b04c0 2024-04-28T13:35:45.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.839+0000 7fbd1cc00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18073370 con 0x7fbd18145960 2024-04-28T13:35:45.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.839+0000 7fbd1cc00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbd18073670 con 0x7fbd1815fa10 2024-04-28T13:35:45.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.839+0000 7fbd15e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd180b04c0 0x7fbd181f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:45.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.839+0000 7fbd15e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd180b04c0 0x7fbd181f5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58692/0 (socket says 172.21.15.196:58692) 2024-04-28T13:35:45.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.839+0000 7fbd15e00640 1 -- 172.21.15.196:0/3830498863 learned_addr learned my addr 172.21.15.196:0/3830498863 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:45.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.839+0000 7fbd15e00640 1 -- 172.21.15.196:0/3830498863 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd1815fa10 msgr2=0x7fbd180c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:45.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.839+0000 7fbd15e00640 1 --2- 172.21.15.196:0/3830498863 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd1815fa10 0x7fbd180c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:45.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.839+0000 7fbd15e00640 1 -- 172.21.15.196:0/3830498863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd18145960 msgr2=0x7fbd18151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:45.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.839+0000 7fbd15400640 1 --2- 172.21.15.196:0/3830498863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd18145960 0x7fbd18151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:45.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.839+0000 7fbd15e00640 1 --2- 172.21.15.196:0/3830498863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd18145960 0x7fbd18151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:45.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.839+0000 7fbd15e00640 1 -- 172.21.15.196:0/3830498863 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1814f8c0 con 0x7fbd180b04c0 2024-04-28T13:35:45.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.839+0000 7fbd15e00640 1 --2- 172.21.15.196:0/3830498863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd180b04c0 0x7fbd181f5eb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800c4d0 tx=0x7fbd0800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:45.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.840+0000 7fbd0e000640 1 -- 172.21.15.196:0/3830498863 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08018070 con 0x7fbd180b04c0 2024-04-28T13:35:45.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.840+0000 7fbd0e000640 1 -- 172.21.15.196:0/3830498863 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd08007500 con 0x7fbd180b04c0 2024-04-28T13:35:45.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.840+0000 7fbd0e000640 1 -- 172.21.15.196:0/3830498863 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08004a20 con 0x7fbd180b04c0 2024-04-28T13:35:45.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.840+0000 7fbd1cc00640 1 -- 172.21.15.196:0/3830498863 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1814fba0 con 0x7fbd180b04c0 2024-04-28T13:35:45.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.840+0000 7fbd1cc00640 1 -- 172.21.15.196:0/3830498863 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd18150160 con 0x7fbd180b04c0 2024-04-28T13:35:45.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.841+0000 7fbd1cc00640 1 -- 172.21.15.196:0/3830498863 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd1806a290 con 0x7fbd180b04c0 2024-04-28T13:35:45.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.842+0000 7fbd0e000640 1 -- 172.21.15.196:0/3830498863 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fbd08004bc0 con 0x7fbd180b04c0 2024-04-28T13:35:45.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.843+0000 7fbd0e000640 1 --2- 172.21.15.196:0/3830498863 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbcf806c980 0x7fbcf806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:45.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.843+0000 7fbd0e000640 1 -- 172.21.15.196:0/3830498863 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fbd0808b080 con 0x7fbd180b04c0 2024-04-28T13:35:45.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.844+0000 7fbd15400640 1 --2- 172.21.15.196:0/3830498863 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbcf806c980 0x7fbcf806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:45.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.845+0000 7fbd15400640 1 --2- 172.21.15.196:0/3830498863 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbcf806c980 0x7fbcf806ee40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbd04034000 tx=0x7fbd0402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:45.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:45.848+0000 7fbd0e000640 1 -- 172.21.15.196:0/3830498863 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbd08059300 con 0x7fbd180b04c0 2024-04-28T13:35:46.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.043+0000 7fbd1cc00640 1 -- 172.21.15.196:0/3830498863 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbd1806d2b0 con 0x7fbd180b04c0 2024-04-28T13:35:46.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.044+0000 7fbd0e000640 1 -- 172.21.15.196:0/3830498863 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7fbd08076e20 con 0x7fbd180b04c0 2024-04-28T13:35:46.043 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:35:46.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.046+0000 7fbd1cc00640 1 -- 172.21.15.196:0/3830498863 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbcf806c980 msgr2=0x7fbcf806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:46.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.046+0000 7fbd1cc00640 1 --2- 172.21.15.196:0/3830498863 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbcf806c980 0x7fbcf806ee40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbd04034000 tx=0x7fbd0402e000 comp rx=0 tx=0).stop 2024-04-28T13:35:46.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.046+0000 7fbd1cc00640 1 -- 172.21.15.196:0/3830498863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd180b04c0 msgr2=0x7fbd181f5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:46.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.046+0000 7fbd1cc00640 1 --2- 172.21.15.196:0/3830498863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd180b04c0 0x7fbd181f5eb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800c4d0 tx=0x7fbd0800c990 comp rx=0 tx=0).stop 2024-04-28T13:35:46.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.046+0000 7fbd1cc00640 1 -- 172.21.15.196:0/3830498863 shutdown_connections 2024-04-28T13:35:46.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.046+0000 7fbd1cc00640 1 --2- 172.21.15.196:0/3830498863 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fbcf806c980 0x7fbcf806ee40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.046+0000 7fbd1cc00640 1 --2- 172.21.15.196:0/3830498863 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd1815fa10 0x7fbd180c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.046+0000 7fbd1cc00640 1 --2- 172.21.15.196:0/3830498863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd18145960 0x7fbd18151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.046+0000 7fbd1cc00640 1 --2- 172.21.15.196:0/3830498863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd180b04c0 0x7fbd181f5eb0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.046+0000 7fbd1cc00640 1 -- 172.21.15.196:0/3830498863 >> 172.21.15.196:0/3830498863 conn(0x7fbd180cd8a0 msgr2=0x7fbd18144970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:46.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.046+0000 7fbd1cc00640 1 -- 172.21.15.196:0/3830498863 shutdown_connections 2024-04-28T13:35:46.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.046+0000 7fbd1cc00640 1 -- 172.21.15.196:0/3830498863 wait complete. 2024-04-28T13:35:46.064 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:35:46.264 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.264+0000 7f70d9000640 1 Processor -- start 2024-04-28T13:35:46.264 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.264+0000 7f70d9000640 1 -- start start 2024-04-28T13:35:46.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.266+0000 7f70d9000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f70d40d3830 0x7f70d40d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:46.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.266+0000 7f70d9000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d40d0600 0x7f70d40d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:46.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.266+0000 7f70d9000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70d40cfb80 con 0x7f70d40d3c10 2024-04-28T13:35:46.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.266+0000 7f70d9000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f70d40d2ee0 con 0x7f70d40d3830 2024-04-28T13:35:46.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.266+0000 7f70d9000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f70d40d31c0 con 0x7f70d40d0600 2024-04-28T13:35:46.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.266+0000 7f70d3e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d40d0600 0x7f70d40d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:46.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.266+0000 7f70d3e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d40d0600 0x7f70d40d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:59988/0 (socket says 172.21.15.196:59988) 2024-04-28T13:35:46.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.267+0000 7f70d3e00640 1 -- 172.21.15.196:0/3153454267 learned_addr learned my addr 172.21.15.196:0/3153454267 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:46.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.267+0000 7f70d3400640 1 --2- 172.21.15.196:0/3153454267 >> v2:172.21.15.203:3300/0 conn(0x7f70d40d3830 0x7f70d40d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:46.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.267+0000 7f70d3e00640 1 -- 172.21.15.196:0/3153454267 >> v2:172.21.15.203:3300/0 conn(0x7f70d40d3830 msgr2=0x7f70d40d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:46.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.267+0000 7f70d3e00640 1 --2- 172.21.15.196:0/3153454267 >> v2:172.21.15.203:3300/0 conn(0x7f70d40d3830 0x7f70d40d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.267+0000 7f70d3e00640 1 -- 172.21.15.196:0/3153454267 >> v1:172.21.15.203:6789/0 conn(0x7f70d40d3c10 legacy=0x7f70d40b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:46.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.267+0000 7f70d3e00640 1 -- 172.21.15.196:0/3153454267 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d40cfe00 con 0x7f70d40d0600 2024-04-28T13:35:46.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.267+0000 7f70d2a00640 1 -- 172.21.15.196:0/3153454267 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2926489663 0 0) 0x7f70bc001110 con 0x7f70d40d3c10 2024-04-28T13:35:46.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.269+0000 7f70d3e00640 1 --2- 172.21.15.196:0/3153454267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d40d0600 0x7f70d40d0a90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009e50 tx=0x7f70cc02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=975cc82cbb434ad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:46.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.269+0000 7f70d2a00640 1 -- 172.21.15.196:0/3153454267 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc036070 con 0x7f70d40d0600 2024-04-28T13:35:46.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.269+0000 7f70d2a00640 1 -- 172.21.15.196:0/3153454267 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70cc031600 con 0x7f70d40d0600 2024-04-28T13:35:46.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.269+0000 7f70d2a00640 1 -- 172.21.15.196:0/3153454267 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc0319d0 con 0x7f70d40d0600 2024-04-28T13:35:46.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.270+0000 7f70d9000640 1 -- 172.21.15.196:0/3153454267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d40d0600 msgr2=0x7f70d40d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:46.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.270+0000 7f70d9000640 1 --2- 172.21.15.196:0/3153454267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d40d0600 0x7f70d40d0a90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009e50 tx=0x7f70cc02a710 comp rx=0 tx=0).stop 2024-04-28T13:35:46.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.270+0000 7f70d9000640 1 -- 172.21.15.196:0/3153454267 shutdown_connections 2024-04-28T13:35:46.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.270+0000 7f70d9000640 1 --2- 172.21.15.196:0/3153454267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d40d0600 0x7f70d40d0a90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.270+0000 7f70d9000640 1 --2- 172.21.15.196:0/3153454267 >> v2:172.21.15.203:3300/0 conn(0x7f70d40d3830 0x7f70d40d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.270+0000 7f70d9000640 1 -- 172.21.15.196:0/3153454267 >> 172.21.15.196:0/3153454267 conn(0x7f70d40d4fe0 msgr2=0x7f70d40d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:46.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.270+0000 7f70d9000640 1 -- 172.21.15.196:0/3153454267 shutdown_connections 2024-04-28T13:35:46.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.270+0000 7f70d9000640 1 -- 172.21.15.196:0/3153454267 wait complete. 2024-04-28T13:35:46.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.271+0000 7f70d9000640 1 Processor -- start 2024-04-28T13:35:46.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.272+0000 7f70d9000640 1 -- start start 2024-04-28T13:35:46.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.272+0000 7f70d9000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70d40d0600 0x7f70d40ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:46.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.272+0000 7f70d9000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70d40d3830 0x7f70d40c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:46.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.272+0000 7f70d9000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d40d3c10 0x7f70d414dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:46.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.272+0000 7f70d9000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70d40d2940 con 0x7f70d40d3830 2024-04-28T13:35:46.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.272+0000 7f70d9000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f70d40b8540 con 0x7f70d40d0600 2024-04-28T13:35:46.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.272+0000 7f70d9000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f70d40d5d90 con 0x7f70d40d3c10 2024-04-28T13:35:46.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.273+0000 7f70cbe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70d40d3830 0x7f70d40c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:46.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.273+0000 7f70cbe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70d40d3830 0x7f70d40c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58696/0 (socket says 172.21.15.196:58696) 2024-04-28T13:35:46.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.273+0000 7f70cbe00640 1 -- 172.21.15.196:0/283851798 learned_addr learned my addr 172.21.15.196:0/283851798 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:46.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.273+0000 7f70d3400640 1 --2- 172.21.15.196:0/283851798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70d40d0600 0x7f70d40ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:46.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.273+0000 7f70d3e00640 1 --2- 172.21.15.196:0/283851798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d40d3c10 0x7f70d414dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:46.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.273+0000 7f70cbe00640 1 -- 172.21.15.196:0/283851798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d40d3c10 msgr2=0x7f70d414dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:46.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.273+0000 7f70cbe00640 1 --2- 172.21.15.196:0/283851798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d40d3c10 0x7f70d414dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.273+0000 7f70cbe00640 1 -- 172.21.15.196:0/283851798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70d40d0600 msgr2=0x7f70d40ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:46.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.273+0000 7f70cbe00640 1 --2- 172.21.15.196:0/283851798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70d40d0600 0x7f70d40ce9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.273+0000 7f70cbe00640 1 -- 172.21.15.196:0/283851798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d40caf80 con 0x7f70d40d3830 2024-04-28T13:35:46.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.273+0000 7f70d3e00640 1 --2- 172.21.15.196:0/283851798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d40d3c10 0x7f70d414dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:35:46.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.274+0000 7f70cbe00640 1 --2- 172.21.15.196:0/283851798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70d40d3830 0x7f70d40c8b10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00c750 tx=0x7f70bc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:46.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.274+0000 7f70cb400640 1 -- 172.21.15.196:0/283851798 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc017070 con 0x7f70d40d3830 2024-04-28T13:35:46.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.274+0000 7f70cb400640 1 -- 172.21.15.196:0/283851798 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70bc0052b0 con 0x7f70d40d3830 2024-04-28T13:35:46.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.274+0000 7f70cb400640 1 -- 172.21.15.196:0/283851798 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc00a880 con 0x7f70d40d3830 2024-04-28T13:35:46.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.274+0000 7f70d9000640 1 -- 172.21.15.196:0/283851798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d40cb260 con 0x7f70d40d3830 2024-04-28T13:35:46.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.274+0000 7f70d9000640 1 -- 172.21.15.196:0/283851798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d40cb820 con 0x7f70d40d3830 2024-04-28T13:35:46.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.275+0000 7f70d9000640 1 -- 172.21.15.196:0/283851798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d4050730 con 0x7f70d40d3830 2024-04-28T13:35:46.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.276+0000 7f70cb400640 1 -- 172.21.15.196:0/283851798 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f70bc01a070 con 0x7f70d40d3830 2024-04-28T13:35:46.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.277+0000 7f70cb400640 1 --2- 172.21.15.196:0/283851798 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f70b006c9b0 0x7f70b006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:46.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.277+0000 7f70cb400640 1 -- 172.21.15.196:0/283851798 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f70bc08af40 con 0x7f70d40d3830 2024-04-28T13:35:46.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.278+0000 7f70d3400640 1 --2- 172.21.15.196:0/283851798 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f70b006c9b0 0x7f70b006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:46.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.279+0000 7f70d3400640 1 --2- 172.21.15.196:0/283851798 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f70b006c9b0 0x7f70b006ee70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009ad0 tx=0x7f70c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:46.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.282+0000 7f70cb400640 1 -- 172.21.15.196:0/283851798 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f70d40cfb80 con 0x7f70d40d3830 2024-04-28T13:35:46.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.481+0000 7f70d9000640 1 -- 172.21.15.196:0/283851798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f70d40d17d0 con 0x7f70d40d3830 2024-04-28T13:35:46.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.482+0000 7f70cb400640 1 -- 172.21.15.196:0/283851798 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) ==== 104+0+11 (secure 0 0 0) 0x7f70bc007d30 con 0x7f70d40d3830 2024-04-28T13:35:46.481 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:35:46.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.484+0000 7f70d9000640 1 -- 172.21.15.196:0/283851798 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f70b006c9b0 msgr2=0x7f70b006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:46.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.484+0000 7f70d9000640 1 --2- 172.21.15.196:0/283851798 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f70b006c9b0 0x7f70b006ee70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009ad0 tx=0x7f70c4008040 comp rx=0 tx=0).stop 2024-04-28T13:35:46.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.484+0000 7f70d9000640 1 -- 172.21.15.196:0/283851798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70d40d3830 msgr2=0x7f70d40c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:46.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.484+0000 7f70d9000640 1 --2- 172.21.15.196:0/283851798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70d40d3830 0x7f70d40c8b10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00c750 tx=0x7f70bc00cc10 comp rx=0 tx=0).stop 2024-04-28T13:35:46.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.484+0000 7f70d9000640 1 -- 172.21.15.196:0/283851798 shutdown_connections 2024-04-28T13:35:46.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.484+0000 7f70d9000640 1 --2- 172.21.15.196:0/283851798 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f70b006c9b0 0x7f70b006ee70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.484+0000 7f70d9000640 1 --2- 172.21.15.196:0/283851798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d40d3c10 0x7f70d414dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.484+0000 7f70d9000640 1 --2- 172.21.15.196:0/283851798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70d40d3830 0x7f70d40c8b10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.484+0000 7f70d9000640 1 --2- 172.21.15.196:0/283851798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70d40d0600 0x7f70d40ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.484+0000 7f70d9000640 1 -- 172.21.15.196:0/283851798 >> 172.21.15.196:0/283851798 conn(0x7f70d40d4fe0 msgr2=0x7f70d40d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:46.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.485+0000 7f70d9000640 1 -- 172.21.15.196:0/283851798 shutdown_connections 2024-04-28T13:35:46.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.485+0000 7f70d9000640 1 -- 172.21.15.196:0/283851798 wait complete. 2024-04-28T13:35:46.504 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:35:46.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.705+0000 7f7413c00640 1 Processor -- start 2024-04-28T13:35:46.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.705+0000 7f7413c00640 1 -- start start 2024-04-28T13:35:46.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.707+0000 7f7413c00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f740c06f340 0x7f740c0b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:46.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.707+0000 7f7413c00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f740c0b3e50 0x7f740c0720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:46.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.707+0000 7f7413c00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f740c0b46a0 con 0x7f740c0d0300 2024-04-28T13:35:46.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.707+0000 7f7413c00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f740c0b5db0 con 0x7f740c06f340 2024-04-28T13:35:46.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.707+0000 7f7413c00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f740c0b6090 con 0x7f740c0b3e50 2024-04-28T13:35:46.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.707+0000 7f7411200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f740c06f340 0x7f740c0b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:46.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.707+0000 7f7403e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f740c0d0300 0x7f740c06f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:55640/0 (socket says 172.21.15.196:55640) 2024-04-28T13:35:46.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.707+0000 7f7403e00640 1 -- 172.21.15.196:0/74946903 learned_addr learned my addr 172.21.15.196:0/74946903 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:46.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.708+0000 7f7411c00640 1 --2- 172.21.15.196:0/74946903 >> v2:172.21.15.203:3300/0 conn(0x7f740c0b3e50 0x7f740c0720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:46.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.708+0000 7f7411200640 1 -- 172.21.15.196:0/74946903 >> v2:172.21.15.203:3300/0 conn(0x7f740c0b3e50 msgr2=0x7f740c0720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:46.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.708+0000 7f7411200640 1 --2- 172.21.15.196:0/74946903 >> v2:172.21.15.203:3300/0 conn(0x7f740c0b3e50 0x7f740c0720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.708+0000 7f7411200640 1 -- 172.21.15.196:0/74946903 >> v1:172.21.15.196:6789/0 conn(0x7f740c0d0300 legacy=0x7f740c06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:46.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.708+0000 7f7411200640 1 -- 172.21.15.196:0/74946903 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740c0b6370 con 0x7f740c06f340 2024-04-28T13:35:46.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.708+0000 7f740be00640 1 -- 172.21.15.196:0/74946903 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3598862241 0 0) 0x7f740c0b46a0 con 0x7f740c0d0300 2024-04-28T13:35:46.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.708+0000 7f7411c00640 1 --2- 172.21.15.196:0/74946903 >> v2:172.21.15.203:3300/0 conn(0x7f740c0b3e50 0x7f740c0720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:35:46.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.709+0000 7f7411200640 1 --2- 172.21.15.196:0/74946903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c06f340 0x7f740c0b6b70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f740c070020 tx=0x7f73fc02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1541fc53e93243bd server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:46.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.709+0000 7f740be00640 1 -- 172.21.15.196:0/74946903 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc037070 con 0x7f740c06f340 2024-04-28T13:35:46.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.709+0000 7f740be00640 1 -- 172.21.15.196:0/74946903 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73fc02add0 con 0x7f740c06f340 2024-04-28T13:35:46.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.709+0000 7f740be00640 1 -- 172.21.15.196:0/74946903 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc032660 con 0x7f740c06f340 2024-04-28T13:35:46.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.710+0000 7f7413c00640 1 -- 172.21.15.196:0/74946903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c06f340 msgr2=0x7f740c0b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:46.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.710+0000 7f7413c00640 1 --2- 172.21.15.196:0/74946903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c06f340 0x7f740c0b6b70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f740c070020 tx=0x7f73fc02a080 comp rx=0 tx=0).stop 2024-04-28T13:35:46.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.710+0000 7f7413c00640 1 -- 172.21.15.196:0/74946903 shutdown_connections 2024-04-28T13:35:46.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.710+0000 7f7413c00640 1 --2- 172.21.15.196:0/74946903 >> v2:172.21.15.203:3300/0 conn(0x7f740c0b3e50 0x7f740c0720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.710+0000 7f7413c00640 1 --2- 172.21.15.196:0/74946903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c06f340 0x7f740c0b6b70 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.710+0000 7f7413c00640 1 -- 172.21.15.196:0/74946903 >> 172.21.15.196:0/74946903 conn(0x7f740c0cd8a0 msgr2=0x7f740c0cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:46.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.710+0000 7f7413c00640 1 -- 172.21.15.196:0/74946903 shutdown_connections 2024-04-28T13:35:46.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.710+0000 7f7413c00640 1 -- 172.21.15.196:0/74946903 wait complete. 2024-04-28T13:35:46.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.712+0000 7f7413c00640 1 Processor -- start 2024-04-28T13:35:46.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.712+0000 7f7413c00640 1 -- start start 2024-04-28T13:35:46.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.714+0000 7f7413c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f740c06f340 0x7f740c15e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:46.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.714+0000 7f7413c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f740c0b3e50 0x7f740c14bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:46.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.714+0000 7f7413c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c0d0300 0x7f740c14c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:46.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.714+0000 7f7413c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f740c0b4460 con 0x7f740c0d0300 2024-04-28T13:35:46.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.714+0000 7f7413c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f740c0b42e0 con 0x7f740c0b3e50 2024-04-28T13:35:46.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.714+0000 7f7413c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f740c142a30 con 0x7f740c06f340 2024-04-28T13:35:46.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.714+0000 7f7411c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c0d0300 0x7f740c14c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:46.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.714+0000 7f7411c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c0d0300 0x7f740c14c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58722/0 (socket says 172.21.15.196:58722) 2024-04-28T13:35:46.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.714+0000 7f7411c00640 1 -- 172.21.15.196:0/2074395576 learned_addr learned my addr 172.21.15.196:0/2074395576 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:46.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.714+0000 7f7411c00640 1 -- 172.21.15.196:0/2074395576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f740c06f340 msgr2=0x7f740c15e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:46.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.714+0000 7f7411c00640 1 --2- 172.21.15.196:0/2074395576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f740c06f340 0x7f740c15e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.714+0000 7f7411c00640 1 -- 172.21.15.196:0/2074395576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f740c0b3e50 msgr2=0x7f740c14bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:46.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.714+0000 7f7411c00640 1 --2- 172.21.15.196:0/2074395576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f740c0b3e50 0x7f740c14bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.714+0000 7f7411c00640 1 -- 172.21.15.196:0/2074395576 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740c15f0d0 con 0x7f740c0d0300 2024-04-28T13:35:46.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.715+0000 7f7411c00640 1 --2- 172.21.15.196:0/2074395576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c0d0300 0x7f740c14c200 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f740400cef0 tx=0x7f740400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:46.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.715+0000 7f7409600640 1 -- 172.21.15.196:0/2074395576 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740401a070 con 0x7f740c0d0300 2024-04-28T13:35:46.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.715+0000 7f7409600640 1 -- 172.21.15.196:0/2074395576 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f740400d860 con 0x7f740c0d0300 2024-04-28T13:35:46.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.715+0000 7f7413c00640 1 -- 172.21.15.196:0/2074395576 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740c15f3b0 con 0x7f740c0d0300 2024-04-28T13:35:46.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.715+0000 7f7409600640 1 -- 172.21.15.196:0/2074395576 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740400dc50 con 0x7f740c0d0300 2024-04-28T13:35:46.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.715+0000 7f7413c00640 1 -- 172.21.15.196:0/2074395576 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740c15ed30 con 0x7f740c0d0300 2024-04-28T13:35:46.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.716+0000 7f7413c00640 1 -- 172.21.15.196:0/2074395576 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f740c06a290 con 0x7f740c0d0300 2024-04-28T13:35:46.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.717+0000 7f7409600640 1 -- 172.21.15.196:0/2074395576 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f740401d070 con 0x7f740c0d0300 2024-04-28T13:35:46.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.718+0000 7f7409600640 1 --2- 172.21.15.196:0/2074395576 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73e806c980 0x7f73e806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:46.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.718+0000 7f7409600640 1 -- 172.21.15.196:0/2074395576 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f740408c480 con 0x7f740c0d0300 2024-04-28T13:35:46.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.719+0000 7f7411200640 1 --2- 172.21.15.196:0/2074395576 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73e806c980 0x7f73e806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:46.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.720+0000 7f7411200640 1 --2- 172.21.15.196:0/2074395576 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73e806c980 0x7f73e806ee40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f73fc034000 tx=0x7f73fc02c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:46.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.723+0000 7f7409600640 1 -- 172.21.15.196:0/2074395576 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f740405a940 con 0x7f740c0d0300 2024-04-28T13:35:46.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.923+0000 7f7413c00640 1 -- 172.21.15.196:0/2074395576 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f740c0cfdd0 con 0x7f740c0d0300 2024-04-28T13:35:46.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.924+0000 7f7409600640 1 -- 172.21.15.196:0/2074395576 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7f740405a240 con 0x7f740c0d0300 2024-04-28T13:35:46.924 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:35:46.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.927+0000 7f7413c00640 1 -- 172.21.15.196:0/2074395576 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73e806c980 msgr2=0x7f73e806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:46.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.927+0000 7f7413c00640 1 --2- 172.21.15.196:0/2074395576 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73e806c980 0x7f73e806ee40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f73fc034000 tx=0x7f73fc02c000 comp rx=0 tx=0).stop 2024-04-28T13:35:46.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.927+0000 7f7413c00640 1 -- 172.21.15.196:0/2074395576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c0d0300 msgr2=0x7f740c14c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:46.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.927+0000 7f7413c00640 1 --2- 172.21.15.196:0/2074395576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c0d0300 0x7f740c14c200 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f740400cef0 tx=0x7f740400a710 comp rx=0 tx=0).stop 2024-04-28T13:35:46.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.927+0000 7f7413c00640 1 -- 172.21.15.196:0/2074395576 shutdown_connections 2024-04-28T13:35:46.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.927+0000 7f7413c00640 1 --2- 172.21.15.196:0/2074395576 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f73e806c980 0x7f73e806ee40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.927+0000 7f7413c00640 1 --2- 172.21.15.196:0/2074395576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f740c0d0300 0x7f740c14c200 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.927+0000 7f7413c00640 1 --2- 172.21.15.196:0/2074395576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f740c0b3e50 0x7f740c14bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.927+0000 7f7413c00640 1 --2- 172.21.15.196:0/2074395576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f740c06f340 0x7f740c15e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:46.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.927+0000 7f7413c00640 1 -- 172.21.15.196:0/2074395576 >> 172.21.15.196:0/2074395576 conn(0x7f740c0cd8a0 msgr2=0x7f740c1523f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:46.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.927+0000 7f7413c00640 1 -- 172.21.15.196:0/2074395576 shutdown_connections 2024-04-28T13:35:46.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:46.927+0000 7f7413c00640 1 -- 172.21.15.196:0/2074395576 wait complete. 2024-04-28T13:35:46.951 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T13:35:47.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.153+0000 7f1737000640 1 Processor -- start 2024-04-28T13:35:47.154 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.153+0000 7f1737000640 1 -- start start 2024-04-28T13:35:47.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.155+0000 7f1737000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f173009ba40 0x7f173015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:47.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.155+0000 7f1737000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f173015fce0 con 0x7f17300b5530 2024-04-28T13:35:47.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.155+0000 7f1737000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f173015ffc0 con 0x7f1730070570 2024-04-28T13:35:47.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.155+0000 7f1737000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f17300b59c0 con 0x7f173009ba40 2024-04-28T13:35:47.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.155+0000 7f1735000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f17300b5530 0x7f17300d1040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:55646/0 (socket says 172.21.15.196:55646) 2024-04-28T13:35:47.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.155+0000 7f1735000640 1 -- 172.21.15.196:0/3372181947 learned_addr learned my addr 172.21.15.196:0/3372181947 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:47.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.155+0000 7f172fe00640 1 --2- 172.21.15.196:0/3372181947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f173009ba40 0x7f173015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:47.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.156+0000 7f172ea00640 1 -- 172.21.15.196:0/3372181947 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 794962918 0 0) 0x7f173015fce0 con 0x7f17300b5530 2024-04-28T13:35:47.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.156+0000 7f172ea00640 1 -- 172.21.15.196:0/3372181947 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1714003520 con 0x7f17300b5530 2024-04-28T13:35:47.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.156+0000 7f172ea00640 1 -- 172.21.15.196:0/3372181947 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 827813888 0 0) 0x7f1714003520 con 0x7f17300b5530 2024-04-28T13:35:47.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.156+0000 7f172ea00640 1 -- 172.21.15.196:0/3372181947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f173009ba40 msgr2=0x7f173015f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:47.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.156+0000 7f172ea00640 1 --2- 172.21.15.196:0/3372181947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f173009ba40 0x7f173015f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:47.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.156+0000 7f172ea00640 1 -- 172.21.15.196:0/3372181947 >> v1:172.21.15.203:6789/0 conn(0x7f1730070570 legacy=0x7f173009be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:47.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.156+0000 7f172ea00640 1 -- 172.21.15.196:0/3372181947 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17301602a0 con 0x7f17300b5530 2024-04-28T13:35:47.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.156+0000 7f172ea00640 1 -- 172.21.15.196:0/3372181947 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f17280019f0 con 0x7f17300b5530 2024-04-28T13:35:47.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.156+0000 7f172ea00640 1 -- 172.21.15.196:0/3372181947 >> v1:172.21.15.196:6789/0 conn(0x7f17300b5530 legacy=0x7f17300d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:47.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.156+0000 7f172ea00640 1 --2- 172.21.15.196:0/3372181947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17140039e0 0x7f1714003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:47.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.156+0000 7f172ea00640 1 --2- 172.21.15.196:0/3372181947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1714004800 0x7f1714004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:47.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.156+0000 7f172ea00640 1 --2- 172.21.15.196:0/3372181947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f173009ba40 0x7f17140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:47.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.156+0000 7f172ea00640 1 -- 172.21.15.196:0/3372181947 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17300b59c0 con 0x7f1714004800 2024-04-28T13:35:47.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.156+0000 7f172ea00640 1 -- 172.21.15.196:0/3372181947 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f171400baa0 con 0x7f17140039e0 2024-04-28T13:35:47.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.156+0000 7f172ea00640 1 -- 172.21.15.196:0/3372181947 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f171400bca0 con 0x7f173009ba40 2024-04-28T13:35:47.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.157+0000 7f172f400640 1 --2- 172.21.15.196:0/3372181947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1714004800 0x7f1714004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:47.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.157+0000 7f172fe00640 1 --2- 172.21.15.196:0/3372181947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17140039e0 0x7f1714003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:47.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.157+0000 7f1735000640 1 --2- 172.21.15.196:0/3372181947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f173009ba40 0x7f17140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:47.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.157+0000 7f172f400640 1 -- 172.21.15.196:0/3372181947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f173009ba40 msgr2=0x7f17140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:47.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.157+0000 7f172f400640 1 --2- 172.21.15.196:0/3372181947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f173009ba40 0x7f17140051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:47.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.157+0000 7f172f400640 1 -- 172.21.15.196:0/3372181947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17140039e0 msgr2=0x7f1714003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:47.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.157+0000 7f172f400640 1 --2- 172.21.15.196:0/3372181947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17140039e0 0x7f1714003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:47.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.157+0000 7f172f400640 1 -- 172.21.15.196:0/3372181947 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1714003520 con 0x7f1714004800 2024-04-28T13:35:47.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.158+0000 7f172f400640 1 --2- 172.21.15.196:0/3372181947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1714004800 0x7f1714004c80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f171c001720 tx=0x7f1714005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722143f083637255 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:47.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.158+0000 7f172ea00640 1 -- 172.21.15.196:0/3372181947 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c001370 con 0x7f1714004800 2024-04-28T13:35:47.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.158+0000 7f172ea00640 1 -- 172.21.15.196:0/3372181947 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f171c0052d0 con 0x7f1714004800 2024-04-28T13:35:47.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.158+0000 7f172ea00640 1 -- 172.21.15.196:0/3372181947 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c02ab90 con 0x7f1714004800 2024-04-28T13:35:47.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.159+0000 7f1737000640 1 -- 172.21.15.196:0/3372181947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1714004800 msgr2=0x7f1714004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:47.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.159+0000 7f1737000640 1 --2- 172.21.15.196:0/3372181947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1714004800 0x7f1714004c80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f171c001720 tx=0x7f1714005830 comp rx=0 tx=0).stop 2024-04-28T13:35:47.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.159+0000 7f1735000640 1 -- 172.21.15.196:0/3372181947 reap_dead start 2024-04-28T13:35:47.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.159+0000 7f1737000640 1 -- 172.21.15.196:0/3372181947 shutdown_connections 2024-04-28T13:35:47.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.159+0000 7f1737000640 1 -- 172.21.15.196:0/3372181947 >> 172.21.15.196:0/3372181947 conn(0x7f17300b4a10 msgr2=0x7f17300b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:47.163 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.159+0000 7f1737000640 1 -- 172.21.15.196:0/3372181947 shutdown_connections 2024-04-28T13:35:47.163 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.159+0000 7f1737000640 1 -- 172.21.15.196:0/3372181947 wait complete. 2024-04-28T13:35:47.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.161+0000 7f1737000640 1 Processor -- start 2024-04-28T13:35:47.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.161+0000 7f1737000640 1 -- start start 2024-04-28T13:35:47.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.162+0000 7f1737000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1714004800 0x7f173015a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:47.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.162+0000 7f1737000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17300c92f0 0x7f1730161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:47.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.162+0000 7f1737000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17301619c0 0x7f17301627a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:47.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.162+0000 7f1737000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f173015fa30 con 0x7f17301619c0 2024-04-28T13:35:47.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.162+0000 7f1737000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f173015f8b0 con 0x7f1714004800 2024-04-28T13:35:47.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.162+0000 7f1737000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1730160610 con 0x7f17300c92f0 2024-04-28T13:35:47.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.162+0000 7f1735000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17301619c0 0x7f17301627a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:47.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.162+0000 7f1735000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17301619c0 0x7f17301627a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58742/0 (socket says 172.21.15.196:58742) 2024-04-28T13:35:47.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.162+0000 7f1735000640 1 -- 172.21.15.196:0/3071125486 learned_addr learned my addr 172.21.15.196:0/3071125486 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:47.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.163+0000 7f172fe00640 1 --2- 172.21.15.196:0/3071125486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1714004800 0x7f173015a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:47.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.163+0000 7f1735000640 1 -- 172.21.15.196:0/3071125486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17300c92f0 msgr2=0x7f1730161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:47.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.163+0000 7f172f400640 1 --2- 172.21.15.196:0/3071125486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17300c92f0 0x7f1730161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:47.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.163+0000 7f1735000640 1 --2- 172.21.15.196:0/3071125486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17300c92f0 0x7f1730161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:47.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.163+0000 7f1735000640 1 -- 172.21.15.196:0/3071125486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1714004800 msgr2=0x7f173015a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:47.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.163+0000 7f1735000640 1 --2- 172.21.15.196:0/3071125486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1714004800 0x7f173015a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:47.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.163+0000 7f1735000640 1 -- 172.21.15.196:0/3071125486 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1730161e40 con 0x7f17301619c0 2024-04-28T13:35:47.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.163+0000 7f172fe00640 1 --2- 172.21.15.196:0/3071125486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1714004800 0x7f173015a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:35:47.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.163+0000 7f1735000640 1 --2- 172.21.15.196:0/3071125486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17301619c0 0x7f17301627a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f17300b5910 tx=0x7f172800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:47.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.163+0000 7f1727e00640 1 -- 172.21.15.196:0/3071125486 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728004d60 con 0x7f17301619c0 2024-04-28T13:35:47.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.163+0000 7f1727e00640 1 -- 172.21.15.196:0/3071125486 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172800bb80 con 0x7f17301619c0 2024-04-28T13:35:47.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.163+0000 7f1737000640 1 -- 172.21.15.196:0/3071125486 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17301e8ed0 con 0x7f17301619c0 2024-04-28T13:35:47.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.163+0000 7f1727e00640 1 -- 172.21.15.196:0/3071125486 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728007070 con 0x7f17301619c0 2024-04-28T13:35:47.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.164+0000 7f1737000640 1 -- 172.21.15.196:0/3071125486 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17301e9490 con 0x7f17301619c0 2024-04-28T13:35:47.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.165+0000 7f1737000640 1 -- 172.21.15.196:0/3071125486 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f173006a290 con 0x7f17301619c0 2024-04-28T13:35:47.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.166+0000 7f1727e00640 1 -- 172.21.15.196:0/3071125486 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f172801b020 con 0x7f17301619c0 2024-04-28T13:35:47.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.167+0000 7f1727e00640 1 --2- 172.21.15.196:0/3071125486 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f171006c980 0x7f171006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:47.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.167+0000 7f1727e00640 1 -- 172.21.15.196:0/3071125486 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f172808bfe0 con 0x7f17301619c0 2024-04-28T13:35:47.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.167+0000 7f172fe00640 1 --2- 172.21.15.196:0/3071125486 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f171006c980 0x7f171006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:47.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.168+0000 7f172fe00640 1 --2- 172.21.15.196:0/3071125486 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f171006c980 0x7f171006ee40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1720009a00 tx=0x7f1720008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:47.171 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.172+0000 7f1727e00640 1 -- 172.21.15.196:0/3071125486 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f172805a2f0 con 0x7f17301619c0 2024-04-28T13:35:47.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.385+0000 7f1737000640 1 -- 172.21.15.196:0/3071125486 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f173015ed30 con 0x7f17301619c0 2024-04-28T13:35:47.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.386+0000 7f1727e00640 1 -- 172.21.15.196:0/3071125486 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) ==== 80+0+0 (secure 0 0 0) 0x7f1728059bf0 con 0x7f17301619c0 2024-04-28T13:35:47.386 INFO:teuthology.orchestra.run.smithi196.stderr:listed 0 entries 2024-04-28T13:35:47.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.389+0000 7f1737000640 1 -- 172.21.15.196:0/3071125486 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f171006c980 msgr2=0x7f171006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:47.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.389+0000 7f1737000640 1 --2- 172.21.15.196:0/3071125486 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f171006c980 0x7f171006ee40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1720009a00 tx=0x7f1720008040 comp rx=0 tx=0).stop 2024-04-28T13:35:47.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.389+0000 7f1737000640 1 -- 172.21.15.196:0/3071125486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17301619c0 msgr2=0x7f17301627a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:47.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.389+0000 7f1737000640 1 --2- 172.21.15.196:0/3071125486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17301619c0 0x7f17301627a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f17300b5910 tx=0x7f172800b4a0 comp rx=0 tx=0).stop 2024-04-28T13:35:47.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.389+0000 7f1737000640 1 -- 172.21.15.196:0/3071125486 shutdown_connections 2024-04-28T13:35:47.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.389+0000 7f1737000640 1 --2- 172.21.15.196:0/3071125486 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f171006c980 0x7f171006ee40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:47.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.389+0000 7f1737000640 1 --2- 172.21.15.196:0/3071125486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17301619c0 0x7f17301627a0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:47.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.389+0000 7f1737000640 1 --2- 172.21.15.196:0/3071125486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17300c92f0 0x7f1730161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:47.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.389+0000 7f1737000640 1 --2- 172.21.15.196:0/3071125486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1714004800 0x7f173015a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:47.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.389+0000 7f1737000640 1 -- 172.21.15.196:0/3071125486 >> 172.21.15.196:0/3071125486 conn(0x7f17300b4a10 msgr2=0x7f173015d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:47.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.389+0000 7f1737000640 1 -- 172.21.15.196:0/3071125486 shutdown_connections 2024-04-28T13:35:47.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.389+0000 7f1737000640 1 -- 172.21.15.196:0/3071125486 wait complete. 2024-04-28T13:35:47.408 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-28T13:35:47.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:47.408 DEBUG:teuthology.orchestra.run.smithi196:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-28T13:35:47.498 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:35:47.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:47.499 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-28T13:35:47.546 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-28T13:35:47.546 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:35:47.594 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-28T13:35:47.594 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:35:47.594 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:35:47.651 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:35:47.651 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:35:47.704 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:35:47.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.915+0000 7fd6fe200640 1 Processor -- start 2024-04-28T13:35:47.915 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.915+0000 7fd6fe200640 1 -- start start 2024-04-28T13:35:47.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.917+0000 7fd6fe200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd6f81629c0 0x7fd6f80d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:47.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.917+0000 7fd6fe200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd6f8163740 0x7fd6f80b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:47.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.917+0000 7fd6fe200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6f8162da0 con 0x7fd6f8163bf0 2024-04-28T13:35:47.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.917+0000 7fd6fe200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd6f80b6190 con 0x7fd6f8163740 2024-04-28T13:35:47.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.917+0000 7fd6fe200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd6f80b6470 con 0x7fd6f81629c0 2024-04-28T13:35:47.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.917+0000 7fd6fd800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd6f8163740 0x7fd6f80b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:47.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.917+0000 7fd6fd800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd6f8163740 0x7fd6f80b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58758/0 (socket says 172.21.15.196:58758) 2024-04-28T13:35:47.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.917+0000 7fd6efe00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fd6f8163bf0 0x7fd6f8163030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:55656/0 (socket says 172.21.15.196:55656) 2024-04-28T13:35:47.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.917+0000 7fd6fd800640 1 -- 172.21.15.196:0/3816667789 learned_addr learned my addr 172.21.15.196:0/3816667789 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:47.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.918+0000 7fd6fce00640 1 --2- 172.21.15.196:0/3816667789 >> v2:172.21.15.203:3300/0 conn(0x7fd6f81629c0 0x7fd6f80d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:47.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.918+0000 7fd6fd800640 1 -- 172.21.15.196:0/3816667789 >> v2:172.21.15.203:3300/0 conn(0x7fd6f81629c0 msgr2=0x7fd6f80d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:47.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.918+0000 7fd6fd800640 1 --2- 172.21.15.196:0/3816667789 >> v2:172.21.15.203:3300/0 conn(0x7fd6f81629c0 0x7fd6f80d3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:47.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.918+0000 7fd6fd800640 1 -- 172.21.15.196:0/3816667789 >> v1:172.21.15.196:6789/0 conn(0x7fd6f8163bf0 legacy=0x7fd6f8163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:47.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.918+0000 7fd6fd800640 1 -- 172.21.15.196:0/3816667789 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f80b6750 con 0x7fd6f8163740 2024-04-28T13:35:47.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.918+0000 7fd6f7e00640 1 -- 172.21.15.196:0/3816667789 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 653840568 0 0) 0x7fd6f8162da0 con 0x7fd6f8163bf0 2024-04-28T13:35:47.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.919+0000 7fd6fd800640 1 --2- 172.21.15.196:0/3816667789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f8163740 0x7fd6f80b2380 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd6f808f8a0 tx=0x7fd6f002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9eddb76161231ba server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:47.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.919+0000 7fd6f7e00640 1 -- 172.21.15.196:0/3816667789 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0036070 con 0x7fd6f8163740 2024-04-28T13:35:47.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.919+0000 7fd6f7e00640 1 -- 172.21.15.196:0/3816667789 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f0031600 con 0x7fd6f8163740 2024-04-28T13:35:47.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.919+0000 7fd6f7e00640 1 -- 172.21.15.196:0/3816667789 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00319d0 con 0x7fd6f8163740 2024-04-28T13:35:47.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.920+0000 7fd6fe200640 1 -- 172.21.15.196:0/3816667789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f8163740 msgr2=0x7fd6f80b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:47.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.920+0000 7fd6fe200640 1 --2- 172.21.15.196:0/3816667789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f8163740 0x7fd6f80b2380 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd6f808f8a0 tx=0x7fd6f002a730 comp rx=0 tx=0).stop 2024-04-28T13:35:47.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.920+0000 7fd6fe200640 1 -- 172.21.15.196:0/3816667789 shutdown_connections 2024-04-28T13:35:47.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.920+0000 7fd6fe200640 1 --2- 172.21.15.196:0/3816667789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f8163740 0x7fd6f80b2380 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:47.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.920+0000 7fd6fe200640 1 --2- 172.21.15.196:0/3816667789 >> v2:172.21.15.203:3300/0 conn(0x7fd6f81629c0 0x7fd6f80d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:47.922 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.920+0000 7fd6fe200640 1 -- 172.21.15.196:0/3816667789 >> 172.21.15.196:0/3816667789 conn(0x7fd6f809e570 msgr2=0x7fd6f80d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:47.922 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.920+0000 7fd6fe200640 1 -- 172.21.15.196:0/3816667789 shutdown_connections 2024-04-28T13:35:47.922 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.920+0000 7fd6fe200640 1 -- 172.21.15.196:0/3816667789 wait complete. 2024-04-28T13:35:47.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.922+0000 7fd6fe200640 1 Processor -- start 2024-04-28T13:35:47.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.922+0000 7fd6fe200640 1 -- start start 2024-04-28T13:35:47.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.923+0000 7fd6fe200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6f81629c0 0x7fd6f80b8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:47.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.923+0000 7fd6fe200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f8163740 0x7fd6f8161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:47.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.923+0000 7fd6fe200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6f8163bf0 0x7fd6f81f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:47.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.923+0000 7fd6fe200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f80d3c50 con 0x7fd6f8163740 2024-04-28T13:35:47.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.923+0000 7fd6fe200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f80b1e60 con 0x7fd6f81629c0 2024-04-28T13:35:47.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.923+0000 7fd6fe200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f80d3dd0 con 0x7fd6f8163bf0 2024-04-28T13:35:47.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.923+0000 7fd6efe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f8163740 0x7fd6f8161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:47.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.923+0000 7fd6efe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f8163740 0x7fd6f8161010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58760/0 (socket says 172.21.15.196:58760) 2024-04-28T13:35:47.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.923+0000 7fd6efe00640 1 -- 172.21.15.196:0/851109638 learned_addr learned my addr 172.21.15.196:0/851109638 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:47.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.923+0000 7fd6fce00640 1 --2- 172.21.15.196:0/851109638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6f81629c0 0x7fd6f80b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:47.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.924+0000 7fd6fd800640 1 --2- 172.21.15.196:0/851109638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6f8163bf0 0x7fd6f81f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:47.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.924+0000 7fd6efe00640 1 -- 172.21.15.196:0/851109638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6f8163bf0 msgr2=0x7fd6f81f8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:47.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.924+0000 7fd6efe00640 1 --2- 172.21.15.196:0/851109638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6f8163bf0 0x7fd6f81f8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:47.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.924+0000 7fd6efe00640 1 -- 172.21.15.196:0/851109638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6f81629c0 msgr2=0x7fd6f80b8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:47.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.924+0000 7fd6efe00640 1 --2- 172.21.15.196:0/851109638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6f81629c0 0x7fd6f80b8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:47.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.924+0000 7fd6efe00640 1 -- 172.21.15.196:0/851109638 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f81f97b0 con 0x7fd6f8163740 2024-04-28T13:35:47.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.924+0000 7fd6efe00640 1 --2- 172.21.15.196:0/851109638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f8163740 0x7fd6f8161010 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000c410 tx=0x7fd6e000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:47.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.924+0000 7fd6f5600640 1 -- 172.21.15.196:0/851109638 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00074e0 con 0x7fd6f8163740 2024-04-28T13:35:47.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.924+0000 7fd6f5600640 1 -- 172.21.15.196:0/851109638 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e00047e0 con 0x7fd6f8163740 2024-04-28T13:35:47.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.924+0000 7fd6f5600640 1 -- 172.21.15.196:0/851109638 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0004bd0 con 0x7fd6f8163740 2024-04-28T13:35:47.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.924+0000 7fd6fe200640 1 -- 172.21.15.196:0/851109638 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f81f9a90 con 0x7fd6f8163740 2024-04-28T13:35:47.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.925+0000 7fd6fe200640 1 -- 172.21.15.196:0/851109638 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f80b8d80 con 0x7fd6f8163740 2024-04-28T13:35:47.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.926+0000 7fd6fe200640 1 -- 172.21.15.196:0/851109638 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f8069ae0 con 0x7fd6f8163740 2024-04-28T13:35:47.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.927+0000 7fd6f5600640 1 -- 172.21.15.196:0/851109638 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd6e0011070 con 0x7fd6f8163740 2024-04-28T13:35:47.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.927+0000 7fd6f5600640 1 --2- 172.21.15.196:0/851109638 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd6d406c980 0x7fd6d406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:47.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.928+0000 7fd6f5600640 1 -- 172.21.15.196:0/851109638 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fd6e008b980 con 0x7fd6f8163740 2024-04-28T13:35:47.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.928+0000 7fd6fce00640 1 --2- 172.21.15.196:0/851109638 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd6d406c980 0x7fd6d406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:47.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.930+0000 7fd6fce00640 1 --2- 172.21.15.196:0/851109638 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd6d406c980 0x7fd6d406ee40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8006fd0 tx=0x7fd6e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:47.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:47.932+0000 7fd6f5600640 1 -- 172.21.15.196:0/851109638 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd6f8069ae0 con 0x7fd6f8163740 2024-04-28T13:35:48.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.131+0000 7fd6fe200640 1 -- 172.21.15.196:0/851109638 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6f806a290 con 0x7fd6f8163740 2024-04-28T13:35:48.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.133+0000 7fd6f5600640 1 -- 172.21.15.196:0/851109638 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16741 (secure 0 0 0) 0x7fd6e0059c80 con 0x7fd6f8163740 2024-04-28T13:35:48.133 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:35:48.134 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":47,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:34:04.329925+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:33:31.193265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-28T13:33:32.415392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.32","mappings":[{"from":0,"to":5}]},{"pgid":"7.38","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:35:48.136 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.136+0000 7fd6fe200640 1 -- 172.21.15.196:0/851109638 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd6d406c980 msgr2=0x7fd6d406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:48.136 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.136+0000 7fd6fe200640 1 --2- 172.21.15.196:0/851109638 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd6d406c980 0x7fd6d406ee40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8006fd0 tx=0x7fd6e8008040 comp rx=0 tx=0).stop 2024-04-28T13:35:48.136 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.136+0000 7fd6fe200640 1 -- 172.21.15.196:0/851109638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f8163740 msgr2=0x7fd6f8161010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:48.136 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.136+0000 7fd6fe200640 1 --2- 172.21.15.196:0/851109638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f8163740 0x7fd6f8161010 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000c410 tx=0x7fd6e000c8d0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.137 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.136+0000 7fd6fe200640 1 -- 172.21.15.196:0/851109638 shutdown_connections 2024-04-28T13:35:48.137 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.136+0000 7fd6fe200640 1 --2- 172.21.15.196:0/851109638 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fd6d406c980 0x7fd6d406ee40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.137 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.136+0000 7fd6fe200640 1 --2- 172.21.15.196:0/851109638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6f8163bf0 0x7fd6f81f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.137 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.136+0000 7fd6fe200640 1 --2- 172.21.15.196:0/851109638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6f8163740 0x7fd6f8161010 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.137 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.136+0000 7fd6fe200640 1 --2- 172.21.15.196:0/851109638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6f81629c0 0x7fd6f80b8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.137 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.136+0000 7fd6fe200640 1 -- 172.21.15.196:0/851109638 >> 172.21.15.196:0/851109638 conn(0x7fd6f809e570 msgr2=0x7fd6f80b6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:48.137 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.136+0000 7fd6fe200640 1 -- 172.21.15.196:0/851109638 shutdown_connections 2024-04-28T13:35:48.137 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.136+0000 7fd6fe200640 1 -- 172.21.15.196:0/851109638 wait complete. 2024-04-28T13:35:48.157 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T13:31:10.214495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T13:33:31.193265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T13:33:32.415392+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T13:35:48.158 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:35:48.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.361+0000 7fa018400640 1 Processor -- start 2024-04-28T13:35:48.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.361+0000 7fa018400640 1 -- start start 2024-04-28T13:35:48.362 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.363+0000 7fa018400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa01006f340 0x7fa0100b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:48.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.363+0000 7fa018400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0100d0300 0x7fa01006f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:48.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.363+0000 7fa018400640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa01006fc60 con 0x7fa0100b3e50 2024-04-28T13:35:48.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.363+0000 7fa018400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa0100b46a0 con 0x7fa01006f340 2024-04-28T13:35:48.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.363+0000 7fa018400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa0100b5db0 con 0x7fa0100d0300 2024-04-28T13:35:48.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.363+0000 7fa016400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa0100b3e50 0x7fa0100720f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:55670/0 (socket says 172.21.15.196:55670) 2024-04-28T13:35:48.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.363+0000 7fa016400640 1 -- 172.21.15.196:0/1591736874 learned_addr learned my addr 172.21.15.196:0/1591736874 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:48.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.363+0000 7fa015a00640 1 --2- 172.21.15.196:0/1591736874 >> v2:172.21.15.203:3300/0 conn(0x7fa01006f340 0x7fa0100b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:48.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.363+0000 7fa015000640 1 --2- 172.21.15.196:0/1591736874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0100d0300 0x7fa01006f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:48.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.363+0000 7fa00fe00640 1 -- 172.21.15.196:0/1591736874 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1001933814 0 0) 0x7fa01006fc60 con 0x7fa0100b3e50 2024-04-28T13:35:48.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.363+0000 7fa00fe00640 1 -- 172.21.15.196:0/1591736874 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ff4003520 con 0x7fa0100b3e50 2024-04-28T13:35:48.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa00fe00640 1 -- 172.21.15.196:0/1591736874 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1359697361 0 0) 0x7f9ff4003520 con 0x7fa0100b3e50 2024-04-28T13:35:48.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa00fe00640 1 -- 172.21.15.196:0/1591736874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0100d0300 msgr2=0x7fa01006f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:48.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa00fe00640 1 --2- 172.21.15.196:0/1591736874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0100d0300 0x7fa01006f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa00fe00640 1 -- 172.21.15.196:0/1591736874 >> v2:172.21.15.203:3300/0 conn(0x7fa01006f340 msgr2=0x7fa0100b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:48.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa00fe00640 1 --2- 172.21.15.196:0/1591736874 >> v2:172.21.15.203:3300/0 conn(0x7fa01006f340 0x7fa0100b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa00fe00640 1 -- 172.21.15.196:0/1591736874 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0100b6090 con 0x7fa0100b3e50 2024-04-28T13:35:48.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa00fe00640 1 -- 172.21.15.196:0/1591736874 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fa0080019f0 con 0x7fa0100b3e50 2024-04-28T13:35:48.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa015000640 1 --2- 172.21.15.196:0/1591736874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0100d0300 0x7fa01006f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:35:48.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa00fe00640 1 -- 172.21.15.196:0/1591736874 >> v1:172.21.15.196:6789/0 conn(0x7fa0100b3e50 legacy=0x7fa0100720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:48.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa00fe00640 1 --2- 172.21.15.196:0/1591736874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff40039b0 0x7f9ff4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:48.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa00fe00640 1 --2- 172.21.15.196:0/1591736874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0100d0300 0x7f9ff4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:48.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa00fe00640 1 --2- 172.21.15.196:0/1591736874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ff4004d40 0x7f9ff40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:48.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa00fe00640 1 -- 172.21.15.196:0/1591736874 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0100b46a0 con 0x7f9ff40039b0 2024-04-28T13:35:48.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa00fe00640 1 -- 172.21.15.196:0/1591736874 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa0100b5db0 con 0x7f9ff4004d40 2024-04-28T13:35:48.367 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa00fe00640 1 -- 172.21.15.196:0/1591736874 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff400dcd0 con 0x7fa0100d0300 2024-04-28T13:35:48.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.364+0000 7fa015a00640 1 --2- 172.21.15.196:0/1591736874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff40039b0 0x7f9ff4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:48.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.365+0000 7fa015000640 1 --2- 172.21.15.196:0/1591736874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0100d0300 0x7f9ff4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:48.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.365+0000 7fa015a00640 1 -- 172.21.15.196:0/1591736874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0100d0300 msgr2=0x7f9ff4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:48.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.365+0000 7fa016400640 1 --2- 172.21.15.196:0/1591736874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ff4004d40 0x7f9ff40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:48.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.365+0000 7fa015a00640 1 --2- 172.21.15.196:0/1591736874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0100d0300 0x7f9ff4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.365+0000 7fa015a00640 1 -- 172.21.15.196:0/1591736874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ff4004d40 msgr2=0x7f9ff40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:48.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.365+0000 7fa015a00640 1 --2- 172.21.15.196:0/1591736874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ff4004d40 0x7f9ff40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.365+0000 7fa015a00640 1 -- 172.21.15.196:0/1591736874 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff4003520 con 0x7f9ff40039b0 2024-04-28T13:35:48.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.365+0000 7fa015000640 1 --2- 172.21.15.196:0/1591736874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0100d0300 0x7f9ff4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:35:48.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.366+0000 7fa015a00640 1 --2- 172.21.15.196:0/1591736874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff40039b0 0x7f9ff4003dd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa000009af0 tx=0x7f9ff4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27afa5a55a89fcb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:48.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.366+0000 7fa00fe00640 1 -- 172.21.15.196:0/1591736874 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa000036070 con 0x7f9ff40039b0 2024-04-28T13:35:48.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.366+0000 7fa00fe00640 1 -- 172.21.15.196:0/1591736874 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa000031400 con 0x7f9ff40039b0 2024-04-28T13:35:48.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.366+0000 7fa00fe00640 1 -- 172.21.15.196:0/1591736874 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa000009690 con 0x7f9ff40039b0 2024-04-28T13:35:48.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.367+0000 7fa018400640 1 -- 172.21.15.196:0/1591736874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff40039b0 msgr2=0x7f9ff4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:48.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.367+0000 7fa018400640 1 --2- 172.21.15.196:0/1591736874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ff40039b0 0x7f9ff4003dd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa000009af0 tx=0x7f9ff4005830 comp rx=0 tx=0).stop 2024-04-28T13:35:48.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.367+0000 7fa016400640 1 -- 172.21.15.196:0/1591736874 reap_dead start 2024-04-28T13:35:48.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.367+0000 7fa018400640 1 -- 172.21.15.196:0/1591736874 shutdown_connections 2024-04-28T13:35:48.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.367+0000 7fa018400640 1 -- 172.21.15.196:0/1591736874 >> 172.21.15.196:0/1591736874 conn(0x7fa0100cd8a0 msgr2=0x7fa0100cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:48.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.367+0000 7fa018400640 1 -- 172.21.15.196:0/1591736874 shutdown_connections 2024-04-28T13:35:48.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.367+0000 7fa018400640 1 -- 172.21.15.196:0/1591736874 wait complete. 2024-04-28T13:35:48.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.369+0000 7fa018400640 1 Processor -- start 2024-04-28T13:35:48.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.369+0000 7fa018400640 1 -- start start 2024-04-28T13:35:48.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.370+0000 7fa018400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ff40039b0 0x7fa01013c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:48.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.370+0000 7fa018400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0100cab50 0x7fa0100ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:48.373 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.370+0000 7fa018400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0100725d0 0x7fa010072a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:48.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.370+0000 7fa018400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0100b42e0 con 0x7fa0100cab50 2024-04-28T13:35:48.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.370+0000 7fa018400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa0100b6650 con 0x7fa0100725d0 2024-04-28T13:35:48.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.370+0000 7fa018400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa0100b4460 con 0x7f9ff40039b0 2024-04-28T13:35:48.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.370+0000 7fa015000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0100cab50 0x7fa0100ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:48.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.370+0000 7fa015000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0100cab50 0x7fa0100ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58780/0 (socket says 172.21.15.196:58780) 2024-04-28T13:35:48.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.370+0000 7fa015000640 1 -- 172.21.15.196:0/4283914358 learned_addr learned my addr 172.21.15.196:0/4283914358 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:48.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.370+0000 7fa015a00640 1 --2- 172.21.15.196:0/4283914358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ff40039b0 0x7fa01013c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:48.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.370+0000 7fa016400640 1 --2- 172.21.15.196:0/4283914358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0100725d0 0x7fa010072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:48.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.370+0000 7fa015000640 1 -- 172.21.15.196:0/4283914358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ff40039b0 msgr2=0x7fa01013c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:48.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.370+0000 7fa015000640 1 --2- 172.21.15.196:0/4283914358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ff40039b0 0x7fa01013c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.370+0000 7fa015000640 1 -- 172.21.15.196:0/4283914358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0100725d0 msgr2=0x7fa010072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:48.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.370+0000 7fa015000640 1 --2- 172.21.15.196:0/4283914358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0100725d0 0x7fa010072a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.371+0000 7fa015000640 1 -- 172.21.15.196:0/4283914358 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01015b060 con 0x7fa0100cab50 2024-04-28T13:35:48.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.371+0000 7fa015a00640 1 --2- 172.21.15.196:0/4283914358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ff40039b0 0x7fa01013c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:35:48.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.371+0000 7fa015000640 1 --2- 172.21.15.196:0/4283914358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0100cab50 0x7fa0100ca240 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00f550 tx=0x7f9ffc00fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:48.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.371+0000 7fa00d600640 1 -- 172.21.15.196:0/4283914358 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc018070 con 0x7fa0100cab50 2024-04-28T13:35:48.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.371+0000 7fa00d600640 1 -- 172.21.15.196:0/4283914358 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ffc014890 con 0x7fa0100cab50 2024-04-28T13:35:48.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.371+0000 7fa00d600640 1 -- 172.21.15.196:0/4283914358 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc014c40 con 0x7fa0100cab50 2024-04-28T13:35:48.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.371+0000 7fa018400640 1 -- 172.21.15.196:0/4283914358 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0100cafe0 con 0x7fa0100cab50 2024-04-28T13:35:48.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.371+0000 7fa018400640 1 -- 172.21.15.196:0/4283914358 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa01013e0c0 con 0x7fa0100cab50 2024-04-28T13:35:48.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.372+0000 7fa018400640 1 -- 172.21.15.196:0/4283914358 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa01006a290 con 0x7fa0100cab50 2024-04-28T13:35:48.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.373+0000 7fa00d600640 1 -- 172.21.15.196:0/4283914358 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f9ffc01b070 con 0x7fa0100cab50 2024-04-28T13:35:48.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.375+0000 7fa00d600640 1 --2- 172.21.15.196:0/4283914358 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9ff006c9b0 0x7f9ff006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:48.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.375+0000 7fa00d600640 1 -- 172.21.15.196:0/4283914358 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f9ffc08bfc0 con 0x7fa0100cab50 2024-04-28T13:35:48.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.375+0000 7fa015a00640 1 --2- 172.21.15.196:0/4283914358 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9ff006c9b0 0x7f9ff006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:48.378 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.377+0000 7fa015a00640 1 --2- 172.21.15.196:0/4283914358 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9ff006c9b0 0x7f9ff006ee70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa000033000 tx=0x7fa000008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:48.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.380+0000 7fa00d600640 1 -- 172.21.15.196:0/4283914358 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa01015b060 con 0x7fa0100cab50 2024-04-28T13:35:48.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.580+0000 7fa018400640 1 -- 172.21.15.196:0/4283914358 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa01013e780 con 0x7fa0100cab50 2024-04-28T13:35:48.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.581+0000 7fa00d600640 1 -- 172.21.15.196:0/4283914358 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7f9ffc05a240 con 0x7fa0100cab50 2024-04-28T13:35:48.582 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2024-04-28T13:35:48.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.584+0000 7fa018400640 1 -- 172.21.15.196:0/4283914358 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9ff006c9b0 msgr2=0x7f9ff006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:48.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.584+0000 7fa018400640 1 --2- 172.21.15.196:0/4283914358 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9ff006c9b0 0x7f9ff006ee70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa000033000 tx=0x7fa000008030 comp rx=0 tx=0).stop 2024-04-28T13:35:48.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.584+0000 7fa018400640 1 -- 172.21.15.196:0/4283914358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0100cab50 msgr2=0x7fa0100ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:48.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.584+0000 7fa018400640 1 --2- 172.21.15.196:0/4283914358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0100cab50 0x7fa0100ca240 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00f550 tx=0x7f9ffc00fa10 comp rx=0 tx=0).stop 2024-04-28T13:35:48.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.584+0000 7fa018400640 1 -- 172.21.15.196:0/4283914358 shutdown_connections 2024-04-28T13:35:48.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.584+0000 7fa018400640 1 --2- 172.21.15.196:0/4283914358 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9ff006c9b0 0x7f9ff006ee70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.584+0000 7fa018400640 1 --2- 172.21.15.196:0/4283914358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0100725d0 0x7fa010072a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.584+0000 7fa018400640 1 --2- 172.21.15.196:0/4283914358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0100cab50 0x7fa0100ca240 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.584+0000 7fa018400640 1 --2- 172.21.15.196:0/4283914358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ff40039b0 0x7fa01013c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.585 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.584+0000 7fa018400640 1 -- 172.21.15.196:0/4283914358 >> 172.21.15.196:0/4283914358 conn(0x7fa0100cd8a0 msgr2=0x7fa01006f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:48.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.584+0000 7fa018400640 1 -- 172.21.15.196:0/4283914358 shutdown_connections 2024-04-28T13:35:48.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.584+0000 7fa018400640 1 -- 172.21.15.196:0/4283914358 wait complete. 2024-04-28T13:35:48.603 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:35:48.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.798+0000 7f24abe00640 1 Processor -- start 2024-04-28T13:35:48.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.798+0000 7f24abe00640 1 -- start start 2024-04-28T13:35:48.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.799+0000 7f24abe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f24a40b4eb0 0x7f24a40b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:48.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.799+0000 7f24abe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24a40ceff0 0x7f24a40cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:48.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.799+0000 7f24abe00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a415dca0 con 0x7f24a4143840 2024-04-28T13:35:48.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.799+0000 7f24abe00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f24a4143c20 con 0x7f24a40b4eb0 2024-04-28T13:35:48.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.799+0000 7f24abe00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f24a415df80 con 0x7f24a40ceff0 2024-04-28T13:35:48.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.800+0000 7f24a8a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f24a40b4eb0 0x7f24a40b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:48.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.800+0000 7f24a8a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f24a40b4eb0 0x7f24a40b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58794/0 (socket says 172.21.15.196:58794) 2024-04-28T13:35:48.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.800+0000 7f24a8a00640 1 -- 172.21.15.196:0/2210545433 learned_addr learned my addr 172.21.15.196:0/2210545433 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:48.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.800+0000 7f24a9e00640 1 --2- 172.21.15.196:0/2210545433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24a40ceff0 0x7f24a40cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:48.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.800+0000 7f24a8a00640 1 -- 172.21.15.196:0/2210545433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24a40ceff0 msgr2=0x7f24a40cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:48.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.800+0000 7f24a8a00640 1 --2- 172.21.15.196:0/2210545433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24a40ceff0 0x7f24a40cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.800+0000 7f24a8a00640 1 -- 172.21.15.196:0/2210545433 >> v1:172.21.15.203:6789/0 conn(0x7f24a4143840 legacy=0x7f24a40b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:48.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.800+0000 7f24a8a00640 1 -- 172.21.15.196:0/2210545433 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a415e260 con 0x7f24a40b4eb0 2024-04-28T13:35:48.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.800+0000 7f24a9e00640 1 --2- 172.21.15.196:0/2210545433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24a40ceff0 0x7f24a40cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:35:48.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.801+0000 7f24a8a00640 1 --2- 172.21.15.196:0/2210545433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a40b4eb0 0x7f24a40b5330 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2490009870 tx=0x7f249002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0663a4c2d646fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:48.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.802+0000 7f24a3e00640 1 -- 172.21.15.196:0/2210545433 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490037070 con 0x7f24a40b4eb0 2024-04-28T13:35:48.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.802+0000 7f24a3e00640 1 -- 172.21.15.196:0/2210545433 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2490032440 con 0x7f24a40b4eb0 2024-04-28T13:35:48.802 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.802+0000 7f24a3e00640 1 -- 172.21.15.196:0/2210545433 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490032810 con 0x7f24a40b4eb0 2024-04-28T13:35:48.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.802+0000 7f24abe00640 1 -- 172.21.15.196:0/2210545433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a40b4eb0 msgr2=0x7f24a40b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:48.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.802+0000 7f24abe00640 1 --2- 172.21.15.196:0/2210545433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a40b4eb0 0x7f24a40b5330 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2490009870 tx=0x7f249002a210 comp rx=0 tx=0).stop 2024-04-28T13:35:48.803 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.803+0000 7f24abe00640 1 -- 172.21.15.196:0/2210545433 shutdown_connections 2024-04-28T13:35:48.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.803+0000 7f24abe00640 1 --2- 172.21.15.196:0/2210545433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24a40ceff0 0x7f24a40cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.803+0000 7f24abe00640 1 --2- 172.21.15.196:0/2210545433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a40b4eb0 0x7f24a40b5330 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.803+0000 7f24abe00640 1 -- 172.21.15.196:0/2210545433 >> 172.21.15.196:0/2210545433 conn(0x7f24a40cd8a0 msgr2=0x7f24a40cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:48.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.803+0000 7f24abe00640 1 -- 172.21.15.196:0/2210545433 shutdown_connections 2024-04-28T13:35:48.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.803+0000 7f24abe00640 1 -- 172.21.15.196:0/2210545433 wait complete. 2024-04-28T13:35:48.804 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.804+0000 7f24abe00640 1 Processor -- start 2024-04-28T13:35:48.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.805+0000 7f24abe00640 1 -- start start 2024-04-28T13:35:48.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.806+0000 7f24abe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24a40b4eb0 0x7f24a415aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:48.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.806+0000 7f24abe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a40ceff0 0x7f24a4136120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:48.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.806+0000 7f24abe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24a4143840 0x7f24a4136660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:48.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.806+0000 7f24abe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24a415d8c0 con 0x7f24a40ceff0 2024-04-28T13:35:48.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.806+0000 7f24abe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f24a40b46f0 con 0x7f24a4143840 2024-04-28T13:35:48.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.806+0000 7f24abe00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f24a415da40 con 0x7f24a40b4eb0 2024-04-28T13:35:48.806 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.806+0000 7f24a8a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a40ceff0 0x7f24a4136120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:48.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.806+0000 7f24a8a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a40ceff0 0x7f24a4136120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58810/0 (socket says 172.21.15.196:58810) 2024-04-28T13:35:48.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.806+0000 7f24a8a00640 1 -- 172.21.15.196:0/3998417571 learned_addr learned my addr 172.21.15.196:0/3998417571 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:48.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.806+0000 7f24a8a00640 1 -- 172.21.15.196:0/3998417571 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24a40b4eb0 msgr2=0x7f24a415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:48.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.806+0000 7f24a8a00640 1 --2- 172.21.15.196:0/3998417571 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24a40b4eb0 0x7f24a415aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.806+0000 7f24a8a00640 1 -- 172.21.15.196:0/3998417571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24a4143840 msgr2=0x7f24a4136660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:48.807 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.806+0000 7f24a8a00640 1 --2- 172.21.15.196:0/3998417571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24a4143840 0x7f24a4136660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:48.808 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.806+0000 7f24a8a00640 1 -- 172.21.15.196:0/3998417571 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a4145b30 con 0x7f24a40ceff0 2024-04-28T13:35:48.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.807+0000 7f24a8a00640 1 --2- 172.21.15.196:0/3998417571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a40ceff0 0x7f24a4136120 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2490000c00 tx=0x7f24900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:48.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.807+0000 7f24a1600640 1 -- 172.21.15.196:0/3998417571 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490037040 con 0x7f24a40ceff0 2024-04-28T13:35:48.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.807+0000 7f24a1600640 1 -- 172.21.15.196:0/3998417571 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2490042070 con 0x7f24a40ceff0 2024-04-28T13:35:48.811 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.807+0000 7f24a1600640 1 -- 172.21.15.196:0/3998417571 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490030bb0 con 0x7f24a40ceff0 2024-04-28T13:35:48.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.807+0000 7f24abe00640 1 -- 172.21.15.196:0/3998417571 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a4145e10 con 0x7f24a40ceff0 2024-04-28T13:35:48.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.807+0000 7f24abe00640 1 -- 172.21.15.196:0/3998417571 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a41463d0 con 0x7f24a40ceff0 2024-04-28T13:35:48.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.808+0000 7f24abe00640 1 -- 172.21.15.196:0/3998417571 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a406a290 con 0x7f24a40ceff0 2024-04-28T13:35:48.812 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.809+0000 7f24a1600640 1 -- 172.21.15.196:0/3998417571 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f2490047460 con 0x7f24a40ceff0 2024-04-28T13:35:48.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.810+0000 7f24a1600640 1 --2- 172.21.15.196:0/3998417571 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f248406c980 0x7f248406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:48.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.810+0000 7f24a1600640 1 -- 172.21.15.196:0/3998417571 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f24900ac960 con 0x7f24a40ceff0 2024-04-28T13:35:48.814 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.811+0000 7f24a9400640 1 --2- 172.21.15.196:0/3998417571 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f248406c980 0x7f248406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:48.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.812+0000 7f24a9400640 1 --2- 172.21.15.196:0/3998417571 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f248406c980 0x7f248406ee40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2498006fd0 tx=0x7f2498006bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:48.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:48.814+0000 7f24a1600640 1 -- 172.21.15.196:0/3998417571 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f249007ac60 con 0x7f24a40ceff0 2024-04-28T13:35:49.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.009+0000 7f24abe00640 1 -- 172.21.15.196:0/3998417571 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f24a40ce420 con 0x7f24a40ceff0 2024-04-28T13:35:49.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.010+0000 7f24a1600640 1 -- 172.21.15.196:0/3998417571 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) ==== 104+0+11 (secure 0 0 0) 0x7f249007a560 con 0x7f24a40ceff0 2024-04-28T13:35:49.010 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 32 2024-04-28T13:35:49.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.013+0000 7f24abe00640 1 -- 172.21.15.196:0/3998417571 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f248406c980 msgr2=0x7f248406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:49.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.013+0000 7f24abe00640 1 --2- 172.21.15.196:0/3998417571 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f248406c980 0x7f248406ee40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2498006fd0 tx=0x7f2498006bc0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.013+0000 7f24abe00640 1 -- 172.21.15.196:0/3998417571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a40ceff0 msgr2=0x7f24a4136120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:49.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.013+0000 7f24abe00640 1 --2- 172.21.15.196:0/3998417571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a40ceff0 0x7f24a4136120 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2490000c00 tx=0x7f24900046c0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.013+0000 7f24abe00640 1 -- 172.21.15.196:0/3998417571 shutdown_connections 2024-04-28T13:35:49.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.013+0000 7f24abe00640 1 --2- 172.21.15.196:0/3998417571 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f248406c980 0x7f248406ee40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.013+0000 7f24abe00640 1 --2- 172.21.15.196:0/3998417571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24a4143840 0x7f24a4136660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.013+0000 7f24abe00640 1 --2- 172.21.15.196:0/3998417571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a40ceff0 0x7f24a4136120 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.013+0000 7f24abe00640 1 --2- 172.21.15.196:0/3998417571 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24a40b4eb0 0x7f24a415aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.013+0000 7f24abe00640 1 -- 172.21.15.196:0/3998417571 >> 172.21.15.196:0/3998417571 conn(0x7f24a40cd8a0 msgr2=0x7f24a40b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:49.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.013+0000 7f24abe00640 1 -- 172.21.15.196:0/3998417571 shutdown_connections 2024-04-28T13:35:49.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.013+0000 7f24abe00640 1 -- 172.21.15.196:0/3998417571 wait complete. 2024-04-28T13:35:49.029 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:35:49.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.225+0000 7fa6b9600640 1 Processor -- start 2024-04-28T13:35:49.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.225+0000 7fa6b9600640 1 -- start start 2024-04-28T13:35:49.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.227+0000 7fa6b9600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa6b4072600 0x7fa6b4073de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:49.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.227+0000 7fa6b9600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa6b40b5930 0x7fa6b406e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:49.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.227+0000 7fa6b9600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6b40b5630 con 0x7fa6b40d0820 2024-04-28T13:35:49.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.227+0000 7fa6b9600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa6b406e050 con 0x7fa6b4072600 2024-04-28T13:35:49.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.227+0000 7fa6b9600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa6b40b4d90 con 0x7fa6b40b5930 2024-04-28T13:35:49.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.227+0000 7fa6b3e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa6b40d0820 0x7fa6b4071ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:55672/0 (socket says 172.21.15.196:55672) 2024-04-28T13:35:49.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.227+0000 7fa6b3e00640 1 -- 172.21.15.196:0/432165692 learned_addr learned my addr 172.21.15.196:0/432165692 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:49.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.228+0000 7fa6ab400640 1 --2- 172.21.15.196:0/432165692 >> v2:172.21.15.196:3300/0 conn(0x7fa6b4072600 0x7fa6b4073de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:49.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.228+0000 7fa6b8c00640 1 --2- 172.21.15.196:0/432165692 >> v2:172.21.15.203:3300/0 conn(0x7fa6b40b5930 0x7fa6b406e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:49.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.228+0000 7fa6b3400640 1 -- 172.21.15.196:0/432165692 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1005598483 0 0) 0x7fa6b40b5630 con 0x7fa6b40d0820 2024-04-28T13:35:49.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.228+0000 7fa6ab400640 1 -- 172.21.15.196:0/432165692 >> v2:172.21.15.203:3300/0 conn(0x7fa6b40b5930 msgr2=0x7fa6b406e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:49.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.228+0000 7fa6ab400640 1 --2- 172.21.15.196:0/432165692 >> v2:172.21.15.203:3300/0 conn(0x7fa6b40b5930 0x7fa6b406e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.228+0000 7fa6ab400640 1 -- 172.21.15.196:0/432165692 >> v1:172.21.15.196:6789/0 conn(0x7fa6b40d0820 legacy=0x7fa6b4071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:49.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.228+0000 7fa6ab400640 1 -- 172.21.15.196:0/432165692 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b40b5070 con 0x7fa6b4072600 2024-04-28T13:35:49.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.229+0000 7fa6ab400640 1 --2- 172.21.15.196:0/432165692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b4072600 0x7fa6b4073de0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40b3d40 tx=0x7fa69c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6e366a69b693d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:49.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.229+0000 7fa6b3400640 1 -- 172.21.15.196:0/432165692 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c037070 con 0x7fa6b4072600 2024-04-28T13:35:49.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.229+0000 7fa6b3400640 1 -- 172.21.15.196:0/432165692 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa69c032440 con 0x7fa6b4072600 2024-04-28T13:35:49.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.229+0000 7fa6b3400640 1 -- 172.21.15.196:0/432165692 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c02ae60 con 0x7fa6b4072600 2024-04-28T13:35:49.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.230+0000 7fa6b9600640 1 -- 172.21.15.196:0/432165692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b4072600 msgr2=0x7fa6b4073de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:49.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.230+0000 7fa6b9600640 1 --2- 172.21.15.196:0/432165692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b4072600 0x7fa6b4073de0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40b3d40 tx=0x7fa69c02a110 comp rx=0 tx=0).stop 2024-04-28T13:35:49.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.230+0000 7fa6b9600640 1 -- 172.21.15.196:0/432165692 shutdown_connections 2024-04-28T13:35:49.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.230+0000 7fa6b9600640 1 --2- 172.21.15.196:0/432165692 >> v2:172.21.15.203:3300/0 conn(0x7fa6b40b5930 0x7fa6b406e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.230+0000 7fa6b9600640 1 --2- 172.21.15.196:0/432165692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b4072600 0x7fa6b4073de0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.230+0000 7fa6b9600640 1 -- 172.21.15.196:0/432165692 >> 172.21.15.196:0/432165692 conn(0x7fa6b40b2990 msgr2=0x7fa6b40b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:49.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.230+0000 7fa6b9600640 1 -- 172.21.15.196:0/432165692 shutdown_connections 2024-04-28T13:35:49.232 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.230+0000 7fa6b9600640 1 -- 172.21.15.196:0/432165692 wait complete. 2024-04-28T13:35:49.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.232+0000 7fa6b9600640 1 Processor -- start 2024-04-28T13:35:49.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.232+0000 7fa6b9600640 1 -- start start 2024-04-28T13:35:49.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.233+0000 7fa6b9600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b4072600 0x7fa6b4161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:49.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.233+0000 7fa6b9600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b40b5930 0x7fa6b409da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:49.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.233+0000 7fa6b9600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6b40d0820 0x7fa6b409df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:49.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.233+0000 7fa6b9600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b40b4b30 con 0x7fa6b40b5930 2024-04-28T13:35:49.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.233+0000 7fa6b9600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b40b49b0 con 0x7fa6b4072600 2024-04-28T13:35:49.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.233+0000 7fa6b9600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b406ea20 con 0x7fa6b40d0820 2024-04-28T13:35:49.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.233+0000 7fa6ab400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b40b5930 0x7fa6b409da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:49.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.233+0000 7fa6ab400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b40b5930 0x7fa6b409da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58832/0 (socket says 172.21.15.196:58832) 2024-04-28T13:35:49.234 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.233+0000 7fa6ab400640 1 -- 172.21.15.196:0/2935234306 learned_addr learned my addr 172.21.15.196:0/2935234306 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:49.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.234+0000 7fa6ab400640 1 -- 172.21.15.196:0/2935234306 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6b40d0820 msgr2=0x7fa6b409df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:49.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.234+0000 7fa6ab400640 1 --2- 172.21.15.196:0/2935234306 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6b40d0820 0x7fa6b409df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.234+0000 7fa6ab400640 1 -- 172.21.15.196:0/2935234306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b4072600 msgr2=0x7fa6b4161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:49.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.234+0000 7fa6ab400640 1 --2- 172.21.15.196:0/2935234306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b4072600 0x7fa6b4161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.234+0000 7fa6ab400640 1 -- 172.21.15.196:0/2935234306 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b41655a0 con 0x7fa6b40b5930 2024-04-28T13:35:49.235 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.234+0000 7fa6ab400640 1 --2- 172.21.15.196:0/2935234306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b40b5930 0x7fa6b409da50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009890 tx=0x7fa69c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:49.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.234+0000 7fa6b0c00640 1 -- 172.21.15.196:0/2935234306 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c037040 con 0x7fa6b40b5930 2024-04-28T13:35:49.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.234+0000 7fa6b0c00640 1 -- 172.21.15.196:0/2935234306 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa69c042070 con 0x7fa6b40b5930 2024-04-28T13:35:49.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.234+0000 7fa6b0c00640 1 -- 172.21.15.196:0/2935234306 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c030bd0 con 0x7fa6b40b5930 2024-04-28T13:35:49.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.234+0000 7fa6b9600640 1 -- 172.21.15.196:0/2935234306 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b40c5d30 con 0x7fa6b40b5930 2024-04-28T13:35:49.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.234+0000 7fa6b9600640 1 -- 172.21.15.196:0/2935234306 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b40c62f0 con 0x7fa6b40b5930 2024-04-28T13:35:49.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.235+0000 7fa6b9600640 1 -- 172.21.15.196:0/2935234306 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b4050730 con 0x7fa6b40b5930 2024-04-28T13:35:49.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.236+0000 7fa6b0c00640 1 -- 172.21.15.196:0/2935234306 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa69c047460 con 0x7fa6b40b5930 2024-04-28T13:35:49.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.237+0000 7fa6b0c00640 1 --2- 172.21.15.196:0/2935234306 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa69006c980 0x7fa69006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:49.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.237+0000 7fa6b0c00640 1 -- 172.21.15.196:0/2935234306 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fa69c0ac810 con 0x7fa6b40b5930 2024-04-28T13:35:49.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.238+0000 7fa6b3e00640 1 --2- 172.21.15.196:0/2935234306 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa69006c980 0x7fa69006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:49.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.239+0000 7fa6b3e00640 1 --2- 172.21.15.196:0/2935234306 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa69006c980 0x7fa69006ee40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4007fd0 tx=0x7fa6a4007ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:49.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.242+0000 7fa6b0c00640 1 -- 172.21.15.196:0/2935234306 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa69c032e20 con 0x7fa6b40b5930 2024-04-28T13:35:49.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.497+0000 7fa6b9600640 1 -- 172.21.15.196:0/2935234306 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa6b4070150 con 0x7fa6b40b5930 2024-04-28T13:35:49.497 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.497+0000 7fa6b0c00640 1 -- 172.21.15.196:0/2935234306 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7fa69c07aba0 con 0x7fa6b40b5930 2024-04-28T13:35:49.497 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 64 2024-04-28T13:35:49.500 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.500+0000 7fa6b9600640 1 -- 172.21.15.196:0/2935234306 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa69006c980 msgr2=0x7fa69006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:49.500 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.500+0000 7fa6b9600640 1 --2- 172.21.15.196:0/2935234306 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa69006c980 0x7fa69006ee40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4007fd0 tx=0x7fa6a4007ce0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.500 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.500+0000 7fa6b9600640 1 -- 172.21.15.196:0/2935234306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b40b5930 msgr2=0x7fa6b409da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:49.500 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.500+0000 7fa6b9600640 1 --2- 172.21.15.196:0/2935234306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b40b5930 0x7fa6b409da50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009890 tx=0x7fa69c0046c0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.500 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.500+0000 7fa6b9600640 1 -- 172.21.15.196:0/2935234306 shutdown_connections 2024-04-28T13:35:49.500 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.500+0000 7fa6b9600640 1 --2- 172.21.15.196:0/2935234306 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa69006c980 0x7fa69006ee40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.501 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.500+0000 7fa6b9600640 1 --2- 172.21.15.196:0/2935234306 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6b40d0820 0x7fa6b409df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.501 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.500+0000 7fa6b9600640 1 --2- 172.21.15.196:0/2935234306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6b40b5930 0x7fa6b409da50 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.501 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.500+0000 7fa6b9600640 1 --2- 172.21.15.196:0/2935234306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b4072600 0x7fa6b4161ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.501 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.500+0000 7fa6b9600640 1 -- 172.21.15.196:0/2935234306 >> 172.21.15.196:0/2935234306 conn(0x7fa6b40b2990 msgr2=0x7fa6b40aeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:49.501 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.500+0000 7fa6b9600640 1 -- 172.21.15.196:0/2935234306 shutdown_connections 2024-04-28T13:35:49.501 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.500+0000 7fa6b9600640 1 -- 172.21.15.196:0/2935234306 wait complete. 2024-04-28T13:35:49.518 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T13:35:49.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.721+0000 7f7afce00640 1 Processor -- start 2024-04-28T13:35:49.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.721+0000 7f7afce00640 1 -- start start 2024-04-28T13:35:49.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.722+0000 7f7afce00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7af80b5ee0 0x7f7af80b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:49.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.722+0000 7f7afce00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7af80d1760 0x7f7af806df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:49.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.722+0000 7f7afce00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af815e380 con 0x7f7af809ec80 2024-04-28T13:35:49.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.722+0000 7f7afce00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f7af80b4b20 con 0x7f7af80d1760 2024-04-28T13:35:49.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.722+0000 7f7afce00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7af80b3f30 con 0x7f7af80b5ee0 2024-04-28T13:35:49.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.723+0000 7f7af7e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7af80d1760 0x7f7af806df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:49.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.723+0000 7f7af7400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7af80b5ee0 0x7f7af80b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:49.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.723+0000 7f7af7e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7af80d1760 0x7f7af806df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:33678/0 (socket says 172.21.15.196:33678) 2024-04-28T13:35:49.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.723+0000 7f7af7e00640 1 -- 172.21.15.196:0/1818741762 learned_addr learned my addr 172.21.15.196:0/1818741762 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:49.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.723+0000 7f7af7400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7af80b5ee0 0x7f7af80b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:60124/0 (socket says 172.21.15.196:60124) 2024-04-28T13:35:49.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.724+0000 7f7af7400640 1 -- 172.21.15.196:0/1818741762 >> v2:172.21.15.203:3300/0 conn(0x7f7af80d1760 msgr2=0x7f7af806df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:49.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.724+0000 7f7af7400640 1 --2- 172.21.15.196:0/1818741762 >> v2:172.21.15.203:3300/0 conn(0x7f7af80d1760 0x7f7af806df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.724+0000 7f7af7400640 1 -- 172.21.15.196:0/1818741762 >> v1:172.21.15.203:6789/0 conn(0x7f7af809ec80 legacy=0x7f7af80d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:49.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.724+0000 7f7af7400640 1 -- 172.21.15.196:0/1818741762 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af80b41f0 con 0x7f7af80b5ee0 2024-04-28T13:35:49.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.725+0000 7f7af7400640 1 --2- 172.21.15.196:0/1818741762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7af80b5ee0 0x7f7af80b4f40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80097d0 tx=0x7f7ae802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=128bf0017844799c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:49.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.726+0000 7f7af6a00640 1 -- 172.21.15.196:0/1818741762 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8037070 con 0x7f7af80b5ee0 2024-04-28T13:35:49.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.726+0000 7f7af6a00640 1 -- 172.21.15.196:0/1818741762 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae8032440 con 0x7f7af80b5ee0 2024-04-28T13:35:49.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.726+0000 7f7af6a00640 1 -- 172.21.15.196:0/1818741762 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8032810 con 0x7f7af80b5ee0 2024-04-28T13:35:49.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.727+0000 7f7afce00640 1 -- 172.21.15.196:0/1818741762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7af80b5ee0 msgr2=0x7f7af80b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:49.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.727+0000 7f7afce00640 1 --2- 172.21.15.196:0/1818741762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7af80b5ee0 0x7f7af80b4f40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80097d0 tx=0x7f7ae802a1c0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.727+0000 7f7afce00640 1 -- 172.21.15.196:0/1818741762 shutdown_connections 2024-04-28T13:35:49.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.727+0000 7f7afce00640 1 --2- 172.21.15.196:0/1818741762 >> v2:172.21.15.203:3300/0 conn(0x7f7af80d1760 0x7f7af806df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.727+0000 7f7afce00640 1 --2- 172.21.15.196:0/1818741762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7af80b5ee0 0x7f7af80b4f40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.727+0000 7f7afce00640 1 -- 172.21.15.196:0/1818741762 >> 172.21.15.196:0/1818741762 conn(0x7f7af806c6c0 msgr2=0x7f7af806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:49.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.727+0000 7f7afce00640 1 -- 172.21.15.196:0/1818741762 shutdown_connections 2024-04-28T13:35:49.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.727+0000 7f7afce00640 1 -- 172.21.15.196:0/1818741762 wait complete. 2024-04-28T13:35:49.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.729+0000 7f7afce00640 1 Processor -- start 2024-04-28T13:35:49.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.729+0000 7f7afce00640 1 -- start start 2024-04-28T13:35:49.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.731+0000 7f7afce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7af809ec80 0x7f7af814f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:49.731 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.731+0000 7f7afce00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7af80b5ee0 0x7f7af80cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:49.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.731+0000 7f7afce00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af80d1760 0x7f7af80cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:49.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.731+0000 7f7afce00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7af80b3c20 con 0x7f7af80d1760 2024-04-28T13:35:49.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.731+0000 7f7afce00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7af80b47b0 con 0x7f7af809ec80 2024-04-28T13:35:49.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.731+0000 7f7afce00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7af80add70 con 0x7f7af80b5ee0 2024-04-28T13:35:49.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.731+0000 7f7af7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af80d1760 0x7f7af80cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:49.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.731+0000 7f7af7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af80d1760 0x7f7af80cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58844/0 (socket says 172.21.15.196:58844) 2024-04-28T13:35:49.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.731+0000 7f7af7e00640 1 -- 172.21.15.196:0/4102289798 learned_addr learned my addr 172.21.15.196:0/4102289798 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:49.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.731+0000 7f7af7400640 1 --2- 172.21.15.196:0/4102289798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7af809ec80 0x7f7af814f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:49.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.731+0000 7f7aefe00640 1 --2- 172.21.15.196:0/4102289798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7af80b5ee0 0x7f7af80cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:49.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.732+0000 7f7af7e00640 1 -- 172.21.15.196:0/4102289798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7af80b5ee0 msgr2=0x7f7af80cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:49.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.732+0000 7f7af7e00640 1 --2- 172.21.15.196:0/4102289798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7af80b5ee0 0x7f7af80cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.732+0000 7f7af7e00640 1 -- 172.21.15.196:0/4102289798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7af809ec80 msgr2=0x7f7af814f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:49.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.732+0000 7f7af7e00640 1 --2- 172.21.15.196:0/4102289798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7af809ec80 0x7f7af814f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.732+0000 7f7af7e00640 1 -- 172.21.15.196:0/4102289798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af80afa00 con 0x7f7af80d1760 2024-04-28T13:35:49.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.732+0000 7f7af7e00640 1 --2- 172.21.15.196:0/4102289798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af80d1760 0x7f7af80cc4f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7af0009cb0 tx=0x7f7af000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:49.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.732+0000 7f7aef400640 1 -- 172.21.15.196:0/4102289798 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af000cd20 con 0x7f7af80d1760 2024-04-28T13:35:49.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.732+0000 7f7aef400640 1 -- 172.21.15.196:0/4102289798 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7af0003990 con 0x7f7af80d1760 2024-04-28T13:35:49.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.732+0000 7f7afce00640 1 -- 172.21.15.196:0/4102289798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af80afce0 con 0x7f7af80d1760 2024-04-28T13:35:49.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.732+0000 7f7aef400640 1 -- 172.21.15.196:0/4102289798 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af0003d80 con 0x7f7af80d1760 2024-04-28T13:35:49.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.732+0000 7f7afce00640 1 -- 172.21.15.196:0/4102289798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af80b02a0 con 0x7f7af80d1760 2024-04-28T13:35:49.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.734+0000 7f7afce00640 1 -- 172.21.15.196:0/4102289798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af806a290 con 0x7f7af80d1760 2024-04-28T13:35:49.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.735+0000 7f7aef400640 1 -- 172.21.15.196:0/4102289798 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7af0012050 con 0x7f7af80d1760 2024-04-28T13:35:49.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.735+0000 7f7aef400640 1 --2- 172.21.15.196:0/4102289798 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7ad406c980 0x7f7ad406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:49.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.736+0000 7f7aef400640 1 -- 172.21.15.196:0/4102289798 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f7af008c220 con 0x7f7af80d1760 2024-04-28T13:35:49.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.736+0000 7f7af7400640 1 --2- 172.21.15.196:0/4102289798 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7ad406c980 0x7f7ad406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:49.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.737+0000 7f7af7400640 1 --2- 172.21.15.196:0/4102289798 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7ad406c980 0x7f7ad406ee40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8034000 tx=0x7f7ae802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:49.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.740+0000 7f7aef400640 1 -- 172.21.15.196:0/4102289798 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7af0010040 con 0x7f7af80d1760 2024-04-28T13:35:49.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.954+0000 7f7afce00640 1 -- 172.21.15.196:0/4102289798 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7af806ee60 con 0x7f7af80d1760 2024-04-28T13:35:49.954 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.955+0000 7f7aef400640 1 -- 172.21.15.196:0/4102289798 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) ==== 80+0+0 (secure 0 0 0) 0x7f7af005a380 con 0x7f7af80d1760 2024-04-28T13:35:49.955 INFO:teuthology.orchestra.run.smithi196.stderr:listed 0 entries 2024-04-28T13:35:49.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.958+0000 7f7afce00640 1 -- 172.21.15.196:0/4102289798 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7ad406c980 msgr2=0x7f7ad406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:49.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.958+0000 7f7afce00640 1 --2- 172.21.15.196:0/4102289798 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7ad406c980 0x7f7ad406ee40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8034000 tx=0x7f7ae802e000 comp rx=0 tx=0).stop 2024-04-28T13:35:49.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.958+0000 7f7afce00640 1 -- 172.21.15.196:0/4102289798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af80d1760 msgr2=0x7f7af80cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:49.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.958+0000 7f7afce00640 1 --2- 172.21.15.196:0/4102289798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af80d1760 0x7f7af80cc4f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7af0009cb0 tx=0x7f7af000c490 comp rx=0 tx=0).stop 2024-04-28T13:35:49.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.958+0000 7f7afce00640 1 -- 172.21.15.196:0/4102289798 shutdown_connections 2024-04-28T13:35:49.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.958+0000 7f7afce00640 1 --2- 172.21.15.196:0/4102289798 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f7ad406c980 0x7f7ad406ee40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.958+0000 7f7afce00640 1 --2- 172.21.15.196:0/4102289798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af80d1760 0x7f7af80cc4f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.958+0000 7f7afce00640 1 --2- 172.21.15.196:0/4102289798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7af80b5ee0 0x7f7af80cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.958+0000 7f7afce00640 1 --2- 172.21.15.196:0/4102289798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7af809ec80 0x7f7af814f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:49.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.958+0000 7f7afce00640 1 -- 172.21.15.196:0/4102289798 >> 172.21.15.196:0/4102289798 conn(0x7f7af806c6c0 msgr2=0x7f7af809bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:49.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.958+0000 7f7afce00640 1 -- 172.21.15.196:0/4102289798 shutdown_connections 2024-04-28T13:35:49.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:49.958+0000 7f7afce00640 1 -- 172.21.15.196:0/4102289798 wait complete. 2024-04-28T13:35:49.977 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-28T13:35:49.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:49.977 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-28T13:35:50.077 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:35:50.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:50.078 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-28T13:35:50.126 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-28T13:35:50.127 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:35:50.174 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052713,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-28T13:35:50.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:50.174 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-28T13:35:50.232 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-28T13:35:50.232 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-28T13:35:50.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:50.233 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T13:35:50.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:50.318 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T13:35:50.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:50.387 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link delete ceph-brx 2024-04-28T13:35:50.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:50.521 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-28T13:35:50.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:50.560 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link delete ceph-brx 2024-04-28T13:35:50.688 INFO:teuthology.orchestra.run.smithi196.stderr:Cannot find device "ceph-brx" 2024-04-28T13:35:50.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:35:50.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:50.693 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-28T13:35:50.749 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-28T13:35:50.749 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-28T13:35:50.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:50.750 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T13:35:50.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:50.835 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-28T13:35:50.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:50.907 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2024-04-28T13:35:51.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:51.017 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-28T13:35:51.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:35:51.062 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2024-04-28T13:35:51.184 INFO:teuthology.orchestra.run.smithi203.stderr:Cannot find device "ceph-brx" 2024-04-28T13:35:51.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:35:51.187 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-28T13:35:51.200 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-28T13:35:51.200 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T13:35:51.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.411+0000 7f09c6200640 1 Processor -- start 2024-04-28T13:35:51.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.411+0000 7f09c6200640 1 -- start start 2024-04-28T13:35:51.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.413+0000 7f09c6200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f09c0073fc0 0x7f09c0072020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.413+0000 7f09c6200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09c0072560 con 0x7f09c0162e50 2024-04-28T13:35:51.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.413+0000 7f09c6200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09c0163910 con 0x7f09c00d12a0 2024-04-28T13:35:51.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.413+0000 7f09c6200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f09c0163bf0 con 0x7f09c0073fc0 2024-04-28T13:35:51.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.413+0000 7f09bea00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f09c0162e50 0x7f09c00d0b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:55674/0 (socket says 172.21.15.196:55674) 2024-04-28T13:35:51.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.413+0000 7f09bea00640 1 -- 172.21.15.196:0/1738898557 learned_addr learned my addr 172.21.15.196:0/1738898557 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:51.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.413+0000 7f09bf400640 1 --2- 172.21.15.196:0/1738898557 >> v2:172.21.15.203:3300/0 conn(0x7f09c0073fc0 0x7f09c0072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:51.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.413+0000 7f09be000640 1 -- 172.21.15.196:0/1738898557 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396822614 0 0) 0x7f09c0072560 con 0x7f09c0162e50 2024-04-28T13:35:51.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.414+0000 7f09be000640 1 -- 172.21.15.196:0/1738898557 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09a4003520 con 0x7f09c0162e50 2024-04-28T13:35:51.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.414+0000 7f09be000640 1 -- 172.21.15.196:0/1738898557 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 606192194 0 0) 0x7f09a4003520 con 0x7f09c0162e50 2024-04-28T13:35:51.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.414+0000 7f09be000640 1 -- 172.21.15.196:0/1738898557 >> v2:172.21.15.203:3300/0 conn(0x7f09c0073fc0 msgr2=0x7f09c0072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:51.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.414+0000 7f09be000640 1 --2- 172.21.15.196:0/1738898557 >> v2:172.21.15.203:3300/0 conn(0x7f09c0073fc0 0x7f09c0072020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:51.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.414+0000 7f09be000640 1 -- 172.21.15.196:0/1738898557 >> v1:172.21.15.203:6789/0 conn(0x7f09c00d12a0 legacy=0x7f09c006f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:51.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.414+0000 7f09be000640 1 -- 172.21.15.196:0/1738898557 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c006ea50 con 0x7f09c0162e50 2024-04-28T13:35:51.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.414+0000 7f09be000640 1 -- 172.21.15.196:0/1738898557 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f09ac001670 con 0x7f09c0162e50 2024-04-28T13:35:51.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.414+0000 7f09be000640 1 -- 172.21.15.196:0/1738898557 >> v1:172.21.15.196:6789/0 conn(0x7f09c0162e50 legacy=0x7f09c00d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:51.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.414+0000 7f09be000640 1 --2- 172.21.15.196:0/1738898557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09a40039b0 0x7f09a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.414+0000 7f09be000640 1 --2- 172.21.15.196:0/1738898557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a4004800 0x7f09a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.414+0000 7f09be000640 1 --2- 172.21.15.196:0/1738898557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a40051c0 0x7f09a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.414+0000 7f09be000640 1 -- 172.21.15.196:0/1738898557 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0163bf0 con 0x7f09a40039b0 2024-04-28T13:35:51.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.414+0000 7f09be000640 1 -- 172.21.15.196:0/1738898557 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f09a400c0f0 con 0x7f09a4004800 2024-04-28T13:35:51.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.414+0000 7f09be000640 1 -- 172.21.15.196:0/1738898557 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f09a400c2f0 con 0x7f09a40051c0 2024-04-28T13:35:51.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.414+0000 7f09bf400640 1 --2- 172.21.15.196:0/1738898557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09a40039b0 0x7f09a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:51.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.415+0000 7f09bea00640 1 --2- 172.21.15.196:0/1738898557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a4004800 0x7f09a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:51.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.415+0000 7f09bfe00640 1 --2- 172.21.15.196:0/1738898557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a40051c0 0x7f09a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:51.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.415+0000 7f09bf400640 1 -- 172.21.15.196:0/1738898557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a40051c0 msgr2=0x7f09a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:51.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.415+0000 7f09bf400640 1 --2- 172.21.15.196:0/1738898557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a40051c0 0x7f09a400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:51.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.415+0000 7f09bf400640 1 -- 172.21.15.196:0/1738898557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a4004800 msgr2=0x7f09a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:51.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.415+0000 7f09bf400640 1 --2- 172.21.15.196:0/1738898557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a4004800 0x7f09a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:51.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.415+0000 7f09bf400640 1 -- 172.21.15.196:0/1738898557 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a4003520 con 0x7f09a40039b0 2024-04-28T13:35:51.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.415+0000 7f09bfe00640 1 --2- 172.21.15.196:0/1738898557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a40051c0 0x7f09a400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:35:51.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.415+0000 7f09bfe00640 1 -- 172.21.15.196:0/1738898557 reap_dead start 2024-04-28T13:35:51.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.416+0000 7f09bf400640 1 --2- 172.21.15.196:0/1738898557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09a40039b0 0x7f09a4003dd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f09b000ba30 tx=0x7f09a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3521025ebfc20642 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:51.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.416+0000 7f09be000640 1 -- 172.21.15.196:0/1738898557 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0030070 con 0x7f09a40039b0 2024-04-28T13:35:51.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.416+0000 7f09be000640 1 -- 172.21.15.196:0/1738898557 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09b002ade0 con 0x7f09a40039b0 2024-04-28T13:35:51.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.416+0000 7f09be000640 1 -- 172.21.15.196:0/1738898557 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b000b600 con 0x7f09a40039b0 2024-04-28T13:35:51.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.417+0000 7f09c6200640 1 -- 172.21.15.196:0/1738898557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09a40039b0 msgr2=0x7f09a4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:51.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.417+0000 7f09c6200640 1 --2- 172.21.15.196:0/1738898557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09a40039b0 0x7f09a4003dd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f09b000ba30 tx=0x7f09a4005830 comp rx=0 tx=0).stop 2024-04-28T13:35:51.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.417+0000 7f09c6200640 1 -- 172.21.15.196:0/1738898557 shutdown_connections 2024-04-28T13:35:51.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.417+0000 7f09c6200640 1 --2- 172.21.15.196:0/1738898557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09a40039b0 0x7f09a4003dd0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:51.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.417+0000 7f09c6200640 1 -- 172.21.15.196:0/1738898557 >> 172.21.15.196:0/1738898557 conn(0x7f09c00cd830 msgr2=0x7f09c00cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:51.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.417+0000 7f09c6200640 1 -- 172.21.15.196:0/1738898557 shutdown_connections 2024-04-28T13:35:51.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.417+0000 7f09c6200640 1 -- 172.21.15.196:0/1738898557 wait complete. 2024-04-28T13:35:51.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.419+0000 7f09c6200640 1 Processor -- start 2024-04-28T13:35:51.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.419+0000 7f09c6200640 1 -- start start 2024-04-28T13:35:51.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.420+0000 7f09c6200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a40039b0 0x7f09c00cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.420+0000 7f09c6200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09c00c5d50 0x7f09c00b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.420+0000 7f09c6200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09c0146430 0x7f09c01468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.420+0000 7f09c6200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09c00743a0 con 0x7f09c00c5d50 2024-04-28T13:35:51.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.420+0000 7f09c6200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0148de0 con 0x7f09c0146430 2024-04-28T13:35:51.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.420+0000 7f09c6200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f09c0146e20 con 0x7f09a40039b0 2024-04-28T13:35:51.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.420+0000 7f09bea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09c00c5d50 0x7f09c00b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:51.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.420+0000 7f09bea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09c00c5d50 0x7f09c00b5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58860/0 (socket says 172.21.15.196:58860) 2024-04-28T13:35:51.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.420+0000 7f09bea00640 1 -- 172.21.15.196:0/1869191532 learned_addr learned my addr 172.21.15.196:0/1869191532 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:51.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.421+0000 7f09bf400640 1 --2- 172.21.15.196:0/1869191532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a40039b0 0x7f09c00cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:51.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.421+0000 7f09bea00640 1 -- 172.21.15.196:0/1869191532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a40039b0 msgr2=0x7f09c00cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:51.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.421+0000 7f09bea00640 1 --2- 172.21.15.196:0/1869191532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a40039b0 0x7f09c00cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:51.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.421+0000 7f09bea00640 1 -- 172.21.15.196:0/1869191532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09c0146430 msgr2=0x7f09c01468e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:51.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.421+0000 7f09bea00640 1 --2- 172.21.15.196:0/1869191532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09c0146430 0x7f09c01468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:51.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.421+0000 7f09bea00640 1 -- 172.21.15.196:0/1869191532 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c00c6210 con 0x7f09c00c5d50 2024-04-28T13:35:51.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.421+0000 7f09bea00640 1 --2- 172.21.15.196:0/1869191532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09c00c5d50 0x7f09c00b5410 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00e840 tx=0x7f09ac00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:51.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.421+0000 7f09b7400640 1 -- 172.21.15.196:0/1869191532 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac00c8d0 con 0x7f09c00c5d50 2024-04-28T13:35:51.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.421+0000 7f09b7400640 1 -- 172.21.15.196:0/1869191532 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09ac018070 con 0x7f09c00c5d50 2024-04-28T13:35:51.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.421+0000 7f09b7400640 1 -- 172.21.15.196:0/1869191532 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac00ccb0 con 0x7f09c00c5d50 2024-04-28T13:35:51.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.421+0000 7f09c6200640 1 -- 172.21.15.196:0/1869191532 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c00c68f0 con 0x7f09c00c5d50 2024-04-28T13:35:51.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.421+0000 7f09c6200640 1 -- 172.21.15.196:0/1869191532 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c0147760 con 0x7f09c00c5d50 2024-04-28T13:35:51.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.423+0000 7f09c6200640 1 -- 172.21.15.196:0/1869191532 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c00c66e0 con 0x7f09c00c5d50 2024-04-28T13:35:51.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.423+0000 7f09b7400640 1 -- 172.21.15.196:0/1869191532 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f09ac01d460 con 0x7f09c00c5d50 2024-04-28T13:35:51.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.424+0000 7f09b7400640 1 --2- 172.21.15.196:0/1869191532 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f09a006c980 0x7f09a006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.424+0000 7f09b7400640 1 -- 172.21.15.196:0/1869191532 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f09ac08bbd0 con 0x7f09c00c5d50 2024-04-28T13:35:51.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.425+0000 7f09bf400640 1 --2- 172.21.15.196:0/1869191532 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f09a006c980 0x7f09a006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:51.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.426+0000 7f09bf400640 1 --2- 172.21.15.196:0/1869191532 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f09a006c980 0x7f09a006ee40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f09b000ba00 tx=0x7f09b0030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:51.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.429+0000 7f09b7400640 1 -- 172.21.15.196:0/1869191532 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f09ac059ed0 con 0x7f09c00c5d50 2024-04-28T13:35:51.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.626+0000 7f09c6200640 1 -- 172.21.15.196:0/1869191532 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f09c00c6e10 con 0x7f09a006c980 2024-04-28T13:35:51.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.646+0000 7f09b7400640 1 -- 172.21.15.196:0/1869191532 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251882 (secure 0 0 0) 0x7f09b002ac60 con 0x7f09a006c980 2024-04-28T13:35:51.648 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:35:51.650 INFO:teuthology.orchestra.run.smithi196.stderr:dumped all 2024-04-28T13:35:51.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.652+0000 7f09c6200640 1 -- 172.21.15.196:0/1869191532 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f09a006c980 msgr2=0x7f09a006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:51.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.652+0000 7f09c6200640 1 --2- 172.21.15.196:0/1869191532 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f09a006c980 0x7f09a006ee40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f09b000ba00 tx=0x7f09b0030000 comp rx=0 tx=0).stop 2024-04-28T13:35:51.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.652+0000 7f09c6200640 1 -- 172.21.15.196:0/1869191532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09c00c5d50 msgr2=0x7f09c00b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:51.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.652+0000 7f09c6200640 1 --2- 172.21.15.196:0/1869191532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09c00c5d50 0x7f09c00b5410 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00e840 tx=0x7f09ac00ed00 comp rx=0 tx=0).stop 2024-04-28T13:35:51.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.652+0000 7f09c6200640 1 -- 172.21.15.196:0/1869191532 shutdown_connections 2024-04-28T13:35:51.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.652+0000 7f09c6200640 1 --2- 172.21.15.196:0/1869191532 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f09a006c980 0x7f09a006ee40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:51.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.652+0000 7f09c6200640 1 --2- 172.21.15.196:0/1869191532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09c0146430 0x7f09c01468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:51.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.652+0000 7f09c6200640 1 --2- 172.21.15.196:0/1869191532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09c00c5d50 0x7f09c00b5410 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:51.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.652+0000 7f09c6200640 1 --2- 172.21.15.196:0/1869191532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a40039b0 0x7f09c00cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:51.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.652+0000 7f09c6200640 1 -- 172.21.15.196:0/1869191532 >> 172.21.15.196:0/1869191532 conn(0x7f09c00cd830 msgr2=0x7f09c00b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:51.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.652+0000 7f09c6200640 1 -- 172.21.15.196:0/1869191532 shutdown_connections 2024-04-28T13:35:51.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.652+0000 7f09c6200640 1 -- 172.21.15.196:0/1869191532 wait complete. 2024-04-28T13:35:51.670 INFO:teuthology.orchestra.run.smithi196.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-04-28T13:35:50.282761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594563,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":212,"num_write_kb":2437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225876,"kb_used_data":5544,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748844,"statfs":{"total":773094113280,"available":772862816256,"internally_reserved":0,"allocated":5677056,"data_stored":3015396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":282,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004237"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350675+0000","last_change":"2024-04-28T13:33:33.999818+0000","last_active":"2024-04-28T13:34:04.350675+0000","last_peered":"2024-04-28T13:34:04.350675+0000","last_clean":"2024-04-28T13:34:04.350675+0000","last_became_active":"2024-04-28T13:33:33.999486+0000","last_became_peered":"2024-04-28T13:33:33.999486+0000","last_unstale":"2024-04-28T13:34:04.350675+0000","last_undegraded":"2024-04-28T13:34:04.350675+0000","last_fullsized":"2024-04-28T13:34:04.350675+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:32:47.409486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355721+0000","last_change":"2024-04-28T13:33:34.003919+0000","last_active":"2024-04-28T13:34:04.355721+0000","last_peered":"2024-04-28T13:34:04.355721+0000","last_clean":"2024-04-28T13:34:04.355721+0000","last_became_active":"2024-04-28T13:33:34.003551+0000","last_became_peered":"2024-04-28T13:33:34.003551+0000","last_unstale":"2024-04-28T13:34:04.355721+0000","last_undegraded":"2024-04-28T13:34:04.355721+0000","last_fullsized":"2024-04-28T13:34:04.355721+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347343+0000","last_change":"2024-04-28T13:33:33.996733+0000","last_active":"2024-04-28T13:34:04.347343+0000","last_peered":"2024-04-28T13:34:04.347343+0000","last_clean":"2024-04-28T13:34:04.347343+0000","last_became_active":"2024-04-28T13:33:33.996413+0000","last_became_peered":"2024-04-28T13:33:33.996413+0000","last_unstale":"2024-04-28T13:34:04.347343+0000","last_undegraded":"2024-04-28T13:34:04.347343+0000","last_fullsized":"2024-04-28T13:34:04.347343+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:46.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879775+0000","last_change":"2024-04-28T13:33:34.004787+0000","last_active":"2024-04-28T13:34:04.879775+0000","last_peered":"2024-04-28T13:34:04.879775+0000","last_clean":"2024-04-28T13:34:04.879775+0000","last_became_active":"2024-04-28T13:33:33.999818+0000","last_became_peered":"2024-04-28T13:33:33.999818+0000","last_unstale":"2024-04-28T13:34:04.879775+0000","last_undegraded":"2024-04-28T13:34:04.879775+0000","last_fullsized":"2024-04-28T13:34:04.879775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354517+0000","last_change":"2024-04-28T13:33:34.004715+0000","last_active":"2024-04-28T13:34:04.354517+0000","last_peered":"2024-04-28T13:34:04.354517+0000","last_clean":"2024-04-28T13:34:04.354517+0000","last_became_active":"2024-04-28T13:33:34.004436+0000","last_became_peered":"2024-04-28T13:33:34.004436+0000","last_unstale":"2024-04-28T13:34:04.354517+0000","last_undegraded":"2024-04-28T13:34:04.354517+0000","last_fullsized":"2024-04-28T13:34:04.354517+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:46.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344494+0000","last_change":"2024-04-28T13:33:34.012959+0000","last_active":"2024-04-28T13:34:04.344494+0000","last_peered":"2024-04-28T13:34:04.344494+0000","last_clean":"2024-04-28T13:34:04.344494+0000","last_became_active":"2024-04-28T13:33:34.010839+0000","last_became_peered":"2024-04-28T13:33:34.010839+0000","last_unstale":"2024-04-28T13:34:04.344494+0000","last_undegraded":"2024-04-28T13:34:04.344494+0000","last_fullsized":"2024-04-28T13:34:04.344494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:48:58.152999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349347+0000","last_change":"2024-04-28T13:33:34.003448+0000","last_active":"2024-04-28T13:34:04.349347+0000","last_peered":"2024-04-28T13:34:04.349347+0000","last_clean":"2024-04-28T13:34:04.349347+0000","last_became_active":"2024-04-28T13:33:34.002948+0000","last_became_peered":"2024-04-28T13:33:34.002948+0000","last_unstale":"2024-04-28T13:34:04.349347+0000","last_undegraded":"2024-04-28T13:34:04.349347+0000","last_fullsized":"2024-04-28T13:34:04.349347+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:09:39.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.145422+0000","last_change":"2024-04-28T13:34:05.143133+0000","last_active":"2024-04-28T13:34:05.145422+0000","last_peered":"2024-04-28T13:34:05.145422+0000","last_clean":"2024-04-28T13:34:05.145422+0000","last_became_active":"2024-04-28T13:34:05.142351+0000","last_became_peered":"2024-04-28T13:34:05.142351+0000","last_unstale":"2024-04-28T13:34:05.145422+0000","last_undegraded":"2024-04-28T13:34:05.145422+0000","last_fullsized":"2024-04-28T13:34:05.145422+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:53:12.865627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.881028+0000","last_change":"2024-04-28T13:33:34.013366+0000","last_active":"2024-04-28T13:34:04.881028+0000","last_peered":"2024-04-28T13:34:04.881028+0000","last_clean":"2024-04-28T13:34:04.881028+0000","last_became_active":"2024-04-28T13:33:34.012900+0000","last_became_peered":"2024-04-28T13:33:34.012900+0000","last_unstale":"2024-04-28T13:34:04.881028+0000","last_undegraded":"2024-04-28T13:34:04.881028+0000","last_fullsized":"2024-04-28T13:34:04.881028+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:54.852573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880311+0000","last_change":"2024-04-28T13:33:34.007098+0000","last_active":"2024-04-28T13:34:04.880311+0000","last_peered":"2024-04-28T13:34:04.880311+0000","last_clean":"2024-04-28T13:34:04.880311+0000","last_became_active":"2024-04-28T13:33:34.005157+0000","last_became_peered":"2024-04-28T13:33:34.005157+0000","last_unstale":"2024-04-28T13:34:04.880311+0000","last_undegraded":"2024-04-28T13:34:04.880311+0000","last_fullsized":"2024-04-28T13:34:04.880311+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:09:39.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879686+0000","last_change":"2024-04-28T13:33:34.011118+0000","last_active":"2024-04-28T13:34:04.879686+0000","last_peered":"2024-04-28T13:34:04.879686+0000","last_clean":"2024-04-28T13:34:04.879686+0000","last_became_active":"2024-04-28T13:33:34.010811+0000","last_became_peered":"2024-04-28T13:33:34.010811+0000","last_unstale":"2024-04-28T13:34:04.879686+0000","last_undegraded":"2024-04-28T13:34:04.879686+0000","last_fullsized":"2024-04-28T13:34:04.879686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:01:13.143511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355402+0000","last_change":"2024-04-28T13:33:34.011900+0000","last_active":"2024-04-28T13:34:04.355402+0000","last_peered":"2024-04-28T13:34:04.355402+0000","last_clean":"2024-04-28T13:34:04.355402+0000","last_became_active":"2024-04-28T13:33:34.011580+0000","last_became_peered":"2024-04-28T13:33:34.011580+0000","last_unstale":"2024-04-28T13:34:04.355402+0000","last_undegraded":"2024-04-28T13:34:04.355402+0000","last_fullsized":"2024-04-28T13:34:04.355402+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.532062+0000","last_change":"2024-04-28T13:33:34.000874+0000","last_active":"2024-04-28T13:34:05.532062+0000","last_peered":"2024-04-28T13:34:05.532062+0000","last_clean":"2024-04-28T13:34:05.532062+0000","last_became_active":"2024-04-28T13:33:34.000568+0000","last_became_peered":"2024-04-28T13:33:34.000568+0000","last_unstale":"2024-04-28T13:34:05.532062+0000","last_undegraded":"2024-04-28T13:34:05.532062+0000","last_fullsized":"2024-04-28T13:34:05.532062+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:56:40.722825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.502702+0000","last_change":"2024-04-28T13:34:04.496689+0000","last_active":"2024-04-28T13:34:04.502702+0000","last_peered":"2024-04-28T13:34:04.502702+0000","last_clean":"2024-04-28T13:34:04.502702+0000","last_became_active":"2024-04-28T13:34:04.495748+0000","last_became_peered":"2024-04-28T13:34:04.495748+0000","last_unstale":"2024-04-28T13:34:04.502702+0000","last_undegraded":"2024-04-28T13:34:04.502702+0000","last_fullsized":"2024-04-28T13:34:04.502702+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:11.732507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344170+0000","last_change":"2024-04-28T13:33:33.998387+0000","last_active":"2024-04-28T13:34:04.344170+0000","last_peered":"2024-04-28T13:34:04.344170+0000","last_clean":"2024-04-28T13:34:04.344170+0000","last_became_active":"2024-04-28T13:33:33.998087+0000","last_became_peered":"2024-04-28T13:33:33.998087+0000","last_unstale":"2024-04-28T13:34:04.344170+0000","last_undegraded":"2024-04-28T13:34:04.344170+0000","last_fullsized":"2024-04-28T13:34:04.344170+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:52:24.477401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139537+0000","last_change":"2024-04-28T13:33:33.989154+0000","last_active":"2024-04-28T13:34:05.139537+0000","last_peered":"2024-04-28T13:34:05.139537+0000","last_clean":"2024-04-28T13:34:05.139537+0000","last_became_active":"2024-04-28T13:33:33.988827+0000","last_became_peered":"2024-04-28T13:33:33.988827+0000","last_unstale":"2024-04-28T13:34:05.139537+0000","last_undegraded":"2024-04-28T13:34:05.139537+0000","last_fullsized":"2024-04-28T13:34:05.139537+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:52:46.527475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346773+0000","last_change":"2024-04-28T13:33:34.008467+0000","last_active":"2024-04-28T13:34:04.346773+0000","last_peered":"2024-04-28T13:34:04.346773+0000","last_clean":"2024-04-28T13:34:04.346773+0000","last_became_active":"2024-04-28T13:33:34.008181+0000","last_became_peered":"2024-04-28T13:33:34.008181+0000","last_unstale":"2024-04-28T13:34:04.346773+0000","last_undegraded":"2024-04-28T13:34:04.346773+0000","last_fullsized":"2024-04-28T13:34:04.346773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347607+0000","last_change":"2024-04-28T13:33:34.007505+0000","last_active":"2024-04-28T13:34:04.347607+0000","last_peered":"2024-04-28T13:34:04.347607+0000","last_clean":"2024-04-28T13:34:04.347607+0000","last_became_active":"2024-04-28T13:33:34.006793+0000","last_became_peered":"2024-04-28T13:33:34.006793+0000","last_unstale":"2024-04-28T13:34:04.347607+0000","last_undegraded":"2024-04-28T13:34:04.347607+0000","last_fullsized":"2024-04-28T13:34:04.347607+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.140956+0000","last_change":"2024-04-28T13:33:34.009974+0000","last_active":"2024-04-28T13:34:05.140956+0000","last_peered":"2024-04-28T13:34:05.140956+0000","last_clean":"2024-04-28T13:34:05.140956+0000","last_became_active":"2024-04-28T13:33:34.009657+0000","last_became_peered":"2024-04-28T13:33:34.009657+0000","last_unstale":"2024-04-28T13:34:05.140956+0000","last_undegraded":"2024-04-28T13:34:05.140956+0000","last_fullsized":"2024-04-28T13:34:05.140956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:18:56.809253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531828+0000","last_change":"2024-04-28T13:33:34.004233+0000","last_active":"2024-04-28T13:34:05.531828+0000","last_peered":"2024-04-28T13:34:05.531828+0000","last_clean":"2024-04-28T13:34:05.531828+0000","last_became_active":"2024-04-28T13:33:33.999567+0000","last_became_peered":"2024-04-28T13:33:33.999567+0000","last_unstale":"2024-04-28T13:34:05.531828+0000","last_undegraded":"2024-04-28T13:34:05.531828+0000","last_fullsized":"2024-04-28T13:34:05.531828+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:36.248393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344842+0000","last_change":"2024-04-28T13:33:34.008932+0000","last_active":"2024-04-28T13:34:04.344842+0000","last_peered":"2024-04-28T13:34:04.344842+0000","last_clean":"2024-04-28T13:34:04.344842+0000","last_became_active":"2024-04-28T13:33:34.008652+0000","last_became_peered":"2024-04-28T13:33:34.008652+0000","last_unstale":"2024-04-28T13:34:04.344842+0000","last_undegraded":"2024-04-28T13:34:04.344842+0000","last_fullsized":"2024-04-28T13:34:04.344842+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:43:22.000390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880044+0000","last_change":"2024-04-28T13:33:34.006521+0000","last_active":"2024-04-28T13:34:04.880044+0000","last_peered":"2024-04-28T13:34:04.880044+0000","last_clean":"2024-04-28T13:34:04.880044+0000","last_became_active":"2024-04-28T13:33:34.006227+0000","last_became_peered":"2024-04-28T13:33:34.006227+0000","last_unstale":"2024-04-28T13:34:04.880044+0000","last_undegraded":"2024-04-28T13:34:04.880044+0000","last_fullsized":"2024-04-28T13:34:04.880044+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:46.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139737+0000","last_change":"2024-04-28T13:33:33.999169+0000","last_active":"2024-04-28T13:34:05.139737+0000","last_peered":"2024-04-28T13:34:05.139737+0000","last_clean":"2024-04-28T13:34:05.139737+0000","last_became_active":"2024-04-28T13:33:33.998179+0000","last_became_peered":"2024-04-28T13:33:33.998179+0000","last_unstale":"2024-04-28T13:34:05.139737+0000","last_undegraded":"2024-04-28T13:34:05.139737+0000","last_fullsized":"2024-04-28T13:34:05.139737+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:12.546153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139678+0000","last_change":"2024-04-28T13:33:33.998034+0000","last_active":"2024-04-28T13:34:05.139678+0000","last_peered":"2024-04-28T13:34:05.139678+0000","last_clean":"2024-04-28T13:34:05.139678+0000","last_became_active":"2024-04-28T13:33:33.997648+0000","last_became_peered":"2024-04-28T13:33:33.997648+0000","last_unstale":"2024-04-28T13:34:05.139678+0000","last_undegraded":"2024-04-28T13:34:05.139678+0000","last_fullsized":"2024-04-28T13:34:05.139678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:30.570296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.141197+0000","last_change":"2024-04-28T13:33:34.007275+0000","last_active":"2024-04-28T13:34:05.141197+0000","last_peered":"2024-04-28T13:34:05.141197+0000","last_clean":"2024-04-28T13:34:05.141197+0000","last_became_active":"2024-04-28T13:33:34.005017+0000","last_became_peered":"2024-04-28T13:33:34.005017+0000","last_unstale":"2024-04-28T13:34:05.141197+0000","last_undegraded":"2024-04-28T13:34:05.141197+0000","last_fullsized":"2024-04-28T13:34:05.141197+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:37:53.261649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531787+0000","last_change":"2024-04-28T13:33:33.995804+0000","last_active":"2024-04-28T13:34:05.531787+0000","last_peered":"2024-04-28T13:34:05.531787+0000","last_clean":"2024-04-28T13:34:05.531787+0000","last_became_active":"2024-04-28T13:33:33.995502+0000","last_became_peered":"2024-04-28T13:33:33.995502+0000","last_unstale":"2024-04-28T13:34:05.531787+0000","last_undegraded":"2024-04-28T13:34:05.531787+0000","last_fullsized":"2024-04-28T13:34:05.531787+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:27:53.804713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347538+0000","last_change":"2024-04-28T13:33:34.006447+0000","last_active":"2024-04-28T13:34:04.347538+0000","last_peered":"2024-04-28T13:34:04.347538+0000","last_clean":"2024-04-28T13:34:04.347538+0000","last_became_active":"2024-04-28T13:33:34.005656+0000","last_became_peered":"2024-04-28T13:33:34.005656+0000","last_unstale":"2024-04-28T13:34:04.347538+0000","last_undegraded":"2024-04-28T13:34:04.347538+0000","last_fullsized":"2024-04-28T13:34:04.347538+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355564+0000","last_change":"2024-04-28T13:33:34.005608+0000","last_active":"2024-04-28T13:34:04.355564+0000","last_peered":"2024-04-28T13:34:04.355564+0000","last_clean":"2024-04-28T13:34:04.355564+0000","last_became_active":"2024-04-28T13:33:34.005316+0000","last_became_peered":"2024-04-28T13:33:34.005316+0000","last_unstale":"2024-04-28T13:34:04.355564+0000","last_undegraded":"2024-04-28T13:34:04.355564+0000","last_fullsized":"2024-04-28T13:34:04.355564+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139766+0000","last_change":"2024-04-28T13:33:34.002648+0000","last_active":"2024-04-28T13:34:05.139766+0000","last_peered":"2024-04-28T13:34:05.139766+0000","last_clean":"2024-04-28T13:34:05.139766+0000","last_became_active":"2024-04-28T13:33:34.002375+0000","last_became_peered":"2024-04-28T13:33:34.002375+0000","last_unstale":"2024-04-28T13:34:05.139766+0000","last_undegraded":"2024-04-28T13:34:05.139766+0000","last_fullsized":"2024-04-28T13:34:05.139766+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:42.082730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346448+0000","last_change":"2024-04-28T13:33:33.999443+0000","last_active":"2024-04-28T13:34:04.346448+0000","last_peered":"2024-04-28T13:34:04.346448+0000","last_clean":"2024-04-28T13:34:04.346448+0000","last_became_active":"2024-04-28T13:33:33.998905+0000","last_became_peered":"2024-04-28T13:33:33.998905+0000","last_unstale":"2024-04-28T13:34:04.346448+0000","last_undegraded":"2024-04-28T13:34:04.346448+0000","last_fullsized":"2024-04-28T13:34:04.346448+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344364+0000","last_change":"2024-04-28T13:33:34.008881+0000","last_active":"2024-04-28T13:34:04.344364+0000","last_peered":"2024-04-28T13:34:04.344364+0000","last_clean":"2024-04-28T13:34:04.344364+0000","last_became_active":"2024-04-28T13:33:34.008297+0000","last_became_peered":"2024-04-28T13:33:34.008297+0000","last_unstale":"2024-04-28T13:34:04.344364+0000","last_undegraded":"2024-04-28T13:34:04.344364+0000","last_fullsized":"2024-04-28T13:34:04.344364+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:41:26.988112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531605+0000","last_change":"2024-04-28T13:33:34.003176+0000","last_active":"2024-04-28T13:34:05.531605+0000","last_peered":"2024-04-28T13:34:05.531605+0000","last_clean":"2024-04-28T13:34:05.531605+0000","last_became_active":"2024-04-28T13:33:34.002915+0000","last_became_peered":"2024-04-28T13:33:34.002915+0000","last_unstale":"2024-04-28T13:34:05.531605+0000","last_undegraded":"2024-04-28T13:34:05.531605+0000","last_fullsized":"2024-04-28T13:34:05.531605+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:52:06.849498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493215+0000","last_change":"2024-04-28T13:33:34.005669+0000","last_active":"2024-04-28T13:34:04.493215+0000","last_peered":"2024-04-28T13:34:04.493215+0000","last_clean":"2024-04-28T13:34:04.493215+0000","last_became_active":"2024-04-28T13:33:34.005365+0000","last_became_peered":"2024-04-28T13:33:34.005365+0000","last_unstale":"2024-04-28T13:34:04.493215+0000","last_undegraded":"2024-04-28T13:34:04.493215+0000","last_fullsized":"2024-04-28T13:34:04.493215+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:18:56.809253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":23,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345243+0000","last_change":"2024-04-28T13:33:32.998037+0000","last_active":"2024-04-28T13:34:04.345243+0000","last_peered":"2024-04-28T13:34:04.345243+0000","last_clean":"2024-04-28T13:34:04.345243+0000","last_became_active":"2024-04-28T13:33:32.997694+0000","last_became_peered":"2024-04-28T13:33:32.997694+0000","last_unstale":"2024-04-28T13:34:04.345243+0000","last_undegraded":"2024-04-28T13:34:04.345243+0000","last_fullsized":"2024-04-28T13:34:04.345243+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:02.432193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.141337+0000","last_change":"2024-04-28T13:33:33.989048+0000","last_active":"2024-04-28T13:34:05.141337+0000","last_peered":"2024-04-28T13:34:05.141337+0000","last_clean":"2024-04-28T13:34:05.141337+0000","last_became_active":"2024-04-28T13:33:33.988758+0000","last_became_peered":"2024-04-28T13:33:33.988758+0000","last_unstale":"2024-04-28T13:34:05.141337+0000","last_undegraded":"2024-04-28T13:34:05.141337+0000","last_fullsized":"2024-04-28T13:34:05.141337+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:14.152642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347981+0000","last_change":"2024-04-28T13:33:32.999239+0000","last_active":"2024-04-28T13:34:04.347981+0000","last_peered":"2024-04-28T13:34:04.347981+0000","last_clean":"2024-04-28T13:34:04.347981+0000","last_became_active":"2024-04-28T13:33:32.998813+0000","last_became_peered":"2024-04-28T13:33:32.998813+0000","last_unstale":"2024-04-28T13:34:04.347981+0000","last_undegraded":"2024-04-28T13:34:04.347981+0000","last_fullsized":"2024-04-28T13:34:04.347981+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:22:34.444155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355706+0000","last_change":"2024-04-28T13:33:34.003176+0000","last_active":"2024-04-28T13:34:04.355706+0000","last_peered":"2024-04-28T13:34:04.355706+0000","last_clean":"2024-04-28T13:34:04.355706+0000","last_became_active":"2024-04-28T13:33:34.002904+0000","last_became_peered":"2024-04-28T13:33:34.002904+0000","last_unstale":"2024-04-28T13:34:04.355706+0000","last_undegraded":"2024-04-28T13:34:04.355706+0000","last_fullsized":"2024-04-28T13:34:04.355706+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493775+0000","last_change":"2024-04-28T13:33:32.997729+0000","last_active":"2024-04-28T13:34:04.493775+0000","last_peered":"2024-04-28T13:34:04.493775+0000","last_clean":"2024-04-28T13:34:04.493775+0000","last_became_active":"2024-04-28T13:33:32.996881+0000","last_became_peered":"2024-04-28T13:33:32.996881+0000","last_unstale":"2024-04-28T13:34:04.493775+0000","last_undegraded":"2024-04-28T13:34:04.493775+0000","last_fullsized":"2024-04-28T13:34:04.493775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493141+0000","last_change":"2024-04-28T13:33:34.004339+0000","last_active":"2024-04-28T13:34:04.493141+0000","last_peered":"2024-04-28T13:34:04.493141+0000","last_clean":"2024-04-28T13:34:04.493141+0000","last_became_active":"2024-04-28T13:33:34.002361+0000","last_became_peered":"2024-04-28T13:33:34.002361+0000","last_unstale":"2024-04-28T13:34:04.493141+0000","last_undegraded":"2024-04-28T13:34:04.493141+0000","last_fullsized":"2024-04-28T13:34:04.493141+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:31.671210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350207+0000","last_change":"2024-04-28T13:33:33.000754+0000","last_active":"2024-04-28T13:34:04.350207+0000","last_peered":"2024-04-28T13:34:04.350207+0000","last_clean":"2024-04-28T13:34:04.350207+0000","last_became_active":"2024-04-28T13:33:33.000387+0000","last_became_peered":"2024-04-28T13:33:33.000387+0000","last_unstale":"2024-04-28T13:34:04.350207+0000","last_undegraded":"2024-04-28T13:34:04.350207+0000","last_fullsized":"2024-04-28T13:34:04.350207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:28.481114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346329+0000","last_change":"2024-04-28T13:33:33.994316+0000","last_active":"2024-04-28T13:34:04.346329+0000","last_peered":"2024-04-28T13:34:04.346329+0000","last_clean":"2024-04-28T13:34:04.346329+0000","last_became_active":"2024-04-28T13:33:33.994002+0000","last_became_peered":"2024-04-28T13:33:33.994002+0000","last_unstale":"2024-04-28T13:34:04.346329+0000","last_undegraded":"2024-04-28T13:34:04.346329+0000","last_fullsized":"2024-04-28T13:34:04.346329+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:03.446736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344611+0000","last_change":"2024-04-28T13:33:32.999540+0000","last_active":"2024-04-28T13:34:04.344611+0000","last_peered":"2024-04-28T13:34:04.344611+0000","last_clean":"2024-04-28T13:34:04.344611+0000","last_became_active":"2024-04-28T13:33:32.999147+0000","last_became_peered":"2024-04-28T13:33:32.999147+0000","last_unstale":"2024-04-28T13:34:04.344611+0000","last_undegraded":"2024-04-28T13:34:04.344611+0000","last_fullsized":"2024-04-28T13:34:04.344611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:45.657949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880509+0000","last_change":"2024-04-28T13:33:34.000424+0000","last_active":"2024-04-28T13:34:04.880509+0000","last_peered":"2024-04-28T13:34:04.880509+0000","last_clean":"2024-04-28T13:34:04.880509+0000","last_became_active":"2024-04-28T13:33:33.999414+0000","last_became_peered":"2024-04-28T13:33:33.999414+0000","last_unstale":"2024-04-28T13:34:04.880509+0000","last_undegraded":"2024-04-28T13:34:04.880509+0000","last_fullsized":"2024-04-28T13:34:04.880509+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346593+0000","last_change":"2024-04-28T13:33:32.988943+0000","last_active":"2024-04-28T13:34:04.346593+0000","last_peered":"2024-04-28T13:34:04.346593+0000","last_clean":"2024-04-28T13:34:04.346593+0000","last_became_active":"2024-04-28T13:33:32.986596+0000","last_became_peered":"2024-04-28T13:33:32.986596+0000","last_unstale":"2024-04-28T13:34:04.346593+0000","last_undegraded":"2024-04-28T13:34:04.346593+0000","last_fullsized":"2024-04-28T13:34:04.346593+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879866+0000","last_change":"2024-04-28T13:33:34.002579+0000","last_active":"2024-04-28T13:34:04.879866+0000","last_peered":"2024-04-28T13:34:04.879866+0000","last_clean":"2024-04-28T13:34:04.879866+0000","last_became_active":"2024-04-28T13:33:34.002254+0000","last_became_peered":"2024-04-28T13:33:34.002254+0000","last_unstale":"2024-04-28T13:34:04.879866+0000","last_undegraded":"2024-04-28T13:34:04.879866+0000","last_fullsized":"2024-04-28T13:34:04.879866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354254+0000","last_change":"2024-04-28T13:33:32.996052+0000","last_active":"2024-04-28T13:34:04.354254+0000","last_peered":"2024-04-28T13:34:04.354254+0000","last_clean":"2024-04-28T13:34:04.354254+0000","last_became_active":"2024-04-28T13:33:32.990258+0000","last_became_peered":"2024-04-28T13:33:32.990258+0000","last_unstale":"2024-04-28T13:34:04.354254+0000","last_undegraded":"2024-04-28T13:34:04.354254+0000","last_fullsized":"2024-04-28T13:34:04.354254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139482+0000","last_change":"2024-04-28T13:33:33.998180+0000","last_active":"2024-04-28T13:34:05.139482+0000","last_peered":"2024-04-28T13:34:05.139482+0000","last_clean":"2024-04-28T13:34:05.139482+0000","last_became_active":"2024-04-28T13:33:33.997730+0000","last_became_peered":"2024-04-28T13:33:33.997730+0000","last_unstale":"2024-04-28T13:34:05.139482+0000","last_undegraded":"2024-04-28T13:34:05.139482+0000","last_fullsized":"2024-04-28T13:34:05.139482+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:22:35.458698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880139+0000","last_change":"2024-04-28T13:33:32.989759+0000","last_active":"2024-04-28T13:34:04.880139+0000","last_peered":"2024-04-28T13:34:04.880139+0000","last_clean":"2024-04-28T13:34:04.880139+0000","last_became_active":"2024-04-28T13:33:32.988565+0000","last_became_peered":"2024-04-28T13:33:32.988565+0000","last_unstale":"2024-04-28T13:34:04.880139+0000","last_undegraded":"2024-04-28T13:34:04.880139+0000","last_fullsized":"2024-04-28T13:34:04.880139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:27:52.790170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347610+0000","last_change":"2024-04-28T13:33:34.007736+0000","last_active":"2024-04-28T13:34:04.347610+0000","last_peered":"2024-04-28T13:34:04.347610+0000","last_clean":"2024-04-28T13:34:04.347610+0000","last_became_active":"2024-04-28T13:33:34.007170+0000","last_became_peered":"2024-04-28T13:33:34.007170+0000","last_unstale":"2024-04-28T13:34:04.347610+0000","last_undegraded":"2024-04-28T13:34:04.347610+0000","last_fullsized":"2024-04-28T13:34:04.347610+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:01:33.789936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347247+0000","last_change":"2024-04-28T13:33:32.992483+0000","last_active":"2024-04-28T13:34:04.347247+0000","last_peered":"2024-04-28T13:34:04.347247+0000","last_clean":"2024-04-28T13:34:04.347247+0000","last_became_active":"2024-04-28T13:33:32.992216+0000","last_became_peered":"2024-04-28T13:33:32.992216+0000","last_unstale":"2024-04-28T13:34:04.347247+0000","last_undegraded":"2024-04-28T13:34:04.347247+0000","last_fullsized":"2024-04-28T13:34:04.347247+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:11.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350567+0000","last_change":"2024-04-28T13:33:34.009420+0000","last_active":"2024-04-28T13:34:04.350567+0000","last_peered":"2024-04-28T13:34:04.350567+0000","last_clean":"2024-04-28T13:34:04.350567+0000","last_became_active":"2024-04-28T13:33:34.009020+0000","last_became_peered":"2024-04-28T13:33:34.009020+0000","last_unstale":"2024-04-28T13:34:04.350567+0000","last_undegraded":"2024-04-28T13:34:04.350567+0000","last_fullsized":"2024-04-28T13:34:04.350567+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:48:58.152999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880919+0000","last_change":"2024-04-28T13:33:32.997498+0000","last_active":"2024-04-28T13:34:04.880919+0000","last_peered":"2024-04-28T13:34:04.880919+0000","last_clean":"2024-04-28T13:34:04.880919+0000","last_became_active":"2024-04-28T13:33:32.997237+0000","last_became_peered":"2024-04-28T13:33:32.997237+0000","last_unstale":"2024-04-28T13:34:04.880919+0000","last_undegraded":"2024-04-28T13:34:04.880919+0000","last_fullsized":"2024-04-28T13:34:04.880919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:37:18.628510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344809+0000","last_change":"2024-04-28T13:33:34.002865+0000","last_active":"2024-04-28T13:34:04.344809+0000","last_peered":"2024-04-28T13:34:04.344809+0000","last_clean":"2024-04-28T13:34:04.344809+0000","last_became_active":"2024-04-28T13:33:34.002421+0000","last_became_peered":"2024-04-28T13:33:34.002421+0000","last_unstale":"2024-04-28T13:34:04.344809+0000","last_undegraded":"2024-04-28T13:34:04.344809+0000","last_fullsized":"2024-04-28T13:34:04.344809+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:51:46.332983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350650+0000","last_change":"2024-04-28T13:33:33.004559+0000","last_active":"2024-04-28T13:34:04.350650+0000","last_peered":"2024-04-28T13:34:04.350650+0000","last_clean":"2024-04-28T13:34:04.350650+0000","last_became_active":"2024-04-28T13:33:33.004153+0000","last_became_peered":"2024-04-28T13:33:33.004153+0000","last_unstale":"2024-04-28T13:34:04.350650+0000","last_undegraded":"2024-04-28T13:34:04.350650+0000","last_fullsized":"2024-04-28T13:34:04.350650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:02.432193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350366+0000","last_change":"2024-04-28T13:33:34.006854+0000","last_active":"2024-04-28T13:34:04.350366+0000","last_peered":"2024-04-28T13:34:04.350366+0000","last_clean":"2024-04-28T13:34:04.350366+0000","last_became_active":"2024-04-28T13:33:34.006561+0000","last_became_peered":"2024-04-28T13:33:34.006561+0000","last_unstale":"2024-04-28T13:34:04.350366+0000","last_undegraded":"2024-04-28T13:34:04.350366+0000","last_fullsized":"2024-04-28T13:34:04.350366+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:54.852573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'12","reported_seq":30,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:47.492938+0000","last_change":"2024-04-28T13:33:32.991367+0000","last_active":"2024-04-28T13:35:47.492938+0000","last_peered":"2024-04-28T13:35:47.492938+0000","last_clean":"2024-04-28T13:35:47.492938+0000","last_became_active":"2024-04-28T13:33:32.991060+0000","last_became_peered":"2024-04-28T13:33:32.991060+0000","last_unstale":"2024-04-28T13:35:47.492938+0000","last_undegraded":"2024-04-28T13:35:47.492938+0000","last_fullsized":"2024-04-28T13:35:47.492938+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:00:16.144423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3397,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139850+0000","last_change":"2024-04-28T13:33:33.996658+0000","last_active":"2024-04-28T13:34:05.139850+0000","last_peered":"2024-04-28T13:34:05.139850+0000","last_clean":"2024-04-28T13:34:05.139850+0000","last_became_active":"2024-04-28T13:33:33.996260+0000","last_became_peered":"2024-04-28T13:33:33.996260+0000","last_unstale":"2024-04-28T13:34:05.139850+0000","last_undegraded":"2024-04-28T13:34:05.139850+0000","last_fullsized":"2024-04-28T13:34:05.139850+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:52.292838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345011+0000","last_change":"2024-04-28T13:33:32.990968+0000","last_active":"2024-04-28T13:34:04.345011+0000","last_peered":"2024-04-28T13:34:04.345011+0000","last_clean":"2024-04-28T13:34:04.345011+0000","last_became_active":"2024-04-28T13:33:32.990718+0000","last_became_peered":"2024-04-28T13:33:32.990718+0000","last_unstale":"2024-04-28T13:34:04.345011+0000","last_undegraded":"2024-04-28T13:34:04.345011+0000","last_fullsized":"2024-04-28T13:34:04.345011+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:03.358943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880822+0000","last_change":"2024-04-28T13:33:34.003961+0000","last_active":"2024-04-28T13:34:04.880822+0000","last_peered":"2024-04-28T13:34:04.880822+0000","last_clean":"2024-04-28T13:34:04.880822+0000","last_became_active":"2024-04-28T13:33:34.003599+0000","last_became_peered":"2024-04-28T13:33:34.003599+0000","last_unstale":"2024-04-28T13:34:04.880822+0000","last_undegraded":"2024-04-28T13:34:04.880822+0000","last_fullsized":"2024-04-28T13:34:04.880822+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:32:47.409486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349522+0000","last_change":"2024-04-28T13:33:33.000521+0000","last_active":"2024-04-28T13:34:04.349522+0000","last_peered":"2024-04-28T13:34:04.349522+0000","last_clean":"2024-04-28T13:34:04.349522+0000","last_became_active":"2024-04-28T13:33:32.998867+0000","last_became_peered":"2024-04-28T13:33:32.998867+0000","last_unstale":"2024-04-28T13:34:04.349522+0000","last_undegraded":"2024-04-28T13:34:04.349522+0000","last_fullsized":"2024-04-28T13:34:04.349522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:43.562896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344089+0000","last_change":"2024-04-28T13:33:34.006368+0000","last_active":"2024-04-28T13:34:04.344089+0000","last_peered":"2024-04-28T13:34:04.344089+0000","last_clean":"2024-04-28T13:34:04.344089+0000","last_became_active":"2024-04-28T13:33:34.002579+0000","last_became_peered":"2024-04-28T13:33:34.002579+0000","last_unstale":"2024-04-28T13:34:04.344089+0000","last_undegraded":"2024-04-28T13:34:04.344089+0000","last_fullsized":"2024-04-28T13:34:04.344089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:27:51.330692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345395+0000","last_change":"2024-04-28T13:33:32.991648+0000","last_active":"2024-04-28T13:34:04.345395+0000","last_peered":"2024-04-28T13:34:04.345395+0000","last_clean":"2024-04-28T13:34:04.345395+0000","last_became_active":"2024-04-28T13:33:32.991369+0000","last_became_peered":"2024-04-28T13:33:32.991369+0000","last_unstale":"2024-04-28T13:34:04.345395+0000","last_undegraded":"2024-04-28T13:34:04.345395+0000","last_fullsized":"2024-04-28T13:34:04.345395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:43.562896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349607+0000","last_change":"2024-04-28T13:33:34.010407+0000","last_active":"2024-04-28T13:34:04.349607+0000","last_peered":"2024-04-28T13:34:04.349607+0000","last_clean":"2024-04-28T13:34:04.349607+0000","last_became_active":"2024-04-28T13:33:34.010077+0000","last_became_peered":"2024-04-28T13:33:34.010077+0000","last_unstale":"2024-04-28T13:34:04.349607+0000","last_undegraded":"2024-04-28T13:34:04.349607+0000","last_fullsized":"2024-04-28T13:34:04.349607+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:20:07.691466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880423+0000","last_change":"2024-04-28T13:33:32.989587+0000","last_active":"2024-04-28T13:34:04.880423+0000","last_peered":"2024-04-28T13:34:04.880423+0000","last_clean":"2024-04-28T13:34:04.880423+0000","last_became_active":"2024-04-28T13:33:32.988579+0000","last_became_peered":"2024-04-28T13:33:32.988579+0000","last_unstale":"2024-04-28T13:34:04.880423+0000","last_undegraded":"2024-04-28T13:34:04.880423+0000","last_fullsized":"2024-04-28T13:34:04.880423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:35.233850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.532050+0000","last_change":"2024-04-28T13:33:34.012322+0000","last_active":"2024-04-28T13:34:05.532050+0000","last_peered":"2024-04-28T13:34:05.532050+0000","last_clean":"2024-04-28T13:34:05.532050+0000","last_became_active":"2024-04-28T13:33:34.012112+0000","last_became_peered":"2024-04-28T13:33:34.012112+0000","last_unstale":"2024-04-28T13:34:05.532050+0000","last_undegraded":"2024-04-28T13:34:05.532050+0000","last_fullsized":"2024-04-28T13:34:05.532050+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:10.015862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531304+0000","last_change":"2024-04-28T13:33:32.990958+0000","last_active":"2024-04-28T13:34:05.531304+0000","last_peered":"2024-04-28T13:34:05.531304+0000","last_clean":"2024-04-28T13:34:05.531304+0000","last_became_active":"2024-04-28T13:33:32.990659+0000","last_became_peered":"2024-04-28T13:33:32.990659+0000","last_unstale":"2024-04-28T13:34:05.531304+0000","last_undegraded":"2024-04-28T13:34:05.531304+0000","last_fullsized":"2024-04-28T13:34:05.531304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:24:34.028457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.881209+0000","last_change":"2024-04-28T13:33:34.005832+0000","last_active":"2024-04-28T13:34:04.881209+0000","last_peered":"2024-04-28T13:34:04.881209+0000","last_clean":"2024-04-28T13:34:04.881209+0000","last_became_active":"2024-04-28T13:33:34.005364+0000","last_became_peered":"2024-04-28T13:33:34.005364+0000","last_unstale":"2024-04-28T13:34:04.881209+0000","last_undegraded":"2024-04-28T13:34:04.881209+0000","last_fullsized":"2024-04-28T13:34:04.881209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:28.548233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.348084+0000","last_change":"2024-04-28T13:33:32.995338+0000","last_active":"2024-04-28T13:34:04.348084+0000","last_peered":"2024-04-28T13:34:04.348084+0000","last_clean":"2024-04-28T13:34:04.348084+0000","last_became_active":"2024-04-28T13:33:32.994798+0000","last_became_peered":"2024-04-28T13:33:32.994798+0000","last_unstale":"2024-04-28T13:34:04.348084+0000","last_undegraded":"2024-04-28T13:34:04.348084+0000","last_fullsized":"2024-04-28T13:34:04.348084+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:29.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.140801+0000","last_change":"2024-04-28T13:33:34.002242+0000","last_active":"2024-04-28T13:34:05.140801+0000","last_peered":"2024-04-28T13:34:05.140801+0000","last_clean":"2024-04-28T13:34:05.140801+0000","last_became_active":"2024-04-28T13:33:34.001893+0000","last_became_peered":"2024-04-28T13:33:34.001893+0000","last_unstale":"2024-04-28T13:34:05.140801+0000","last_undegraded":"2024-04-28T13:34:05.140801+0000","last_fullsized":"2024-04-28T13:34:05.140801+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:35.969568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355718+0000","last_change":"2024-04-28T13:33:32.991441+0000","last_active":"2024-04-28T13:34:04.355718+0000","last_peered":"2024-04-28T13:34:04.355718+0000","last_clean":"2024-04-28T13:34:04.355718+0000","last_became_active":"2024-04-28T13:33:32.990745+0000","last_became_peered":"2024-04-28T13:33:32.990745+0000","last_unstale":"2024-04-28T13:34:04.355718+0000","last_undegraded":"2024-04-28T13:34:04.355718+0000","last_fullsized":"2024-04-28T13:34:04.355718+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:30.656667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.140616+0000","last_change":"2024-04-28T13:33:34.003094+0000","last_active":"2024-04-28T13:34:05.140616+0000","last_peered":"2024-04-28T13:34:05.140616+0000","last_clean":"2024-04-28T13:34:05.140616+0000","last_became_active":"2024-04-28T13:33:34.002228+0000","last_became_peered":"2024-04-28T13:33:34.002228+0000","last_unstale":"2024-04-28T13:34:05.140616+0000","last_undegraded":"2024-04-28T13:34:05.140616+0000","last_fullsized":"2024-04-28T13:34:05.140616+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:31.671210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344933+0000","last_change":"2024-04-28T13:33:33.003720+0000","last_active":"2024-04-28T13:34:04.344933+0000","last_peered":"2024-04-28T13:34:04.344933+0000","last_clean":"2024-04-28T13:34:04.344933+0000","last_became_active":"2024-04-28T13:33:33.003389+0000","last_became_peered":"2024-04-28T13:33:33.003389+0000","last_unstale":"2024-04-28T13:34:04.344933+0000","last_undegraded":"2024-04-28T13:34:04.344933+0000","last_fullsized":"2024-04-28T13:34:04.344933+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:18.814381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345100+0000","last_change":"2024-04-28T13:33:34.006579+0000","last_active":"2024-04-28T13:34:04.345100+0000","last_peered":"2024-04-28T13:34:04.345100+0000","last_clean":"2024-04-28T13:34:04.345100+0000","last_became_active":"2024-04-28T13:33:34.006282+0000","last_became_peered":"2024-04-28T13:33:34.006282+0000","last_unstale":"2024-04-28T13:34:04.345100+0000","last_undegraded":"2024-04-28T13:34:04.345100+0000","last_fullsized":"2024-04-28T13:34:04.345100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:18:15.243319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345377+0000","last_change":"2024-04-28T13:33:32.999347+0000","last_active":"2024-04-28T13:34:04.345377+0000","last_peered":"2024-04-28T13:34:04.345377+0000","last_clean":"2024-04-28T13:34:04.345377+0000","last_became_active":"2024-04-28T13:33:32.998923+0000","last_became_peered":"2024-04-28T13:33:32.998923+0000","last_unstale":"2024-04-28T13:34:04.345377+0000","last_undegraded":"2024-04-28T13:34:04.345377+0000","last_fullsized":"2024-04-28T13:34:04.345377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:46.522691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.492925+0000","last_change":"2024-04-28T13:33:34.011875+0000","last_active":"2024-04-28T13:34:04.492925+0000","last_peered":"2024-04-28T13:34:04.492925+0000","last_clean":"2024-04-28T13:34:04.492925+0000","last_became_active":"2024-04-28T13:33:34.011271+0000","last_became_peered":"2024-04-28T13:33:34.011271+0000","last_unstale":"2024-04-28T13:34:04.492925+0000","last_undegraded":"2024-04-28T13:34:04.492925+0000","last_fullsized":"2024-04-28T13:34:04.492925+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:35.969568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354592+0000","last_change":"2024-04-28T13:33:33.002216+0000","last_active":"2024-04-28T13:34:04.354592+0000","last_peered":"2024-04-28T13:34:04.354592+0000","last_clean":"2024-04-28T13:34:04.354592+0000","last_became_active":"2024-04-28T13:33:33.000335+0000","last_became_peered":"2024-04-28T13:33:33.000335+0000","last_unstale":"2024-04-28T13:34:04.354592+0000","last_undegraded":"2024-04-28T13:34:04.354592+0000","last_fullsized":"2024-04-28T13:34:04.354592+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:34.955026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531906+0000","last_change":"2024-04-28T13:33:34.000104+0000","last_active":"2024-04-28T13:34:05.531906+0000","last_peered":"2024-04-28T13:34:05.531906+0000","last_clean":"2024-04-28T13:34:05.531906+0000","last_became_active":"2024-04-28T13:33:33.999839+0000","last_became_peered":"2024-04-28T13:33:33.999839+0000","last_unstale":"2024-04-28T13:34:05.531906+0000","last_undegraded":"2024-04-28T13:34:05.531906+0000","last_fullsized":"2024-04-28T13:34:05.531906+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:03:14.657009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879925+0000","last_change":"2024-04-28T13:33:32.992149+0000","last_active":"2024-04-28T13:34:04.879925+0000","last_peered":"2024-04-28T13:34:04.879925+0000","last_clean":"2024-04-28T13:34:04.879925+0000","last_became_active":"2024-04-28T13:33:32.991800+0000","last_became_peered":"2024-04-28T13:33:32.991800+0000","last_unstale":"2024-04-28T13:34:04.879925+0000","last_undegraded":"2024-04-28T13:34:04.879925+0000","last_fullsized":"2024-04-28T13:34:04.879925+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:56:39.708282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879801+0000","last_change":"2024-04-28T13:33:34.011133+0000","last_active":"2024-04-28T13:34:04.879801+0000","last_peered":"2024-04-28T13:34:04.879801+0000","last_clean":"2024-04-28T13:34:04.879801+0000","last_became_active":"2024-04-28T13:33:34.010896+0000","last_became_peered":"2024-04-28T13:33:34.010896+0000","last_unstale":"2024-04-28T13:34:04.879801+0000","last_undegraded":"2024-04-28T13:34:04.879801+0000","last_fullsized":"2024-04-28T13:34:04.879801+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.348967+0000","last_change":"2024-04-28T13:33:33.000859+0000","last_active":"2024-04-28T13:34:04.348967+0000","last_peered":"2024-04-28T13:34:04.348967+0000","last_clean":"2024-04-28T13:34:04.348967+0000","last_became_active":"2024-04-28T13:33:33.000095+0000","last_became_peered":"2024-04-28T13:33:33.000095+0000","last_unstale":"2024-04-28T13:34:04.348967+0000","last_undegraded":"2024-04-28T13:34:04.348967+0000","last_fullsized":"2024-04-28T13:34:04.348967+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:18.221214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.141479+0000","last_change":"2024-04-28T13:33:34.008405+0000","last_active":"2024-04-28T13:34:05.141479+0000","last_peered":"2024-04-28T13:34:05.141479+0000","last_clean":"2024-04-28T13:34:05.141479+0000","last_became_active":"2024-04-28T13:33:34.007425+0000","last_became_peered":"2024-04-28T13:33:34.007425+0000","last_unstale":"2024-04-28T13:34:05.141479+0000","last_undegraded":"2024-04-28T13:34:05.141479+0000","last_fullsized":"2024-04-28T13:34:05.141479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:29.495657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"47'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:45.005269+0000","last_change":"2024-04-28T13:33:32.998594+0000","last_active":"2024-04-28T13:34:45.005269+0000","last_peered":"2024-04-28T13:34:45.005269+0000","last_clean":"2024-04-28T13:34:45.005269+0000","last_became_active":"2024-04-28T13:33:32.998328+0000","last_became_peered":"2024-04-28T13:33:32.998328+0000","last_unstale":"2024-04-28T13:34:45.005269+0000","last_undegraded":"2024-04-28T13:34:45.005269+0000","last_fullsized":"2024-04-28T13:34:45.005269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:18.221214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880058+0000","last_change":"2024-04-28T13:33:33.998134+0000","last_active":"2024-04-28T13:34:04.880058+0000","last_peered":"2024-04-28T13:34:04.880058+0000","last_clean":"2024-04-28T13:34:04.880058+0000","last_became_active":"2024-04-28T13:33:33.997834+0000","last_became_peered":"2024-04-28T13:33:33.997834+0000","last_unstale":"2024-04-28T13:34:04.880058+0000","last_undegraded":"2024-04-28T13:34:04.880058+0000","last_fullsized":"2024-04-28T13:34:04.880058+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:01:33.789936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350856+0000","last_change":"2024-04-28T13:33:33.005737+0000","last_active":"2024-04-28T13:34:04.350856+0000","last_peered":"2024-04-28T13:34:04.350856+0000","last_clean":"2024-04-28T13:34:04.350856+0000","last_became_active":"2024-04-28T13:33:33.005179+0000","last_became_peered":"2024-04-28T13:33:33.005179+0000","last_unstale":"2024-04-28T13:34:04.350856+0000","last_undegraded":"2024-04-28T13:34:04.350856+0000","last_fullsized":"2024-04-28T13:34:04.350856+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:45.657949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350592+0000","last_change":"2024-04-28T13:33:34.011725+0000","last_active":"2024-04-28T13:34:04.350592+0000","last_peered":"2024-04-28T13:34:04.350592+0000","last_clean":"2024-04-28T13:34:04.350592+0000","last_became_active":"2024-04-28T13:33:34.011422+0000","last_became_peered":"2024-04-28T13:33:34.011422+0000","last_unstale":"2024-04-28T13:34:04.350592+0000","last_undegraded":"2024-04-28T13:34:04.350592+0000","last_fullsized":"2024-04-28T13:34:04.350592+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:28.548233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.492969+0000","last_change":"2024-04-28T13:33:32.997338+0000","last_active":"2024-04-28T13:34:04.492969+0000","last_peered":"2024-04-28T13:34:04.492969+0000","last_clean":"2024-04-28T13:34:04.492969+0000","last_became_active":"2024-04-28T13:33:32.996934+0000","last_became_peered":"2024-04-28T13:33:32.996934+0000","last_unstale":"2024-04-28T13:34:04.492969+0000","last_undegraded":"2024-04-28T13:34:04.492969+0000","last_fullsized":"2024-04-28T13:34:04.492969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:29.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345593+0000","last_change":"2024-04-28T13:33:34.020082+0000","last_active":"2024-04-28T13:34:04.345593+0000","last_peered":"2024-04-28T13:34:04.345593+0000","last_clean":"2024-04-28T13:34:04.345593+0000","last_became_active":"2024-04-28T13:33:34.018922+0000","last_became_peered":"2024-04-28T13:33:34.018922+0000","last_unstale":"2024-04-28T13:34:04.345593+0000","last_undegraded":"2024-04-28T13:34:04.345593+0000","last_fullsized":"2024-04-28T13:34:04.345593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:53:12.865627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350434+0000","last_change":"2024-04-28T13:33:33.000626+0000","last_active":"2024-04-28T13:34:04.350434+0000","last_peered":"2024-04-28T13:34:04.350434+0000","last_clean":"2024-04-28T13:34:04.350434+0000","last_became_active":"2024-04-28T13:33:33.000049+0000","last_became_peered":"2024-04-28T13:33:33.000049+0000","last_unstale":"2024-04-28T13:34:04.350434+0000","last_undegraded":"2024-04-28T13:34:04.350434+0000","last_fullsized":"2024-04-28T13:34:04.350434+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:18.814381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344459+0000","last_change":"2024-04-28T13:33:34.004230+0000","last_active":"2024-04-28T13:34:04.344459+0000","last_peered":"2024-04-28T13:34:04.344459+0000","last_clean":"2024-04-28T13:34:04.344459+0000","last_became_active":"2024-04-28T13:33:34.003940+0000","last_became_peered":"2024-04-28T13:33:34.003940+0000","last_unstale":"2024-04-28T13:34:04.344459+0000","last_undegraded":"2024-04-28T13:34:04.344459+0000","last_fullsized":"2024-04-28T13:34:04.344459+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:11.732507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346733+0000","last_change":"2024-04-28T13:33:32.991869+0000","last_active":"2024-04-28T13:34:04.346733+0000","last_peered":"2024-04-28T13:34:04.346733+0000","last_clean":"2024-04-28T13:34:04.346733+0000","last_became_active":"2024-04-28T13:33:32.991435+0000","last_became_peered":"2024-04-28T13:33:32.991435+0000","last_unstale":"2024-04-28T13:34:04.346733+0000","last_undegraded":"2024-04-28T13:34:04.346733+0000","last_fullsized":"2024-04-28T13:34:04.346733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:41.068187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354423+0000","last_change":"2024-04-28T13:33:34.004178+0000","last_active":"2024-04-28T13:34:04.354423+0000","last_peered":"2024-04-28T13:34:04.354423+0000","last_clean":"2024-04-28T13:34:04.354423+0000","last_became_active":"2024-04-28T13:33:34.002092+0000","last_became_peered":"2024-04-28T13:33:34.002092+0000","last_unstale":"2024-04-28T13:34:04.354423+0000","last_undegraded":"2024-04-28T13:34:04.354423+0000","last_fullsized":"2024-04-28T13:34:04.354423+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:03.446736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493055+0000","last_change":"2024-04-28T13:33:33.003796+0000","last_active":"2024-04-28T13:34:04.493055+0000","last_peered":"2024-04-28T13:34:04.493055+0000","last_clean":"2024-04-28T13:34:04.493055+0000","last_became_active":"2024-04-28T13:33:33.003374+0000","last_became_peered":"2024-04-28T13:33:33.003374+0000","last_unstale":"2024-04-28T13:34:04.493055+0000","last_undegraded":"2024-04-28T13:34:04.493055+0000","last_fullsized":"2024-04-28T13:34:04.493055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:11.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347052+0000","last_change":"2024-04-28T13:33:34.001551+0000","last_active":"2024-04-28T13:34:04.347052+0000","last_peered":"2024-04-28T13:34:04.347052+0000","last_clean":"2024-04-28T13:34:04.347052+0000","last_became_active":"2024-04-28T13:33:33.999963+0000","last_became_peered":"2024-04-28T13:33:33.999963+0000","last_unstale":"2024-04-28T13:34:04.347052+0000","last_undegraded":"2024-04-28T13:34:04.347052+0000","last_fullsized":"2024-04-28T13:34:04.347052+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:19.828924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880581+0000","last_change":"2024-04-28T13:33:32.991033+0000","last_active":"2024-04-28T13:34:04.880581+0000","last_peered":"2024-04-28T13:34:04.880581+0000","last_clean":"2024-04-28T13:34:04.880581+0000","last_became_active":"2024-04-28T13:33:32.990726+0000","last_became_peered":"2024-04-28T13:33:32.990726+0000","last_unstale":"2024-04-28T13:34:04.880581+0000","last_undegraded":"2024-04-28T13:34:04.880581+0000","last_fullsized":"2024-04-28T13:34:04.880581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:09.001319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880728+0000","last_change":"2024-04-28T13:33:34.005421+0000","last_active":"2024-04-28T13:34:04.880728+0000","last_peered":"2024-04-28T13:34:04.880728+0000","last_clean":"2024-04-28T13:34:04.880728+0000","last_became_active":"2024-04-28T13:33:34.004776+0000","last_became_peered":"2024-04-28T13:33:34.004776+0000","last_unstale":"2024-04-28T13:34:04.880728+0000","last_undegraded":"2024-04-28T13:34:04.880728+0000","last_fullsized":"2024-04-28T13:34:04.880728+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:20:07.691466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.356034+0000","last_change":"2024-04-28T13:33:32.991462+0000","last_active":"2024-04-28T13:34:04.356034+0000","last_peered":"2024-04-28T13:34:04.356034+0000","last_clean":"2024-04-28T13:34:04.356034+0000","last_became_active":"2024-04-28T13:33:32.991149+0000","last_became_peered":"2024-04-28T13:33:32.991149+0000","last_unstale":"2024-04-28T13:34:04.356034+0000","last_undegraded":"2024-04-28T13:34:04.356034+0000","last_fullsized":"2024-04-28T13:34:04.356034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:41.068187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":220,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349073+0000","last_change":"2024-04-28T13:31:11.831538+0000","last_active":"2024-04-28T13:34:04.349073+0000","last_peered":"2024-04-28T13:34:04.349073+0000","last_clean":"2024-04-28T13:34:04.349073+0000","last_became_active":"2024-04-28T13:31:11.828959+0000","last_became_peered":"2024-04-28T13:31:11.828959+0000","last_unstale":"2024-04-28T13:34:04.349073+0000","last_undegraded":"2024-04-28T13:34:04.349073+0000","last_fullsized":"2024-04-28T13:34:04.349073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_clean_scrub_stamp":"2024-04-28T13:31:10.444388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:25:27.078911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":4195,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343692,"statfs":{"total":96636764160,"available":96607940608,"internally_reserved":0,"allocated":1150976,"data_stored":818777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":561152,"data_stored":228991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":228389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":565248,"data_stored":231575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":565248,"data_stored":228957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":569344,"data_stored":231597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":228389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1150976,"data_stored":818721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T13:35:51.675 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T13:35:51.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.918+0000 7f77c5200640 1 Processor -- start 2024-04-28T13:35:51.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.918+0000 7f77c5200640 1 -- start start 2024-04-28T13:35:51.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.919+0000 7f77c5200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c00b5ee0 0x7f77c00b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.920+0000 7f77c5200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77c015e380 con 0x7f77c00d1760 2024-04-28T13:35:51.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.920+0000 7f77c5200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77c00b4b20 con 0x7f77c009ec80 2024-04-28T13:35:51.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.920+0000 7f77c5200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f77c00b3f30 con 0x7f77c00b5ee0 2024-04-28T13:35:51.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.920+0000 7f77bfe00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f77c00d1760 0x7f77c006df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:53162/0 (socket says 172.21.15.196:53162) 2024-04-28T13:35:51.920 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.920+0000 7f77bfe00640 1 -- 172.21.15.196:0/2654260988 learned_addr learned my addr 172.21.15.196:0/2654260988 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:51.922 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.920+0000 7f77bf400640 1 --2- 172.21.15.196:0/2654260988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c00b5ee0 0x7f77c00b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:51.922 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.920+0000 7f77be000640 1 -- 172.21.15.196:0/2654260988 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1162031738 0 0) 0x7f77c015e380 con 0x7f77c00d1760 2024-04-28T13:35:51.922 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.920+0000 7f77be000640 1 -- 172.21.15.196:0/2654260988 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77a4003520 con 0x7f77c00d1760 2024-04-28T13:35:51.922 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.921+0000 7f77be000640 1 -- 172.21.15.196:0/2654260988 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 728318051 0 0) 0x7f77a4003520 con 0x7f77c00d1760 2024-04-28T13:35:51.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.921+0000 7f77be000640 1 -- 172.21.15.196:0/2654260988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c00b5ee0 msgr2=0x7f77c00b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:51.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.921+0000 7f77be000640 1 --2- 172.21.15.196:0/2654260988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c00b5ee0 0x7f77c00b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:51.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.921+0000 7f77be000640 1 -- 172.21.15.196:0/2654260988 >> v1:172.21.15.203:6789/0 conn(0x7f77c009ec80 legacy=0x7f77c00d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:51.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.921+0000 7f77be000640 1 -- 172.21.15.196:0/2654260988 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c00b41f0 con 0x7f77c00d1760 2024-04-28T13:35:51.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.921+0000 7f77be000640 1 -- 172.21.15.196:0/2654260988 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f77b80019f0 con 0x7f77c00d1760 2024-04-28T13:35:51.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.921+0000 7f77be000640 1 -- 172.21.15.196:0/2654260988 >> v1:172.21.15.196:6789/0 conn(0x7f77c00d1760 legacy=0x7f77c006df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:51.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.921+0000 7f77be000640 1 --2- 172.21.15.196:0/2654260988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77a40039e0 0x7f77a4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.921+0000 7f77be000640 1 --2- 172.21.15.196:0/2654260988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a4004800 0x7f77a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.921+0000 7f77be000640 1 --2- 172.21.15.196:0/2654260988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c00b5ee0 0x7f77a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.921+0000 7f77be000640 1 -- 172.21.15.196:0/2654260988 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00b3f30 con 0x7f77a4004800 2024-04-28T13:35:51.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.921+0000 7f77be000640 1 -- 172.21.15.196:0/2654260988 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f77a400ba70 con 0x7f77a40039e0 2024-04-28T13:35:51.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.921+0000 7f77be000640 1 -- 172.21.15.196:0/2654260988 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f77a400bc70 con 0x7f77c00b5ee0 2024-04-28T13:35:51.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.921+0000 7f77bea00640 1 --2- 172.21.15.196:0/2654260988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a4004800 0x7f77a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:51.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.922+0000 7f77bf400640 1 --2- 172.21.15.196:0/2654260988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77a40039e0 0x7f77a4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:51.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.922+0000 7f77bea00640 1 -- 172.21.15.196:0/2654260988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c00b5ee0 msgr2=0x7f77a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:51.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.922+0000 7f77bea00640 1 --2- 172.21.15.196:0/2654260988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c00b5ee0 0x7f77a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:51.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.922+0000 7f77bea00640 1 -- 172.21.15.196:0/2654260988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77a40039e0 msgr2=0x7f77a4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:51.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.922+0000 7f77bea00640 1 --2- 172.21.15.196:0/2654260988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77a40039e0 0x7f77a4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:51.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.922+0000 7f77bea00640 1 -- 172.21.15.196:0/2654260988 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a4003520 con 0x7f77a4004800 2024-04-28T13:35:51.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.923+0000 7f77bea00640 1 --2- 172.21.15.196:0/2654260988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a4004800 0x7f77a4004c80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f77c009f0c0 tx=0x7f77a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff428c6b571b718b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:51.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.923+0000 7f77be000640 1 -- 172.21.15.196:0/2654260988 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac001370 con 0x7f77a4004800 2024-04-28T13:35:51.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.923+0000 7f77be000640 1 -- 172.21.15.196:0/2654260988 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77ac0052d0 con 0x7f77a4004800 2024-04-28T13:35:51.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.923+0000 7f77be000640 1 -- 172.21.15.196:0/2654260988 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac02ab70 con 0x7f77a4004800 2024-04-28T13:35:51.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.924+0000 7f77c5200640 1 -- 172.21.15.196:0/2654260988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a4004800 msgr2=0x7f77a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:51.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.924+0000 7f77c5200640 1 --2- 172.21.15.196:0/2654260988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a4004800 0x7f77a4004c80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f77c009f0c0 tx=0x7f77a4005830 comp rx=0 tx=0).stop 2024-04-28T13:35:51.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.924+0000 7f77bfe00640 1 -- 172.21.15.196:0/2654260988 reap_dead start 2024-04-28T13:35:51.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.924+0000 7f77c5200640 1 -- 172.21.15.196:0/2654260988 shutdown_connections 2024-04-28T13:35:51.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.924+0000 7f77c5200640 1 -- 172.21.15.196:0/2654260988 >> 172.21.15.196:0/2654260988 conn(0x7f77c006c6c0 msgr2=0x7f77c006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:51.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.924+0000 7f77c5200640 1 -- 172.21.15.196:0/2654260988 shutdown_connections 2024-04-28T13:35:51.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.924+0000 7f77c5200640 1 -- 172.21.15.196:0/2654260988 wait complete. 2024-04-28T13:35:51.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.926+0000 7f77c5200640 1 Processor -- start 2024-04-28T13:35:51.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.926+0000 7f77c5200640 1 -- start start 2024-04-28T13:35:51.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.927+0000 7f77c5200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77a4004800 0x7f77c0146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.927+0000 7f77c5200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00cefe0 0x7f77c00cf460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.927+0000 7f77c5200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c0159000 0x7f77c01594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.927+0000 7f77c5200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00b3c20 con 0x7f77c00cefe0 2024-04-28T13:35:51.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.927+0000 7f77c5200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00b47b0 con 0x7f77a4004800 2024-04-28T13:35:51.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.927+0000 7f77c5200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f77c00add70 con 0x7f77c0159000 2024-04-28T13:35:51.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.927+0000 7f77bea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00cefe0 0x7f77c00cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:51.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.927+0000 7f77bea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00cefe0 0x7f77c00cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:38936/0 (socket says 172.21.15.196:38936) 2024-04-28T13:35:51.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.927+0000 7f77bea00640 1 -- 172.21.15.196:0/4209461717 learned_addr learned my addr 172.21.15.196:0/4209461717 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:51.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.927+0000 7f77bf400640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77a4004800 0x7f77c0146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:51.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.927+0000 7f77bfe00640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c0159000 0x7f77c01594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:51.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.928+0000 7f77bea00640 1 -- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c0159000 msgr2=0x7f77c01594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:51.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.928+0000 7f77bea00640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c0159000 0x7f77c01594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:51.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.928+0000 7f77bea00640 1 -- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77a4004800 msgr2=0x7f77c0146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:51.930 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.928+0000 7f77bea00640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77a4004800 0x7f77c0146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:51.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.928+0000 7f77bea00640 1 -- 172.21.15.196:0/4209461717 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c0159da0 con 0x7f77c00cefe0 2024-04-28T13:35:51.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.928+0000 7f77bf400640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77a4004800 0x7f77c0146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:35:51.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.928+0000 7f77bfe00640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c0159000 0x7f77c01594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:35:51.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.928+0000 7f77bea00640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00cefe0 0x7f77c00cf460 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f77ac02a520 tx=0x7f77ac001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:51.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.928+0000 7f77b7400640 1 -- 172.21.15.196:0/4209461717 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac037040 con 0x7f77c00cefe0 2024-04-28T13:35:51.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.928+0000 7f77b7400640 1 -- 172.21.15.196:0/4209461717 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77ac042070 con 0x7f77c00cefe0 2024-04-28T13:35:51.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.928+0000 7f77b7400640 1 -- 172.21.15.196:0/4209461717 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac007a30 con 0x7f77c00cefe0 2024-04-28T13:35:51.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.928+0000 7f77c5200640 1 -- 172.21.15.196:0/4209461717 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c0149210 con 0x7f77c00cefe0 2024-04-28T13:35:51.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.928+0000 7f77c5200640 1 -- 172.21.15.196:0/4209461717 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c01497d0 con 0x7f77c00cefe0 2024-04-28T13:35:51.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.930+0000 7f77c5200640 1 -- 172.21.15.196:0/4209461717 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c00cf9a0 con 0x7f77c00cefe0 2024-04-28T13:35:51.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.931+0000 7f77b7400640 1 -- 172.21.15.196:0/4209461717 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f77ac0052d0 con 0x7f77c00cefe0 2024-04-28T13:35:51.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.931+0000 7f77b7400640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f77a006c980 0x7f77a006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:51.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.932+0000 7f77b7400640 1 -- 172.21.15.196:0/4209461717 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f77ac0abff0 con 0x7f77c00cefe0 2024-04-28T13:35:51.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.932+0000 7f77bf400640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f77a006c980 0x7f77a006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:51.933 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.933+0000 7f77bf400640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f77a006c980 0x7f77a006ee40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009a00 tx=0x7f77b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:51.936 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:51.936+0000 7f77b7400640 1 -- 172.21.15.196:0/4209461717 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f77ac07a2f0 con 0x7f77c00cefe0 2024-04-28T13:35:52.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.132+0000 7f77c5200640 1 -- 172.21.15.196:0/4209461717 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f77c006ee60 con 0x7f77a006c980 2024-04-28T13:35:52.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.151+0000 7f77b7400640 1 -- 172.21.15.196:0/4209461717 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251882 (secure 0 0 0) 0x7f77c006ee60 con 0x7f77a006c980 2024-04-28T13:35:52.154 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:35:52.156 INFO:teuthology.orchestra.run.smithi196.stderr:dumped all 2024-04-28T13:35:52.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.156+0000 7f77c5200640 1 -- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f77a006c980 msgr2=0x7f77a006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:52.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.156+0000 7f77c5200640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f77a006c980 0x7f77a006ee40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009a00 tx=0x7f77b0008040 comp rx=0 tx=0).stop 2024-04-28T13:35:52.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.156+0000 7f77c5200640 1 -- 172.21.15.196:0/4209461717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00cefe0 msgr2=0x7f77c00cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:52.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.156+0000 7f77c5200640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00cefe0 0x7f77c00cf460 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f77ac02a520 tx=0x7f77ac001120 comp rx=0 tx=0).stop 2024-04-28T13:35:52.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.156+0000 7f77c5200640 1 -- 172.21.15.196:0/4209461717 shutdown_connections 2024-04-28T13:35:52.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.156+0000 7f77c5200640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f77a006c980 0x7f77a006ee40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.156+0000 7f77c5200640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77c0159000 0x7f77c01594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.156+0000 7f77c5200640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77c00cefe0 0x7f77c00cf460 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.156+0000 7f77c5200640 1 --2- 172.21.15.196:0/4209461717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77a4004800 0x7f77c0146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.156+0000 7f77c5200640 1 -- 172.21.15.196:0/4209461717 >> 172.21.15.196:0/4209461717 conn(0x7f77c006c6c0 msgr2=0x7f77c0072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:52.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.157+0000 7f77c5200640 1 -- 172.21.15.196:0/4209461717 shutdown_connections 2024-04-28T13:35:52.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.157+0000 7f77c5200640 1 -- 172.21.15.196:0/4209461717 wait complete. 2024-04-28T13:35:52.176 INFO:teuthology.orchestra.run.smithi196.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-04-28T13:35:50.282761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594563,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":212,"num_write_kb":2437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225876,"kb_used_data":5544,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748844,"statfs":{"total":773094113280,"available":772862816256,"internally_reserved":0,"allocated":5677056,"data_stored":3015396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":282,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004237"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350675+0000","last_change":"2024-04-28T13:33:33.999818+0000","last_active":"2024-04-28T13:34:04.350675+0000","last_peered":"2024-04-28T13:34:04.350675+0000","last_clean":"2024-04-28T13:34:04.350675+0000","last_became_active":"2024-04-28T13:33:33.999486+0000","last_became_peered":"2024-04-28T13:33:33.999486+0000","last_unstale":"2024-04-28T13:34:04.350675+0000","last_undegraded":"2024-04-28T13:34:04.350675+0000","last_fullsized":"2024-04-28T13:34:04.350675+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:32:47.409486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355721+0000","last_change":"2024-04-28T13:33:34.003919+0000","last_active":"2024-04-28T13:34:04.355721+0000","last_peered":"2024-04-28T13:34:04.355721+0000","last_clean":"2024-04-28T13:34:04.355721+0000","last_became_active":"2024-04-28T13:33:34.003551+0000","last_became_peered":"2024-04-28T13:33:34.003551+0000","last_unstale":"2024-04-28T13:34:04.355721+0000","last_undegraded":"2024-04-28T13:34:04.355721+0000","last_fullsized":"2024-04-28T13:34:04.355721+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347343+0000","last_change":"2024-04-28T13:33:33.996733+0000","last_active":"2024-04-28T13:34:04.347343+0000","last_peered":"2024-04-28T13:34:04.347343+0000","last_clean":"2024-04-28T13:34:04.347343+0000","last_became_active":"2024-04-28T13:33:33.996413+0000","last_became_peered":"2024-04-28T13:33:33.996413+0000","last_unstale":"2024-04-28T13:34:04.347343+0000","last_undegraded":"2024-04-28T13:34:04.347343+0000","last_fullsized":"2024-04-28T13:34:04.347343+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:46.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879775+0000","last_change":"2024-04-28T13:33:34.004787+0000","last_active":"2024-04-28T13:34:04.879775+0000","last_peered":"2024-04-28T13:34:04.879775+0000","last_clean":"2024-04-28T13:34:04.879775+0000","last_became_active":"2024-04-28T13:33:33.999818+0000","last_became_peered":"2024-04-28T13:33:33.999818+0000","last_unstale":"2024-04-28T13:34:04.879775+0000","last_undegraded":"2024-04-28T13:34:04.879775+0000","last_fullsized":"2024-04-28T13:34:04.879775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354517+0000","last_change":"2024-04-28T13:33:34.004715+0000","last_active":"2024-04-28T13:34:04.354517+0000","last_peered":"2024-04-28T13:34:04.354517+0000","last_clean":"2024-04-28T13:34:04.354517+0000","last_became_active":"2024-04-28T13:33:34.004436+0000","last_became_peered":"2024-04-28T13:33:34.004436+0000","last_unstale":"2024-04-28T13:34:04.354517+0000","last_undegraded":"2024-04-28T13:34:04.354517+0000","last_fullsized":"2024-04-28T13:34:04.354517+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:46.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344494+0000","last_change":"2024-04-28T13:33:34.012959+0000","last_active":"2024-04-28T13:34:04.344494+0000","last_peered":"2024-04-28T13:34:04.344494+0000","last_clean":"2024-04-28T13:34:04.344494+0000","last_became_active":"2024-04-28T13:33:34.010839+0000","last_became_peered":"2024-04-28T13:33:34.010839+0000","last_unstale":"2024-04-28T13:34:04.344494+0000","last_undegraded":"2024-04-28T13:34:04.344494+0000","last_fullsized":"2024-04-28T13:34:04.344494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:48:58.152999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349347+0000","last_change":"2024-04-28T13:33:34.003448+0000","last_active":"2024-04-28T13:34:04.349347+0000","last_peered":"2024-04-28T13:34:04.349347+0000","last_clean":"2024-04-28T13:34:04.349347+0000","last_became_active":"2024-04-28T13:33:34.002948+0000","last_became_peered":"2024-04-28T13:33:34.002948+0000","last_unstale":"2024-04-28T13:34:04.349347+0000","last_undegraded":"2024-04-28T13:34:04.349347+0000","last_fullsized":"2024-04-28T13:34:04.349347+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:09:39.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.145422+0000","last_change":"2024-04-28T13:34:05.143133+0000","last_active":"2024-04-28T13:34:05.145422+0000","last_peered":"2024-04-28T13:34:05.145422+0000","last_clean":"2024-04-28T13:34:05.145422+0000","last_became_active":"2024-04-28T13:34:05.142351+0000","last_became_peered":"2024-04-28T13:34:05.142351+0000","last_unstale":"2024-04-28T13:34:05.145422+0000","last_undegraded":"2024-04-28T13:34:05.145422+0000","last_fullsized":"2024-04-28T13:34:05.145422+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:53:12.865627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.881028+0000","last_change":"2024-04-28T13:33:34.013366+0000","last_active":"2024-04-28T13:34:04.881028+0000","last_peered":"2024-04-28T13:34:04.881028+0000","last_clean":"2024-04-28T13:34:04.881028+0000","last_became_active":"2024-04-28T13:33:34.012900+0000","last_became_peered":"2024-04-28T13:33:34.012900+0000","last_unstale":"2024-04-28T13:34:04.881028+0000","last_undegraded":"2024-04-28T13:34:04.881028+0000","last_fullsized":"2024-04-28T13:34:04.881028+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:54.852573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880311+0000","last_change":"2024-04-28T13:33:34.007098+0000","last_active":"2024-04-28T13:34:04.880311+0000","last_peered":"2024-04-28T13:34:04.880311+0000","last_clean":"2024-04-28T13:34:04.880311+0000","last_became_active":"2024-04-28T13:33:34.005157+0000","last_became_peered":"2024-04-28T13:33:34.005157+0000","last_unstale":"2024-04-28T13:34:04.880311+0000","last_undegraded":"2024-04-28T13:34:04.880311+0000","last_fullsized":"2024-04-28T13:34:04.880311+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:09:39.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879686+0000","last_change":"2024-04-28T13:33:34.011118+0000","last_active":"2024-04-28T13:34:04.879686+0000","last_peered":"2024-04-28T13:34:04.879686+0000","last_clean":"2024-04-28T13:34:04.879686+0000","last_became_active":"2024-04-28T13:33:34.010811+0000","last_became_peered":"2024-04-28T13:33:34.010811+0000","last_unstale":"2024-04-28T13:34:04.879686+0000","last_undegraded":"2024-04-28T13:34:04.879686+0000","last_fullsized":"2024-04-28T13:34:04.879686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:01:13.143511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355402+0000","last_change":"2024-04-28T13:33:34.011900+0000","last_active":"2024-04-28T13:34:04.355402+0000","last_peered":"2024-04-28T13:34:04.355402+0000","last_clean":"2024-04-28T13:34:04.355402+0000","last_became_active":"2024-04-28T13:33:34.011580+0000","last_became_peered":"2024-04-28T13:33:34.011580+0000","last_unstale":"2024-04-28T13:34:04.355402+0000","last_undegraded":"2024-04-28T13:34:04.355402+0000","last_fullsized":"2024-04-28T13:34:04.355402+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.532062+0000","last_change":"2024-04-28T13:33:34.000874+0000","last_active":"2024-04-28T13:34:05.532062+0000","last_peered":"2024-04-28T13:34:05.532062+0000","last_clean":"2024-04-28T13:34:05.532062+0000","last_became_active":"2024-04-28T13:33:34.000568+0000","last_became_peered":"2024-04-28T13:33:34.000568+0000","last_unstale":"2024-04-28T13:34:05.532062+0000","last_undegraded":"2024-04-28T13:34:05.532062+0000","last_fullsized":"2024-04-28T13:34:05.532062+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:56:40.722825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.502702+0000","last_change":"2024-04-28T13:34:04.496689+0000","last_active":"2024-04-28T13:34:04.502702+0000","last_peered":"2024-04-28T13:34:04.502702+0000","last_clean":"2024-04-28T13:34:04.502702+0000","last_became_active":"2024-04-28T13:34:04.495748+0000","last_became_peered":"2024-04-28T13:34:04.495748+0000","last_unstale":"2024-04-28T13:34:04.502702+0000","last_undegraded":"2024-04-28T13:34:04.502702+0000","last_fullsized":"2024-04-28T13:34:04.502702+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:11.732507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344170+0000","last_change":"2024-04-28T13:33:33.998387+0000","last_active":"2024-04-28T13:34:04.344170+0000","last_peered":"2024-04-28T13:34:04.344170+0000","last_clean":"2024-04-28T13:34:04.344170+0000","last_became_active":"2024-04-28T13:33:33.998087+0000","last_became_peered":"2024-04-28T13:33:33.998087+0000","last_unstale":"2024-04-28T13:34:04.344170+0000","last_undegraded":"2024-04-28T13:34:04.344170+0000","last_fullsized":"2024-04-28T13:34:04.344170+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:52:24.477401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139537+0000","last_change":"2024-04-28T13:33:33.989154+0000","last_active":"2024-04-28T13:34:05.139537+0000","last_peered":"2024-04-28T13:34:05.139537+0000","last_clean":"2024-04-28T13:34:05.139537+0000","last_became_active":"2024-04-28T13:33:33.988827+0000","last_became_peered":"2024-04-28T13:33:33.988827+0000","last_unstale":"2024-04-28T13:34:05.139537+0000","last_undegraded":"2024-04-28T13:34:05.139537+0000","last_fullsized":"2024-04-28T13:34:05.139537+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:52:46.527475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346773+0000","last_change":"2024-04-28T13:33:34.008467+0000","last_active":"2024-04-28T13:34:04.346773+0000","last_peered":"2024-04-28T13:34:04.346773+0000","last_clean":"2024-04-28T13:34:04.346773+0000","last_became_active":"2024-04-28T13:33:34.008181+0000","last_became_peered":"2024-04-28T13:33:34.008181+0000","last_unstale":"2024-04-28T13:34:04.346773+0000","last_undegraded":"2024-04-28T13:34:04.346773+0000","last_fullsized":"2024-04-28T13:34:04.346773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347607+0000","last_change":"2024-04-28T13:33:34.007505+0000","last_active":"2024-04-28T13:34:04.347607+0000","last_peered":"2024-04-28T13:34:04.347607+0000","last_clean":"2024-04-28T13:34:04.347607+0000","last_became_active":"2024-04-28T13:33:34.006793+0000","last_became_peered":"2024-04-28T13:33:34.006793+0000","last_unstale":"2024-04-28T13:34:04.347607+0000","last_undegraded":"2024-04-28T13:34:04.347607+0000","last_fullsized":"2024-04-28T13:34:04.347607+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.140956+0000","last_change":"2024-04-28T13:33:34.009974+0000","last_active":"2024-04-28T13:34:05.140956+0000","last_peered":"2024-04-28T13:34:05.140956+0000","last_clean":"2024-04-28T13:34:05.140956+0000","last_became_active":"2024-04-28T13:33:34.009657+0000","last_became_peered":"2024-04-28T13:33:34.009657+0000","last_unstale":"2024-04-28T13:34:05.140956+0000","last_undegraded":"2024-04-28T13:34:05.140956+0000","last_fullsized":"2024-04-28T13:34:05.140956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:18:56.809253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531828+0000","last_change":"2024-04-28T13:33:34.004233+0000","last_active":"2024-04-28T13:34:05.531828+0000","last_peered":"2024-04-28T13:34:05.531828+0000","last_clean":"2024-04-28T13:34:05.531828+0000","last_became_active":"2024-04-28T13:33:33.999567+0000","last_became_peered":"2024-04-28T13:33:33.999567+0000","last_unstale":"2024-04-28T13:34:05.531828+0000","last_undegraded":"2024-04-28T13:34:05.531828+0000","last_fullsized":"2024-04-28T13:34:05.531828+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:36.248393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344842+0000","last_change":"2024-04-28T13:33:34.008932+0000","last_active":"2024-04-28T13:34:04.344842+0000","last_peered":"2024-04-28T13:34:04.344842+0000","last_clean":"2024-04-28T13:34:04.344842+0000","last_became_active":"2024-04-28T13:33:34.008652+0000","last_became_peered":"2024-04-28T13:33:34.008652+0000","last_unstale":"2024-04-28T13:34:04.344842+0000","last_undegraded":"2024-04-28T13:34:04.344842+0000","last_fullsized":"2024-04-28T13:34:04.344842+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:43:22.000390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880044+0000","last_change":"2024-04-28T13:33:34.006521+0000","last_active":"2024-04-28T13:34:04.880044+0000","last_peered":"2024-04-28T13:34:04.880044+0000","last_clean":"2024-04-28T13:34:04.880044+0000","last_became_active":"2024-04-28T13:33:34.006227+0000","last_became_peered":"2024-04-28T13:33:34.006227+0000","last_unstale":"2024-04-28T13:34:04.880044+0000","last_undegraded":"2024-04-28T13:34:04.880044+0000","last_fullsized":"2024-04-28T13:34:04.880044+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:46.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139737+0000","last_change":"2024-04-28T13:33:33.999169+0000","last_active":"2024-04-28T13:34:05.139737+0000","last_peered":"2024-04-28T13:34:05.139737+0000","last_clean":"2024-04-28T13:34:05.139737+0000","last_became_active":"2024-04-28T13:33:33.998179+0000","last_became_peered":"2024-04-28T13:33:33.998179+0000","last_unstale":"2024-04-28T13:34:05.139737+0000","last_undegraded":"2024-04-28T13:34:05.139737+0000","last_fullsized":"2024-04-28T13:34:05.139737+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:12.546153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139678+0000","last_change":"2024-04-28T13:33:33.998034+0000","last_active":"2024-04-28T13:34:05.139678+0000","last_peered":"2024-04-28T13:34:05.139678+0000","last_clean":"2024-04-28T13:34:05.139678+0000","last_became_active":"2024-04-28T13:33:33.997648+0000","last_became_peered":"2024-04-28T13:33:33.997648+0000","last_unstale":"2024-04-28T13:34:05.139678+0000","last_undegraded":"2024-04-28T13:34:05.139678+0000","last_fullsized":"2024-04-28T13:34:05.139678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:30.570296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.141197+0000","last_change":"2024-04-28T13:33:34.007275+0000","last_active":"2024-04-28T13:34:05.141197+0000","last_peered":"2024-04-28T13:34:05.141197+0000","last_clean":"2024-04-28T13:34:05.141197+0000","last_became_active":"2024-04-28T13:33:34.005017+0000","last_became_peered":"2024-04-28T13:33:34.005017+0000","last_unstale":"2024-04-28T13:34:05.141197+0000","last_undegraded":"2024-04-28T13:34:05.141197+0000","last_fullsized":"2024-04-28T13:34:05.141197+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:37:53.261649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531787+0000","last_change":"2024-04-28T13:33:33.995804+0000","last_active":"2024-04-28T13:34:05.531787+0000","last_peered":"2024-04-28T13:34:05.531787+0000","last_clean":"2024-04-28T13:34:05.531787+0000","last_became_active":"2024-04-28T13:33:33.995502+0000","last_became_peered":"2024-04-28T13:33:33.995502+0000","last_unstale":"2024-04-28T13:34:05.531787+0000","last_undegraded":"2024-04-28T13:34:05.531787+0000","last_fullsized":"2024-04-28T13:34:05.531787+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:27:53.804713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347538+0000","last_change":"2024-04-28T13:33:34.006447+0000","last_active":"2024-04-28T13:34:04.347538+0000","last_peered":"2024-04-28T13:34:04.347538+0000","last_clean":"2024-04-28T13:34:04.347538+0000","last_became_active":"2024-04-28T13:33:34.005656+0000","last_became_peered":"2024-04-28T13:33:34.005656+0000","last_unstale":"2024-04-28T13:34:04.347538+0000","last_undegraded":"2024-04-28T13:34:04.347538+0000","last_fullsized":"2024-04-28T13:34:04.347538+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355564+0000","last_change":"2024-04-28T13:33:34.005608+0000","last_active":"2024-04-28T13:34:04.355564+0000","last_peered":"2024-04-28T13:34:04.355564+0000","last_clean":"2024-04-28T13:34:04.355564+0000","last_became_active":"2024-04-28T13:33:34.005316+0000","last_became_peered":"2024-04-28T13:33:34.005316+0000","last_unstale":"2024-04-28T13:34:04.355564+0000","last_undegraded":"2024-04-28T13:34:04.355564+0000","last_fullsized":"2024-04-28T13:34:04.355564+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139766+0000","last_change":"2024-04-28T13:33:34.002648+0000","last_active":"2024-04-28T13:34:05.139766+0000","last_peered":"2024-04-28T13:34:05.139766+0000","last_clean":"2024-04-28T13:34:05.139766+0000","last_became_active":"2024-04-28T13:33:34.002375+0000","last_became_peered":"2024-04-28T13:33:34.002375+0000","last_unstale":"2024-04-28T13:34:05.139766+0000","last_undegraded":"2024-04-28T13:34:05.139766+0000","last_fullsized":"2024-04-28T13:34:05.139766+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:42.082730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346448+0000","last_change":"2024-04-28T13:33:33.999443+0000","last_active":"2024-04-28T13:34:04.346448+0000","last_peered":"2024-04-28T13:34:04.346448+0000","last_clean":"2024-04-28T13:34:04.346448+0000","last_became_active":"2024-04-28T13:33:33.998905+0000","last_became_peered":"2024-04-28T13:33:33.998905+0000","last_unstale":"2024-04-28T13:34:04.346448+0000","last_undegraded":"2024-04-28T13:34:04.346448+0000","last_fullsized":"2024-04-28T13:34:04.346448+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344364+0000","last_change":"2024-04-28T13:33:34.008881+0000","last_active":"2024-04-28T13:34:04.344364+0000","last_peered":"2024-04-28T13:34:04.344364+0000","last_clean":"2024-04-28T13:34:04.344364+0000","last_became_active":"2024-04-28T13:33:34.008297+0000","last_became_peered":"2024-04-28T13:33:34.008297+0000","last_unstale":"2024-04-28T13:34:04.344364+0000","last_undegraded":"2024-04-28T13:34:04.344364+0000","last_fullsized":"2024-04-28T13:34:04.344364+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:41:26.988112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531605+0000","last_change":"2024-04-28T13:33:34.003176+0000","last_active":"2024-04-28T13:34:05.531605+0000","last_peered":"2024-04-28T13:34:05.531605+0000","last_clean":"2024-04-28T13:34:05.531605+0000","last_became_active":"2024-04-28T13:33:34.002915+0000","last_became_peered":"2024-04-28T13:33:34.002915+0000","last_unstale":"2024-04-28T13:34:05.531605+0000","last_undegraded":"2024-04-28T13:34:05.531605+0000","last_fullsized":"2024-04-28T13:34:05.531605+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:52:06.849498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493215+0000","last_change":"2024-04-28T13:33:34.005669+0000","last_active":"2024-04-28T13:34:04.493215+0000","last_peered":"2024-04-28T13:34:04.493215+0000","last_clean":"2024-04-28T13:34:04.493215+0000","last_became_active":"2024-04-28T13:33:34.005365+0000","last_became_peered":"2024-04-28T13:33:34.005365+0000","last_unstale":"2024-04-28T13:34:04.493215+0000","last_undegraded":"2024-04-28T13:34:04.493215+0000","last_fullsized":"2024-04-28T13:34:04.493215+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:18:56.809253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":23,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345243+0000","last_change":"2024-04-28T13:33:32.998037+0000","last_active":"2024-04-28T13:34:04.345243+0000","last_peered":"2024-04-28T13:34:04.345243+0000","last_clean":"2024-04-28T13:34:04.345243+0000","last_became_active":"2024-04-28T13:33:32.997694+0000","last_became_peered":"2024-04-28T13:33:32.997694+0000","last_unstale":"2024-04-28T13:34:04.345243+0000","last_undegraded":"2024-04-28T13:34:04.345243+0000","last_fullsized":"2024-04-28T13:34:04.345243+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:02.432193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.141337+0000","last_change":"2024-04-28T13:33:33.989048+0000","last_active":"2024-04-28T13:34:05.141337+0000","last_peered":"2024-04-28T13:34:05.141337+0000","last_clean":"2024-04-28T13:34:05.141337+0000","last_became_active":"2024-04-28T13:33:33.988758+0000","last_became_peered":"2024-04-28T13:33:33.988758+0000","last_unstale":"2024-04-28T13:34:05.141337+0000","last_undegraded":"2024-04-28T13:34:05.141337+0000","last_fullsized":"2024-04-28T13:34:05.141337+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:14.152642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347981+0000","last_change":"2024-04-28T13:33:32.999239+0000","last_active":"2024-04-28T13:34:04.347981+0000","last_peered":"2024-04-28T13:34:04.347981+0000","last_clean":"2024-04-28T13:34:04.347981+0000","last_became_active":"2024-04-28T13:33:32.998813+0000","last_became_peered":"2024-04-28T13:33:32.998813+0000","last_unstale":"2024-04-28T13:34:04.347981+0000","last_undegraded":"2024-04-28T13:34:04.347981+0000","last_fullsized":"2024-04-28T13:34:04.347981+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:22:34.444155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355706+0000","last_change":"2024-04-28T13:33:34.003176+0000","last_active":"2024-04-28T13:34:04.355706+0000","last_peered":"2024-04-28T13:34:04.355706+0000","last_clean":"2024-04-28T13:34:04.355706+0000","last_became_active":"2024-04-28T13:33:34.002904+0000","last_became_peered":"2024-04-28T13:33:34.002904+0000","last_unstale":"2024-04-28T13:34:04.355706+0000","last_undegraded":"2024-04-28T13:34:04.355706+0000","last_fullsized":"2024-04-28T13:34:04.355706+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493775+0000","last_change":"2024-04-28T13:33:32.997729+0000","last_active":"2024-04-28T13:34:04.493775+0000","last_peered":"2024-04-28T13:34:04.493775+0000","last_clean":"2024-04-28T13:34:04.493775+0000","last_became_active":"2024-04-28T13:33:32.996881+0000","last_became_peered":"2024-04-28T13:33:32.996881+0000","last_unstale":"2024-04-28T13:34:04.493775+0000","last_undegraded":"2024-04-28T13:34:04.493775+0000","last_fullsized":"2024-04-28T13:34:04.493775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493141+0000","last_change":"2024-04-28T13:33:34.004339+0000","last_active":"2024-04-28T13:34:04.493141+0000","last_peered":"2024-04-28T13:34:04.493141+0000","last_clean":"2024-04-28T13:34:04.493141+0000","last_became_active":"2024-04-28T13:33:34.002361+0000","last_became_peered":"2024-04-28T13:33:34.002361+0000","last_unstale":"2024-04-28T13:34:04.493141+0000","last_undegraded":"2024-04-28T13:34:04.493141+0000","last_fullsized":"2024-04-28T13:34:04.493141+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:31.671210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350207+0000","last_change":"2024-04-28T13:33:33.000754+0000","last_active":"2024-04-28T13:34:04.350207+0000","last_peered":"2024-04-28T13:34:04.350207+0000","last_clean":"2024-04-28T13:34:04.350207+0000","last_became_active":"2024-04-28T13:33:33.000387+0000","last_became_peered":"2024-04-28T13:33:33.000387+0000","last_unstale":"2024-04-28T13:34:04.350207+0000","last_undegraded":"2024-04-28T13:34:04.350207+0000","last_fullsized":"2024-04-28T13:34:04.350207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:28.481114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346329+0000","last_change":"2024-04-28T13:33:33.994316+0000","last_active":"2024-04-28T13:34:04.346329+0000","last_peered":"2024-04-28T13:34:04.346329+0000","last_clean":"2024-04-28T13:34:04.346329+0000","last_became_active":"2024-04-28T13:33:33.994002+0000","last_became_peered":"2024-04-28T13:33:33.994002+0000","last_unstale":"2024-04-28T13:34:04.346329+0000","last_undegraded":"2024-04-28T13:34:04.346329+0000","last_fullsized":"2024-04-28T13:34:04.346329+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:03.446736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344611+0000","last_change":"2024-04-28T13:33:32.999540+0000","last_active":"2024-04-28T13:34:04.344611+0000","last_peered":"2024-04-28T13:34:04.344611+0000","last_clean":"2024-04-28T13:34:04.344611+0000","last_became_active":"2024-04-28T13:33:32.999147+0000","last_became_peered":"2024-04-28T13:33:32.999147+0000","last_unstale":"2024-04-28T13:34:04.344611+0000","last_undegraded":"2024-04-28T13:34:04.344611+0000","last_fullsized":"2024-04-28T13:34:04.344611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:45.657949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880509+0000","last_change":"2024-04-28T13:33:34.000424+0000","last_active":"2024-04-28T13:34:04.880509+0000","last_peered":"2024-04-28T13:34:04.880509+0000","last_clean":"2024-04-28T13:34:04.880509+0000","last_became_active":"2024-04-28T13:33:33.999414+0000","last_became_peered":"2024-04-28T13:33:33.999414+0000","last_unstale":"2024-04-28T13:34:04.880509+0000","last_undegraded":"2024-04-28T13:34:04.880509+0000","last_fullsized":"2024-04-28T13:34:04.880509+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346593+0000","last_change":"2024-04-28T13:33:32.988943+0000","last_active":"2024-04-28T13:34:04.346593+0000","last_peered":"2024-04-28T13:34:04.346593+0000","last_clean":"2024-04-28T13:34:04.346593+0000","last_became_active":"2024-04-28T13:33:32.986596+0000","last_became_peered":"2024-04-28T13:33:32.986596+0000","last_unstale":"2024-04-28T13:34:04.346593+0000","last_undegraded":"2024-04-28T13:34:04.346593+0000","last_fullsized":"2024-04-28T13:34:04.346593+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879866+0000","last_change":"2024-04-28T13:33:34.002579+0000","last_active":"2024-04-28T13:34:04.879866+0000","last_peered":"2024-04-28T13:34:04.879866+0000","last_clean":"2024-04-28T13:34:04.879866+0000","last_became_active":"2024-04-28T13:33:34.002254+0000","last_became_peered":"2024-04-28T13:33:34.002254+0000","last_unstale":"2024-04-28T13:34:04.879866+0000","last_undegraded":"2024-04-28T13:34:04.879866+0000","last_fullsized":"2024-04-28T13:34:04.879866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354254+0000","last_change":"2024-04-28T13:33:32.996052+0000","last_active":"2024-04-28T13:34:04.354254+0000","last_peered":"2024-04-28T13:34:04.354254+0000","last_clean":"2024-04-28T13:34:04.354254+0000","last_became_active":"2024-04-28T13:33:32.990258+0000","last_became_peered":"2024-04-28T13:33:32.990258+0000","last_unstale":"2024-04-28T13:34:04.354254+0000","last_undegraded":"2024-04-28T13:34:04.354254+0000","last_fullsized":"2024-04-28T13:34:04.354254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139482+0000","last_change":"2024-04-28T13:33:33.998180+0000","last_active":"2024-04-28T13:34:05.139482+0000","last_peered":"2024-04-28T13:34:05.139482+0000","last_clean":"2024-04-28T13:34:05.139482+0000","last_became_active":"2024-04-28T13:33:33.997730+0000","last_became_peered":"2024-04-28T13:33:33.997730+0000","last_unstale":"2024-04-28T13:34:05.139482+0000","last_undegraded":"2024-04-28T13:34:05.139482+0000","last_fullsized":"2024-04-28T13:34:05.139482+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:22:35.458698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880139+0000","last_change":"2024-04-28T13:33:32.989759+0000","last_active":"2024-04-28T13:34:04.880139+0000","last_peered":"2024-04-28T13:34:04.880139+0000","last_clean":"2024-04-28T13:34:04.880139+0000","last_became_active":"2024-04-28T13:33:32.988565+0000","last_became_peered":"2024-04-28T13:33:32.988565+0000","last_unstale":"2024-04-28T13:34:04.880139+0000","last_undegraded":"2024-04-28T13:34:04.880139+0000","last_fullsized":"2024-04-28T13:34:04.880139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:27:52.790170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347610+0000","last_change":"2024-04-28T13:33:34.007736+0000","last_active":"2024-04-28T13:34:04.347610+0000","last_peered":"2024-04-28T13:34:04.347610+0000","last_clean":"2024-04-28T13:34:04.347610+0000","last_became_active":"2024-04-28T13:33:34.007170+0000","last_became_peered":"2024-04-28T13:33:34.007170+0000","last_unstale":"2024-04-28T13:34:04.347610+0000","last_undegraded":"2024-04-28T13:34:04.347610+0000","last_fullsized":"2024-04-28T13:34:04.347610+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:01:33.789936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347247+0000","last_change":"2024-04-28T13:33:32.992483+0000","last_active":"2024-04-28T13:34:04.347247+0000","last_peered":"2024-04-28T13:34:04.347247+0000","last_clean":"2024-04-28T13:34:04.347247+0000","last_became_active":"2024-04-28T13:33:32.992216+0000","last_became_peered":"2024-04-28T13:33:32.992216+0000","last_unstale":"2024-04-28T13:34:04.347247+0000","last_undegraded":"2024-04-28T13:34:04.347247+0000","last_fullsized":"2024-04-28T13:34:04.347247+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:11.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350567+0000","last_change":"2024-04-28T13:33:34.009420+0000","last_active":"2024-04-28T13:34:04.350567+0000","last_peered":"2024-04-28T13:34:04.350567+0000","last_clean":"2024-04-28T13:34:04.350567+0000","last_became_active":"2024-04-28T13:33:34.009020+0000","last_became_peered":"2024-04-28T13:33:34.009020+0000","last_unstale":"2024-04-28T13:34:04.350567+0000","last_undegraded":"2024-04-28T13:34:04.350567+0000","last_fullsized":"2024-04-28T13:34:04.350567+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:48:58.152999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880919+0000","last_change":"2024-04-28T13:33:32.997498+0000","last_active":"2024-04-28T13:34:04.880919+0000","last_peered":"2024-04-28T13:34:04.880919+0000","last_clean":"2024-04-28T13:34:04.880919+0000","last_became_active":"2024-04-28T13:33:32.997237+0000","last_became_peered":"2024-04-28T13:33:32.997237+0000","last_unstale":"2024-04-28T13:34:04.880919+0000","last_undegraded":"2024-04-28T13:34:04.880919+0000","last_fullsized":"2024-04-28T13:34:04.880919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:37:18.628510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344809+0000","last_change":"2024-04-28T13:33:34.002865+0000","last_active":"2024-04-28T13:34:04.344809+0000","last_peered":"2024-04-28T13:34:04.344809+0000","last_clean":"2024-04-28T13:34:04.344809+0000","last_became_active":"2024-04-28T13:33:34.002421+0000","last_became_peered":"2024-04-28T13:33:34.002421+0000","last_unstale":"2024-04-28T13:34:04.344809+0000","last_undegraded":"2024-04-28T13:34:04.344809+0000","last_fullsized":"2024-04-28T13:34:04.344809+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:51:46.332983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350650+0000","last_change":"2024-04-28T13:33:33.004559+0000","last_active":"2024-04-28T13:34:04.350650+0000","last_peered":"2024-04-28T13:34:04.350650+0000","last_clean":"2024-04-28T13:34:04.350650+0000","last_became_active":"2024-04-28T13:33:33.004153+0000","last_became_peered":"2024-04-28T13:33:33.004153+0000","last_unstale":"2024-04-28T13:34:04.350650+0000","last_undegraded":"2024-04-28T13:34:04.350650+0000","last_fullsized":"2024-04-28T13:34:04.350650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:02.432193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350366+0000","last_change":"2024-04-28T13:33:34.006854+0000","last_active":"2024-04-28T13:34:04.350366+0000","last_peered":"2024-04-28T13:34:04.350366+0000","last_clean":"2024-04-28T13:34:04.350366+0000","last_became_active":"2024-04-28T13:33:34.006561+0000","last_became_peered":"2024-04-28T13:33:34.006561+0000","last_unstale":"2024-04-28T13:34:04.350366+0000","last_undegraded":"2024-04-28T13:34:04.350366+0000","last_fullsized":"2024-04-28T13:34:04.350366+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:54.852573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'12","reported_seq":30,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:47.492938+0000","last_change":"2024-04-28T13:33:32.991367+0000","last_active":"2024-04-28T13:35:47.492938+0000","last_peered":"2024-04-28T13:35:47.492938+0000","last_clean":"2024-04-28T13:35:47.492938+0000","last_became_active":"2024-04-28T13:33:32.991060+0000","last_became_peered":"2024-04-28T13:33:32.991060+0000","last_unstale":"2024-04-28T13:35:47.492938+0000","last_undegraded":"2024-04-28T13:35:47.492938+0000","last_fullsized":"2024-04-28T13:35:47.492938+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:00:16.144423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3397,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139850+0000","last_change":"2024-04-28T13:33:33.996658+0000","last_active":"2024-04-28T13:34:05.139850+0000","last_peered":"2024-04-28T13:34:05.139850+0000","last_clean":"2024-04-28T13:34:05.139850+0000","last_became_active":"2024-04-28T13:33:33.996260+0000","last_became_peered":"2024-04-28T13:33:33.996260+0000","last_unstale":"2024-04-28T13:34:05.139850+0000","last_undegraded":"2024-04-28T13:34:05.139850+0000","last_fullsized":"2024-04-28T13:34:05.139850+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:52.292838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345011+0000","last_change":"2024-04-28T13:33:32.990968+0000","last_active":"2024-04-28T13:34:04.345011+0000","last_peered":"2024-04-28T13:34:04.345011+0000","last_clean":"2024-04-28T13:34:04.345011+0000","last_became_active":"2024-04-28T13:33:32.990718+0000","last_became_peered":"2024-04-28T13:33:32.990718+0000","last_unstale":"2024-04-28T13:34:04.345011+0000","last_undegraded":"2024-04-28T13:34:04.345011+0000","last_fullsized":"2024-04-28T13:34:04.345011+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:03.358943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880822+0000","last_change":"2024-04-28T13:33:34.003961+0000","last_active":"2024-04-28T13:34:04.880822+0000","last_peered":"2024-04-28T13:34:04.880822+0000","last_clean":"2024-04-28T13:34:04.880822+0000","last_became_active":"2024-04-28T13:33:34.003599+0000","last_became_peered":"2024-04-28T13:33:34.003599+0000","last_unstale":"2024-04-28T13:34:04.880822+0000","last_undegraded":"2024-04-28T13:34:04.880822+0000","last_fullsized":"2024-04-28T13:34:04.880822+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:32:47.409486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349522+0000","last_change":"2024-04-28T13:33:33.000521+0000","last_active":"2024-04-28T13:34:04.349522+0000","last_peered":"2024-04-28T13:34:04.349522+0000","last_clean":"2024-04-28T13:34:04.349522+0000","last_became_active":"2024-04-28T13:33:32.998867+0000","last_became_peered":"2024-04-28T13:33:32.998867+0000","last_unstale":"2024-04-28T13:34:04.349522+0000","last_undegraded":"2024-04-28T13:34:04.349522+0000","last_fullsized":"2024-04-28T13:34:04.349522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:43.562896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344089+0000","last_change":"2024-04-28T13:33:34.006368+0000","last_active":"2024-04-28T13:34:04.344089+0000","last_peered":"2024-04-28T13:34:04.344089+0000","last_clean":"2024-04-28T13:34:04.344089+0000","last_became_active":"2024-04-28T13:33:34.002579+0000","last_became_peered":"2024-04-28T13:33:34.002579+0000","last_unstale":"2024-04-28T13:34:04.344089+0000","last_undegraded":"2024-04-28T13:34:04.344089+0000","last_fullsized":"2024-04-28T13:34:04.344089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:27:51.330692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345395+0000","last_change":"2024-04-28T13:33:32.991648+0000","last_active":"2024-04-28T13:34:04.345395+0000","last_peered":"2024-04-28T13:34:04.345395+0000","last_clean":"2024-04-28T13:34:04.345395+0000","last_became_active":"2024-04-28T13:33:32.991369+0000","last_became_peered":"2024-04-28T13:33:32.991369+0000","last_unstale":"2024-04-28T13:34:04.345395+0000","last_undegraded":"2024-04-28T13:34:04.345395+0000","last_fullsized":"2024-04-28T13:34:04.345395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:43.562896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349607+0000","last_change":"2024-04-28T13:33:34.010407+0000","last_active":"2024-04-28T13:34:04.349607+0000","last_peered":"2024-04-28T13:34:04.349607+0000","last_clean":"2024-04-28T13:34:04.349607+0000","last_became_active":"2024-04-28T13:33:34.010077+0000","last_became_peered":"2024-04-28T13:33:34.010077+0000","last_unstale":"2024-04-28T13:34:04.349607+0000","last_undegraded":"2024-04-28T13:34:04.349607+0000","last_fullsized":"2024-04-28T13:34:04.349607+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:20:07.691466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880423+0000","last_change":"2024-04-28T13:33:32.989587+0000","last_active":"2024-04-28T13:34:04.880423+0000","last_peered":"2024-04-28T13:34:04.880423+0000","last_clean":"2024-04-28T13:34:04.880423+0000","last_became_active":"2024-04-28T13:33:32.988579+0000","last_became_peered":"2024-04-28T13:33:32.988579+0000","last_unstale":"2024-04-28T13:34:04.880423+0000","last_undegraded":"2024-04-28T13:34:04.880423+0000","last_fullsized":"2024-04-28T13:34:04.880423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:35.233850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.532050+0000","last_change":"2024-04-28T13:33:34.012322+0000","last_active":"2024-04-28T13:34:05.532050+0000","last_peered":"2024-04-28T13:34:05.532050+0000","last_clean":"2024-04-28T13:34:05.532050+0000","last_became_active":"2024-04-28T13:33:34.012112+0000","last_became_peered":"2024-04-28T13:33:34.012112+0000","last_unstale":"2024-04-28T13:34:05.532050+0000","last_undegraded":"2024-04-28T13:34:05.532050+0000","last_fullsized":"2024-04-28T13:34:05.532050+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:10.015862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531304+0000","last_change":"2024-04-28T13:33:32.990958+0000","last_active":"2024-04-28T13:34:05.531304+0000","last_peered":"2024-04-28T13:34:05.531304+0000","last_clean":"2024-04-28T13:34:05.531304+0000","last_became_active":"2024-04-28T13:33:32.990659+0000","last_became_peered":"2024-04-28T13:33:32.990659+0000","last_unstale":"2024-04-28T13:34:05.531304+0000","last_undegraded":"2024-04-28T13:34:05.531304+0000","last_fullsized":"2024-04-28T13:34:05.531304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:24:34.028457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.881209+0000","last_change":"2024-04-28T13:33:34.005832+0000","last_active":"2024-04-28T13:34:04.881209+0000","last_peered":"2024-04-28T13:34:04.881209+0000","last_clean":"2024-04-28T13:34:04.881209+0000","last_became_active":"2024-04-28T13:33:34.005364+0000","last_became_peered":"2024-04-28T13:33:34.005364+0000","last_unstale":"2024-04-28T13:34:04.881209+0000","last_undegraded":"2024-04-28T13:34:04.881209+0000","last_fullsized":"2024-04-28T13:34:04.881209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:28.548233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.348084+0000","last_change":"2024-04-28T13:33:32.995338+0000","last_active":"2024-04-28T13:34:04.348084+0000","last_peered":"2024-04-28T13:34:04.348084+0000","last_clean":"2024-04-28T13:34:04.348084+0000","last_became_active":"2024-04-28T13:33:32.994798+0000","last_became_peered":"2024-04-28T13:33:32.994798+0000","last_unstale":"2024-04-28T13:34:04.348084+0000","last_undegraded":"2024-04-28T13:34:04.348084+0000","last_fullsized":"2024-04-28T13:34:04.348084+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:29.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.140801+0000","last_change":"2024-04-28T13:33:34.002242+0000","last_active":"2024-04-28T13:34:05.140801+0000","last_peered":"2024-04-28T13:34:05.140801+0000","last_clean":"2024-04-28T13:34:05.140801+0000","last_became_active":"2024-04-28T13:33:34.001893+0000","last_became_peered":"2024-04-28T13:33:34.001893+0000","last_unstale":"2024-04-28T13:34:05.140801+0000","last_undegraded":"2024-04-28T13:34:05.140801+0000","last_fullsized":"2024-04-28T13:34:05.140801+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:35.969568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355718+0000","last_change":"2024-04-28T13:33:32.991441+0000","last_active":"2024-04-28T13:34:04.355718+0000","last_peered":"2024-04-28T13:34:04.355718+0000","last_clean":"2024-04-28T13:34:04.355718+0000","last_became_active":"2024-04-28T13:33:32.990745+0000","last_became_peered":"2024-04-28T13:33:32.990745+0000","last_unstale":"2024-04-28T13:34:04.355718+0000","last_undegraded":"2024-04-28T13:34:04.355718+0000","last_fullsized":"2024-04-28T13:34:04.355718+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:30.656667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.140616+0000","last_change":"2024-04-28T13:33:34.003094+0000","last_active":"2024-04-28T13:34:05.140616+0000","last_peered":"2024-04-28T13:34:05.140616+0000","last_clean":"2024-04-28T13:34:05.140616+0000","last_became_active":"2024-04-28T13:33:34.002228+0000","last_became_peered":"2024-04-28T13:33:34.002228+0000","last_unstale":"2024-04-28T13:34:05.140616+0000","last_undegraded":"2024-04-28T13:34:05.140616+0000","last_fullsized":"2024-04-28T13:34:05.140616+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:31.671210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344933+0000","last_change":"2024-04-28T13:33:33.003720+0000","last_active":"2024-04-28T13:34:04.344933+0000","last_peered":"2024-04-28T13:34:04.344933+0000","last_clean":"2024-04-28T13:34:04.344933+0000","last_became_active":"2024-04-28T13:33:33.003389+0000","last_became_peered":"2024-04-28T13:33:33.003389+0000","last_unstale":"2024-04-28T13:34:04.344933+0000","last_undegraded":"2024-04-28T13:34:04.344933+0000","last_fullsized":"2024-04-28T13:34:04.344933+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:18.814381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345100+0000","last_change":"2024-04-28T13:33:34.006579+0000","last_active":"2024-04-28T13:34:04.345100+0000","last_peered":"2024-04-28T13:34:04.345100+0000","last_clean":"2024-04-28T13:34:04.345100+0000","last_became_active":"2024-04-28T13:33:34.006282+0000","last_became_peered":"2024-04-28T13:33:34.006282+0000","last_unstale":"2024-04-28T13:34:04.345100+0000","last_undegraded":"2024-04-28T13:34:04.345100+0000","last_fullsized":"2024-04-28T13:34:04.345100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:18:15.243319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345377+0000","last_change":"2024-04-28T13:33:32.999347+0000","last_active":"2024-04-28T13:34:04.345377+0000","last_peered":"2024-04-28T13:34:04.345377+0000","last_clean":"2024-04-28T13:34:04.345377+0000","last_became_active":"2024-04-28T13:33:32.998923+0000","last_became_peered":"2024-04-28T13:33:32.998923+0000","last_unstale":"2024-04-28T13:34:04.345377+0000","last_undegraded":"2024-04-28T13:34:04.345377+0000","last_fullsized":"2024-04-28T13:34:04.345377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:46.522691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.492925+0000","last_change":"2024-04-28T13:33:34.011875+0000","last_active":"2024-04-28T13:34:04.492925+0000","last_peered":"2024-04-28T13:34:04.492925+0000","last_clean":"2024-04-28T13:34:04.492925+0000","last_became_active":"2024-04-28T13:33:34.011271+0000","last_became_peered":"2024-04-28T13:33:34.011271+0000","last_unstale":"2024-04-28T13:34:04.492925+0000","last_undegraded":"2024-04-28T13:34:04.492925+0000","last_fullsized":"2024-04-28T13:34:04.492925+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:35.969568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354592+0000","last_change":"2024-04-28T13:33:33.002216+0000","last_active":"2024-04-28T13:34:04.354592+0000","last_peered":"2024-04-28T13:34:04.354592+0000","last_clean":"2024-04-28T13:34:04.354592+0000","last_became_active":"2024-04-28T13:33:33.000335+0000","last_became_peered":"2024-04-28T13:33:33.000335+0000","last_unstale":"2024-04-28T13:34:04.354592+0000","last_undegraded":"2024-04-28T13:34:04.354592+0000","last_fullsized":"2024-04-28T13:34:04.354592+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:34.955026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531906+0000","last_change":"2024-04-28T13:33:34.000104+0000","last_active":"2024-04-28T13:34:05.531906+0000","last_peered":"2024-04-28T13:34:05.531906+0000","last_clean":"2024-04-28T13:34:05.531906+0000","last_became_active":"2024-04-28T13:33:33.999839+0000","last_became_peered":"2024-04-28T13:33:33.999839+0000","last_unstale":"2024-04-28T13:34:05.531906+0000","last_undegraded":"2024-04-28T13:34:05.531906+0000","last_fullsized":"2024-04-28T13:34:05.531906+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:03:14.657009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879925+0000","last_change":"2024-04-28T13:33:32.992149+0000","last_active":"2024-04-28T13:34:04.879925+0000","last_peered":"2024-04-28T13:34:04.879925+0000","last_clean":"2024-04-28T13:34:04.879925+0000","last_became_active":"2024-04-28T13:33:32.991800+0000","last_became_peered":"2024-04-28T13:33:32.991800+0000","last_unstale":"2024-04-28T13:34:04.879925+0000","last_undegraded":"2024-04-28T13:34:04.879925+0000","last_fullsized":"2024-04-28T13:34:04.879925+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:56:39.708282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879801+0000","last_change":"2024-04-28T13:33:34.011133+0000","last_active":"2024-04-28T13:34:04.879801+0000","last_peered":"2024-04-28T13:34:04.879801+0000","last_clean":"2024-04-28T13:34:04.879801+0000","last_became_active":"2024-04-28T13:33:34.010896+0000","last_became_peered":"2024-04-28T13:33:34.010896+0000","last_unstale":"2024-04-28T13:34:04.879801+0000","last_undegraded":"2024-04-28T13:34:04.879801+0000","last_fullsized":"2024-04-28T13:34:04.879801+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.348967+0000","last_change":"2024-04-28T13:33:33.000859+0000","last_active":"2024-04-28T13:34:04.348967+0000","last_peered":"2024-04-28T13:34:04.348967+0000","last_clean":"2024-04-28T13:34:04.348967+0000","last_became_active":"2024-04-28T13:33:33.000095+0000","last_became_peered":"2024-04-28T13:33:33.000095+0000","last_unstale":"2024-04-28T13:34:04.348967+0000","last_undegraded":"2024-04-28T13:34:04.348967+0000","last_fullsized":"2024-04-28T13:34:04.348967+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:18.221214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.141479+0000","last_change":"2024-04-28T13:33:34.008405+0000","last_active":"2024-04-28T13:34:05.141479+0000","last_peered":"2024-04-28T13:34:05.141479+0000","last_clean":"2024-04-28T13:34:05.141479+0000","last_became_active":"2024-04-28T13:33:34.007425+0000","last_became_peered":"2024-04-28T13:33:34.007425+0000","last_unstale":"2024-04-28T13:34:05.141479+0000","last_undegraded":"2024-04-28T13:34:05.141479+0000","last_fullsized":"2024-04-28T13:34:05.141479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:29.495657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"47'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:45.005269+0000","last_change":"2024-04-28T13:33:32.998594+0000","last_active":"2024-04-28T13:34:45.005269+0000","last_peered":"2024-04-28T13:34:45.005269+0000","last_clean":"2024-04-28T13:34:45.005269+0000","last_became_active":"2024-04-28T13:33:32.998328+0000","last_became_peered":"2024-04-28T13:33:32.998328+0000","last_unstale":"2024-04-28T13:34:45.005269+0000","last_undegraded":"2024-04-28T13:34:45.005269+0000","last_fullsized":"2024-04-28T13:34:45.005269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:18.221214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880058+0000","last_change":"2024-04-28T13:33:33.998134+0000","last_active":"2024-04-28T13:34:04.880058+0000","last_peered":"2024-04-28T13:34:04.880058+0000","last_clean":"2024-04-28T13:34:04.880058+0000","last_became_active":"2024-04-28T13:33:33.997834+0000","last_became_peered":"2024-04-28T13:33:33.997834+0000","last_unstale":"2024-04-28T13:34:04.880058+0000","last_undegraded":"2024-04-28T13:34:04.880058+0000","last_fullsized":"2024-04-28T13:34:04.880058+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:01:33.789936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350856+0000","last_change":"2024-04-28T13:33:33.005737+0000","last_active":"2024-04-28T13:34:04.350856+0000","last_peered":"2024-04-28T13:34:04.350856+0000","last_clean":"2024-04-28T13:34:04.350856+0000","last_became_active":"2024-04-28T13:33:33.005179+0000","last_became_peered":"2024-04-28T13:33:33.005179+0000","last_unstale":"2024-04-28T13:34:04.350856+0000","last_undegraded":"2024-04-28T13:34:04.350856+0000","last_fullsized":"2024-04-28T13:34:04.350856+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:45.657949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350592+0000","last_change":"2024-04-28T13:33:34.011725+0000","last_active":"2024-04-28T13:34:04.350592+0000","last_peered":"2024-04-28T13:34:04.350592+0000","last_clean":"2024-04-28T13:34:04.350592+0000","last_became_active":"2024-04-28T13:33:34.011422+0000","last_became_peered":"2024-04-28T13:33:34.011422+0000","last_unstale":"2024-04-28T13:34:04.350592+0000","last_undegraded":"2024-04-28T13:34:04.350592+0000","last_fullsized":"2024-04-28T13:34:04.350592+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:28.548233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.492969+0000","last_change":"2024-04-28T13:33:32.997338+0000","last_active":"2024-04-28T13:34:04.492969+0000","last_peered":"2024-04-28T13:34:04.492969+0000","last_clean":"2024-04-28T13:34:04.492969+0000","last_became_active":"2024-04-28T13:33:32.996934+0000","last_became_peered":"2024-04-28T13:33:32.996934+0000","last_unstale":"2024-04-28T13:34:04.492969+0000","last_undegraded":"2024-04-28T13:34:04.492969+0000","last_fullsized":"2024-04-28T13:34:04.492969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:29.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345593+0000","last_change":"2024-04-28T13:33:34.020082+0000","last_active":"2024-04-28T13:34:04.345593+0000","last_peered":"2024-04-28T13:34:04.345593+0000","last_clean":"2024-04-28T13:34:04.345593+0000","last_became_active":"2024-04-28T13:33:34.018922+0000","last_became_peered":"2024-04-28T13:33:34.018922+0000","last_unstale":"2024-04-28T13:34:04.345593+0000","last_undegraded":"2024-04-28T13:34:04.345593+0000","last_fullsized":"2024-04-28T13:34:04.345593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:53:12.865627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350434+0000","last_change":"2024-04-28T13:33:33.000626+0000","last_active":"2024-04-28T13:34:04.350434+0000","last_peered":"2024-04-28T13:34:04.350434+0000","last_clean":"2024-04-28T13:34:04.350434+0000","last_became_active":"2024-04-28T13:33:33.000049+0000","last_became_peered":"2024-04-28T13:33:33.000049+0000","last_unstale":"2024-04-28T13:34:04.350434+0000","last_undegraded":"2024-04-28T13:34:04.350434+0000","last_fullsized":"2024-04-28T13:34:04.350434+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:18.814381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344459+0000","last_change":"2024-04-28T13:33:34.004230+0000","last_active":"2024-04-28T13:34:04.344459+0000","last_peered":"2024-04-28T13:34:04.344459+0000","last_clean":"2024-04-28T13:34:04.344459+0000","last_became_active":"2024-04-28T13:33:34.003940+0000","last_became_peered":"2024-04-28T13:33:34.003940+0000","last_unstale":"2024-04-28T13:34:04.344459+0000","last_undegraded":"2024-04-28T13:34:04.344459+0000","last_fullsized":"2024-04-28T13:34:04.344459+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:11.732507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346733+0000","last_change":"2024-04-28T13:33:32.991869+0000","last_active":"2024-04-28T13:34:04.346733+0000","last_peered":"2024-04-28T13:34:04.346733+0000","last_clean":"2024-04-28T13:34:04.346733+0000","last_became_active":"2024-04-28T13:33:32.991435+0000","last_became_peered":"2024-04-28T13:33:32.991435+0000","last_unstale":"2024-04-28T13:34:04.346733+0000","last_undegraded":"2024-04-28T13:34:04.346733+0000","last_fullsized":"2024-04-28T13:34:04.346733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:41.068187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354423+0000","last_change":"2024-04-28T13:33:34.004178+0000","last_active":"2024-04-28T13:34:04.354423+0000","last_peered":"2024-04-28T13:34:04.354423+0000","last_clean":"2024-04-28T13:34:04.354423+0000","last_became_active":"2024-04-28T13:33:34.002092+0000","last_became_peered":"2024-04-28T13:33:34.002092+0000","last_unstale":"2024-04-28T13:34:04.354423+0000","last_undegraded":"2024-04-28T13:34:04.354423+0000","last_fullsized":"2024-04-28T13:34:04.354423+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:03.446736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493055+0000","last_change":"2024-04-28T13:33:33.003796+0000","last_active":"2024-04-28T13:34:04.493055+0000","last_peered":"2024-04-28T13:34:04.493055+0000","last_clean":"2024-04-28T13:34:04.493055+0000","last_became_active":"2024-04-28T13:33:33.003374+0000","last_became_peered":"2024-04-28T13:33:33.003374+0000","last_unstale":"2024-04-28T13:34:04.493055+0000","last_undegraded":"2024-04-28T13:34:04.493055+0000","last_fullsized":"2024-04-28T13:34:04.493055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:11.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347052+0000","last_change":"2024-04-28T13:33:34.001551+0000","last_active":"2024-04-28T13:34:04.347052+0000","last_peered":"2024-04-28T13:34:04.347052+0000","last_clean":"2024-04-28T13:34:04.347052+0000","last_became_active":"2024-04-28T13:33:33.999963+0000","last_became_peered":"2024-04-28T13:33:33.999963+0000","last_unstale":"2024-04-28T13:34:04.347052+0000","last_undegraded":"2024-04-28T13:34:04.347052+0000","last_fullsized":"2024-04-28T13:34:04.347052+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:19.828924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880581+0000","last_change":"2024-04-28T13:33:32.991033+0000","last_active":"2024-04-28T13:34:04.880581+0000","last_peered":"2024-04-28T13:34:04.880581+0000","last_clean":"2024-04-28T13:34:04.880581+0000","last_became_active":"2024-04-28T13:33:32.990726+0000","last_became_peered":"2024-04-28T13:33:32.990726+0000","last_unstale":"2024-04-28T13:34:04.880581+0000","last_undegraded":"2024-04-28T13:34:04.880581+0000","last_fullsized":"2024-04-28T13:34:04.880581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:09.001319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880728+0000","last_change":"2024-04-28T13:33:34.005421+0000","last_active":"2024-04-28T13:34:04.880728+0000","last_peered":"2024-04-28T13:34:04.880728+0000","last_clean":"2024-04-28T13:34:04.880728+0000","last_became_active":"2024-04-28T13:33:34.004776+0000","last_became_peered":"2024-04-28T13:33:34.004776+0000","last_unstale":"2024-04-28T13:34:04.880728+0000","last_undegraded":"2024-04-28T13:34:04.880728+0000","last_fullsized":"2024-04-28T13:34:04.880728+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:20:07.691466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.356034+0000","last_change":"2024-04-28T13:33:32.991462+0000","last_active":"2024-04-28T13:34:04.356034+0000","last_peered":"2024-04-28T13:34:04.356034+0000","last_clean":"2024-04-28T13:34:04.356034+0000","last_became_active":"2024-04-28T13:33:32.991149+0000","last_became_peered":"2024-04-28T13:33:32.991149+0000","last_unstale":"2024-04-28T13:34:04.356034+0000","last_undegraded":"2024-04-28T13:34:04.356034+0000","last_fullsized":"2024-04-28T13:34:04.356034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:41.068187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":220,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349073+0000","last_change":"2024-04-28T13:31:11.831538+0000","last_active":"2024-04-28T13:34:04.349073+0000","last_peered":"2024-04-28T13:34:04.349073+0000","last_clean":"2024-04-28T13:34:04.349073+0000","last_became_active":"2024-04-28T13:31:11.828959+0000","last_became_peered":"2024-04-28T13:31:11.828959+0000","last_unstale":"2024-04-28T13:34:04.349073+0000","last_undegraded":"2024-04-28T13:34:04.349073+0000","last_fullsized":"2024-04-28T13:34:04.349073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_clean_scrub_stamp":"2024-04-28T13:31:10.444388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:25:27.078911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":4195,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343692,"statfs":{"total":96636764160,"available":96607940608,"internally_reserved":0,"allocated":1150976,"data_stored":818777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":561152,"data_stored":228991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":228389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":565248,"data_stored":231575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":565248,"data_stored":228957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":569344,"data_stored":231597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":228389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1150976,"data_stored":818721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T13:35:52.179 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-28T13:35:52.179 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T13:35:52.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.411+0000 7fa336400640 1 Processor -- start 2024-04-28T13:35:52.411 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.411+0000 7fa336400640 1 -- start start 2024-04-28T13:35:52.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.413+0000 7fa336400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa33015b6f0 0x7fa33006f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:52.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.413+0000 7fa336400640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa33006e330 0x7fa330072930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:52.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.413+0000 7fa336400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3300cefb0 con 0x7fa330072e70 2024-04-28T13:35:52.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.413+0000 7fa336400640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa3300b6bc0 con 0x7fa33006e330 2024-04-28T13:35:52.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.413+0000 7fa336400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa33015bad0 con 0x7fa33015b6f0 2024-04-28T13:35:52.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.414+0000 7fa32ea00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa33006e330 0x7fa330072930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:52.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.414+0000 7fa32f400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa33015b6f0 0x7fa33006f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:52.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.414+0000 7fa32ea00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa33006e330 0x7fa330072930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:54946/0 (socket says 172.21.15.196:54946) 2024-04-28T13:35:52.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.414+0000 7fa32ea00640 1 -- 172.21.15.196:0/2904735436 learned_addr learned my addr 172.21.15.196:0/2904735436 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:52.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.415+0000 7fa32ea00640 1 -- 172.21.15.196:0/2904735436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa33015b6f0 msgr2=0x7fa33006f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:52.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.415+0000 7fa32ea00640 1 --2- 172.21.15.196:0/2904735436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa33015b6f0 0x7fa33006f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.415+0000 7fa32ea00640 1 -- 172.21.15.196:0/2904735436 >> v1:172.21.15.203:6789/0 conn(0x7fa330072e70 legacy=0x7fa33015f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:52.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.415+0000 7fa32ea00640 1 -- 172.21.15.196:0/2904735436 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33006f8e0 con 0x7fa33006e330 2024-04-28T13:35:52.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.416+0000 7fa32ea00640 1 --2- 172.21.15.196:0/2904735436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa33006e330 0x7fa330072930 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00aa40 tx=0x7fa31c02b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a52134fdb9c443f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:52.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.417+0000 7fa32e000640 1 -- 172.21.15.196:0/2904735436 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c00b040 con 0x7fa33006e330 2024-04-28T13:35:52.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.417+0000 7fa32e000640 1 -- 172.21.15.196:0/2904735436 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa31c030710 con 0x7fa33006e330 2024-04-28T13:35:52.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.417+0000 7fa32e000640 1 -- 172.21.15.196:0/2904735436 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c030ae0 con 0x7fa33006e330 2024-04-28T13:35:52.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.417+0000 7fa336400640 1 -- 172.21.15.196:0/2904735436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa33006e330 msgr2=0x7fa330072930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:52.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.417+0000 7fa336400640 1 --2- 172.21.15.196:0/2904735436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa33006e330 0x7fa330072930 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00aa40 tx=0x7fa31c02b650 comp rx=0 tx=0).stop 2024-04-28T13:35:52.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.417+0000 7fa336400640 1 -- 172.21.15.196:0/2904735436 shutdown_connections 2024-04-28T13:35:52.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.417+0000 7fa336400640 1 --2- 172.21.15.196:0/2904735436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa33006e330 0x7fa330072930 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.417+0000 7fa336400640 1 --2- 172.21.15.196:0/2904735436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa33015b6f0 0x7fa33006f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.417+0000 7fa336400640 1 -- 172.21.15.196:0/2904735436 >> 172.21.15.196:0/2904735436 conn(0x7fa3300cd830 msgr2=0x7fa3300cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:52.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.418+0000 7fa336400640 1 -- 172.21.15.196:0/2904735436 shutdown_connections 2024-04-28T13:35:52.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.418+0000 7fa336400640 1 -- 172.21.15.196:0/2904735436 wait complete. 2024-04-28T13:35:52.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.419+0000 7fa336400640 1 Processor -- start 2024-04-28T13:35:52.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.419+0000 7fa336400640 1 -- start start 2024-04-28T13:35:52.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.420+0000 7fa336400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa33006e330 0x7fa330147440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:52.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.420+0000 7fa336400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa330072e70 0x7fa3300b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:52.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.420+0000 7fa336400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa33015b6f0 0x7fa3300b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:52.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.420+0000 7fa336400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3300cf550 con 0x7fa33015b6f0 2024-04-28T13:35:52.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.420+0000 7fa336400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa3300cf270 con 0x7fa33006e330 2024-04-28T13:35:52.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.420+0000 7fa336400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa3300d0a80 con 0x7fa330072e70 2024-04-28T13:35:52.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.421+0000 7fa32fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa33015b6f0 0x7fa3300b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:52.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.421+0000 7fa32fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa33015b6f0 0x7fa3300b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:38948/0 (socket says 172.21.15.196:38948) 2024-04-28T13:35:52.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.421+0000 7fa32fe00640 1 -- 172.21.15.196:0/1023229550 learned_addr learned my addr 172.21.15.196:0/1023229550 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:52.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.421+0000 7fa32ea00640 1 --2- 172.21.15.196:0/1023229550 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa330072e70 0x7fa3300b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:52.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.421+0000 7fa32f400640 1 --2- 172.21.15.196:0/1023229550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa33006e330 0x7fa330147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:52.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.421+0000 7fa32fe00640 1 -- 172.21.15.196:0/1023229550 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa330072e70 msgr2=0x7fa3300b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:52.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.421+0000 7fa32fe00640 1 --2- 172.21.15.196:0/1023229550 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa330072e70 0x7fa3300b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.421+0000 7fa32fe00640 1 -- 172.21.15.196:0/1023229550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa33006e330 msgr2=0x7fa330147440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:52.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.421+0000 7fa32fe00640 1 --2- 172.21.15.196:0/1023229550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa33006e330 0x7fa330147440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.421+0000 7fa32fe00640 1 -- 172.21.15.196:0/1023229550 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33015e930 con 0x7fa33015b6f0 2024-04-28T13:35:52.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.421+0000 7fa32f400640 1 --2- 172.21.15.196:0/1023229550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa33006e330 0x7fa330147440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T13:35:52.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.422+0000 7fa32fe00640 1 --2- 172.21.15.196:0/1023229550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa33015b6f0 0x7fa3300b2580 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa32800c730 tx=0x7fa32800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:52.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.422+0000 7fa327400640 1 -- 172.21.15.196:0/1023229550 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328003b70 con 0x7fa33015b6f0 2024-04-28T13:35:52.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.422+0000 7fa327400640 1 -- 172.21.15.196:0/1023229550 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa328003d30 con 0x7fa33015b6f0 2024-04-28T13:35:52.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.422+0000 7fa327400640 1 -- 172.21.15.196:0/1023229550 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328005310 con 0x7fa33015b6f0 2024-04-28T13:35:52.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.422+0000 7fa336400640 1 -- 172.21.15.196:0/1023229550 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33015ec10 con 0x7fa33015b6f0 2024-04-28T13:35:52.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.422+0000 7fa336400640 1 -- 172.21.15.196:0/1023229550 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa330141a10 con 0x7fa33015b6f0 2024-04-28T13:35:52.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.423+0000 7fa336400640 1 -- 172.21.15.196:0/1023229550 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33006a290 con 0x7fa33015b6f0 2024-04-28T13:35:52.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.425+0000 7fa327400640 1 -- 172.21.15.196:0/1023229550 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa3280056c0 con 0x7fa33015b6f0 2024-04-28T13:35:52.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.425+0000 7fa327400640 1 --2- 172.21.15.196:0/1023229550 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa31006c980 0x7fa31006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:52.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.426+0000 7fa327400640 1 -- 172.21.15.196:0/1023229550 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fa32808bba0 con 0x7fa33015b6f0 2024-04-28T13:35:52.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.426+0000 7fa32f400640 1 --2- 172.21.15.196:0/1023229550 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa31006c980 0x7fa31006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:52.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.427+0000 7fa32f400640 1 --2- 172.21.15.196:0/1023229550 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa31006c980 0x7fa31006ee40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa320009af0 tx=0x7fa320008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:52.430 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.430+0000 7fa327400640 1 -- 172.21.15.196:0/1023229550 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa32805a030 con 0x7fa33015b6f0 2024-04-28T13:35:52.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.626+0000 7fa336400640 1 -- 172.21.15.196:0/1023229550 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3301416c0 con 0x7fa31006c980 2024-04-28T13:35:52.644 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.644+0000 7fa327400640 1 -- 172.21.15.196:0/1023229550 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251882 (secure 0 0 0) 0x7fa3301416c0 con 0x7fa31006c980 2024-04-28T13:35:52.647 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:35:52.649 INFO:teuthology.orchestra.run.smithi196.stderr:dumped all 2024-04-28T13:35:52.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.649+0000 7fa336400640 1 -- 172.21.15.196:0/1023229550 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa31006c980 msgr2=0x7fa31006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:52.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.649+0000 7fa336400640 1 --2- 172.21.15.196:0/1023229550 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa31006c980 0x7fa31006ee40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa320009af0 tx=0x7fa320008040 comp rx=0 tx=0).stop 2024-04-28T13:35:52.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.649+0000 7fa336400640 1 -- 172.21.15.196:0/1023229550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa33015b6f0 msgr2=0x7fa3300b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:52.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.649+0000 7fa336400640 1 --2- 172.21.15.196:0/1023229550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa33015b6f0 0x7fa3300b2580 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa32800c730 tx=0x7fa32800cbf0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.649+0000 7fa336400640 1 -- 172.21.15.196:0/1023229550 shutdown_connections 2024-04-28T13:35:52.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.649+0000 7fa336400640 1 --2- 172.21.15.196:0/1023229550 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa31006c980 0x7fa31006ee40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.649+0000 7fa336400640 1 --2- 172.21.15.196:0/1023229550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa33015b6f0 0x7fa3300b2580 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.649+0000 7fa336400640 1 --2- 172.21.15.196:0/1023229550 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa330072e70 0x7fa3300b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.649+0000 7fa336400640 1 --2- 172.21.15.196:0/1023229550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa33006e330 0x7fa330147440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.649+0000 7fa336400640 1 -- 172.21.15.196:0/1023229550 >> 172.21.15.196:0/1023229550 conn(0x7fa3300cd830 msgr2=0x7fa3300b5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:52.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.650+0000 7fa336400640 1 -- 172.21.15.196:0/1023229550 shutdown_connections 2024-04-28T13:35:52.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.650+0000 7fa336400640 1 -- 172.21.15.196:0/1023229550 wait complete. 2024-04-28T13:35:52.667 INFO:teuthology.orchestra.run.smithi196.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-04-28T13:35:52.283888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594563,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":212,"num_write_kb":2437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225876,"kb_used_data":5544,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748844,"statfs":{"total":773094113280,"available":772862816256,"internally_reserved":0,"allocated":5677056,"data_stored":3015396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":282,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004760"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350675+0000","last_change":"2024-04-28T13:33:33.999818+0000","last_active":"2024-04-28T13:34:04.350675+0000","last_peered":"2024-04-28T13:34:04.350675+0000","last_clean":"2024-04-28T13:34:04.350675+0000","last_became_active":"2024-04-28T13:33:33.999486+0000","last_became_peered":"2024-04-28T13:33:33.999486+0000","last_unstale":"2024-04-28T13:34:04.350675+0000","last_undegraded":"2024-04-28T13:34:04.350675+0000","last_fullsized":"2024-04-28T13:34:04.350675+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:32:47.409486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355721+0000","last_change":"2024-04-28T13:33:34.003919+0000","last_active":"2024-04-28T13:34:04.355721+0000","last_peered":"2024-04-28T13:34:04.355721+0000","last_clean":"2024-04-28T13:34:04.355721+0000","last_became_active":"2024-04-28T13:33:34.003551+0000","last_became_peered":"2024-04-28T13:33:34.003551+0000","last_unstale":"2024-04-28T13:34:04.355721+0000","last_undegraded":"2024-04-28T13:34:04.355721+0000","last_fullsized":"2024-04-28T13:34:04.355721+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347343+0000","last_change":"2024-04-28T13:33:33.996733+0000","last_active":"2024-04-28T13:34:04.347343+0000","last_peered":"2024-04-28T13:34:04.347343+0000","last_clean":"2024-04-28T13:34:04.347343+0000","last_became_active":"2024-04-28T13:33:33.996413+0000","last_became_peered":"2024-04-28T13:33:33.996413+0000","last_unstale":"2024-04-28T13:34:04.347343+0000","last_undegraded":"2024-04-28T13:34:04.347343+0000","last_fullsized":"2024-04-28T13:34:04.347343+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:46.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879775+0000","last_change":"2024-04-28T13:33:34.004787+0000","last_active":"2024-04-28T13:34:04.879775+0000","last_peered":"2024-04-28T13:34:04.879775+0000","last_clean":"2024-04-28T13:34:04.879775+0000","last_became_active":"2024-04-28T13:33:33.999818+0000","last_became_peered":"2024-04-28T13:33:33.999818+0000","last_unstale":"2024-04-28T13:34:04.879775+0000","last_undegraded":"2024-04-28T13:34:04.879775+0000","last_fullsized":"2024-04-28T13:34:04.879775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354517+0000","last_change":"2024-04-28T13:33:34.004715+0000","last_active":"2024-04-28T13:34:04.354517+0000","last_peered":"2024-04-28T13:34:04.354517+0000","last_clean":"2024-04-28T13:34:04.354517+0000","last_became_active":"2024-04-28T13:33:34.004436+0000","last_became_peered":"2024-04-28T13:33:34.004436+0000","last_unstale":"2024-04-28T13:34:04.354517+0000","last_undegraded":"2024-04-28T13:34:04.354517+0000","last_fullsized":"2024-04-28T13:34:04.354517+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:46.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344494+0000","last_change":"2024-04-28T13:33:34.012959+0000","last_active":"2024-04-28T13:34:04.344494+0000","last_peered":"2024-04-28T13:34:04.344494+0000","last_clean":"2024-04-28T13:34:04.344494+0000","last_became_active":"2024-04-28T13:33:34.010839+0000","last_became_peered":"2024-04-28T13:33:34.010839+0000","last_unstale":"2024-04-28T13:34:04.344494+0000","last_undegraded":"2024-04-28T13:34:04.344494+0000","last_fullsized":"2024-04-28T13:34:04.344494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:48:58.152999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349347+0000","last_change":"2024-04-28T13:33:34.003448+0000","last_active":"2024-04-28T13:34:04.349347+0000","last_peered":"2024-04-28T13:34:04.349347+0000","last_clean":"2024-04-28T13:34:04.349347+0000","last_became_active":"2024-04-28T13:33:34.002948+0000","last_became_peered":"2024-04-28T13:33:34.002948+0000","last_unstale":"2024-04-28T13:34:04.349347+0000","last_undegraded":"2024-04-28T13:34:04.349347+0000","last_fullsized":"2024-04-28T13:34:04.349347+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:09:39.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.145422+0000","last_change":"2024-04-28T13:34:05.143133+0000","last_active":"2024-04-28T13:34:05.145422+0000","last_peered":"2024-04-28T13:34:05.145422+0000","last_clean":"2024-04-28T13:34:05.145422+0000","last_became_active":"2024-04-28T13:34:05.142351+0000","last_became_peered":"2024-04-28T13:34:05.142351+0000","last_unstale":"2024-04-28T13:34:05.145422+0000","last_undegraded":"2024-04-28T13:34:05.145422+0000","last_fullsized":"2024-04-28T13:34:05.145422+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:53:12.865627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.881028+0000","last_change":"2024-04-28T13:33:34.013366+0000","last_active":"2024-04-28T13:34:04.881028+0000","last_peered":"2024-04-28T13:34:04.881028+0000","last_clean":"2024-04-28T13:34:04.881028+0000","last_became_active":"2024-04-28T13:33:34.012900+0000","last_became_peered":"2024-04-28T13:33:34.012900+0000","last_unstale":"2024-04-28T13:34:04.881028+0000","last_undegraded":"2024-04-28T13:34:04.881028+0000","last_fullsized":"2024-04-28T13:34:04.881028+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:54.852573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880311+0000","last_change":"2024-04-28T13:33:34.007098+0000","last_active":"2024-04-28T13:34:04.880311+0000","last_peered":"2024-04-28T13:34:04.880311+0000","last_clean":"2024-04-28T13:34:04.880311+0000","last_became_active":"2024-04-28T13:33:34.005157+0000","last_became_peered":"2024-04-28T13:33:34.005157+0000","last_unstale":"2024-04-28T13:34:04.880311+0000","last_undegraded":"2024-04-28T13:34:04.880311+0000","last_fullsized":"2024-04-28T13:34:04.880311+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:09:39.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879686+0000","last_change":"2024-04-28T13:33:34.011118+0000","last_active":"2024-04-28T13:34:04.879686+0000","last_peered":"2024-04-28T13:34:04.879686+0000","last_clean":"2024-04-28T13:34:04.879686+0000","last_became_active":"2024-04-28T13:33:34.010811+0000","last_became_peered":"2024-04-28T13:33:34.010811+0000","last_unstale":"2024-04-28T13:34:04.879686+0000","last_undegraded":"2024-04-28T13:34:04.879686+0000","last_fullsized":"2024-04-28T13:34:04.879686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:01:13.143511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355402+0000","last_change":"2024-04-28T13:33:34.011900+0000","last_active":"2024-04-28T13:34:04.355402+0000","last_peered":"2024-04-28T13:34:04.355402+0000","last_clean":"2024-04-28T13:34:04.355402+0000","last_became_active":"2024-04-28T13:33:34.011580+0000","last_became_peered":"2024-04-28T13:33:34.011580+0000","last_unstale":"2024-04-28T13:34:04.355402+0000","last_undegraded":"2024-04-28T13:34:04.355402+0000","last_fullsized":"2024-04-28T13:34:04.355402+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.532062+0000","last_change":"2024-04-28T13:33:34.000874+0000","last_active":"2024-04-28T13:34:05.532062+0000","last_peered":"2024-04-28T13:34:05.532062+0000","last_clean":"2024-04-28T13:34:05.532062+0000","last_became_active":"2024-04-28T13:33:34.000568+0000","last_became_peered":"2024-04-28T13:33:34.000568+0000","last_unstale":"2024-04-28T13:34:05.532062+0000","last_undegraded":"2024-04-28T13:34:05.532062+0000","last_fullsized":"2024-04-28T13:34:05.532062+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:56:40.722825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.502702+0000","last_change":"2024-04-28T13:34:04.496689+0000","last_active":"2024-04-28T13:34:04.502702+0000","last_peered":"2024-04-28T13:34:04.502702+0000","last_clean":"2024-04-28T13:34:04.502702+0000","last_became_active":"2024-04-28T13:34:04.495748+0000","last_became_peered":"2024-04-28T13:34:04.495748+0000","last_unstale":"2024-04-28T13:34:04.502702+0000","last_undegraded":"2024-04-28T13:34:04.502702+0000","last_fullsized":"2024-04-28T13:34:04.502702+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:11.732507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344170+0000","last_change":"2024-04-28T13:33:33.998387+0000","last_active":"2024-04-28T13:34:04.344170+0000","last_peered":"2024-04-28T13:34:04.344170+0000","last_clean":"2024-04-28T13:34:04.344170+0000","last_became_active":"2024-04-28T13:33:33.998087+0000","last_became_peered":"2024-04-28T13:33:33.998087+0000","last_unstale":"2024-04-28T13:34:04.344170+0000","last_undegraded":"2024-04-28T13:34:04.344170+0000","last_fullsized":"2024-04-28T13:34:04.344170+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:52:24.477401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139537+0000","last_change":"2024-04-28T13:33:33.989154+0000","last_active":"2024-04-28T13:34:05.139537+0000","last_peered":"2024-04-28T13:34:05.139537+0000","last_clean":"2024-04-28T13:34:05.139537+0000","last_became_active":"2024-04-28T13:33:33.988827+0000","last_became_peered":"2024-04-28T13:33:33.988827+0000","last_unstale":"2024-04-28T13:34:05.139537+0000","last_undegraded":"2024-04-28T13:34:05.139537+0000","last_fullsized":"2024-04-28T13:34:05.139537+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:52:46.527475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346773+0000","last_change":"2024-04-28T13:33:34.008467+0000","last_active":"2024-04-28T13:34:04.346773+0000","last_peered":"2024-04-28T13:34:04.346773+0000","last_clean":"2024-04-28T13:34:04.346773+0000","last_became_active":"2024-04-28T13:33:34.008181+0000","last_became_peered":"2024-04-28T13:33:34.008181+0000","last_unstale":"2024-04-28T13:34:04.346773+0000","last_undegraded":"2024-04-28T13:34:04.346773+0000","last_fullsized":"2024-04-28T13:34:04.346773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347607+0000","last_change":"2024-04-28T13:33:34.007505+0000","last_active":"2024-04-28T13:34:04.347607+0000","last_peered":"2024-04-28T13:34:04.347607+0000","last_clean":"2024-04-28T13:34:04.347607+0000","last_became_active":"2024-04-28T13:33:34.006793+0000","last_became_peered":"2024-04-28T13:33:34.006793+0000","last_unstale":"2024-04-28T13:34:04.347607+0000","last_undegraded":"2024-04-28T13:34:04.347607+0000","last_fullsized":"2024-04-28T13:34:04.347607+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.140956+0000","last_change":"2024-04-28T13:33:34.009974+0000","last_active":"2024-04-28T13:34:05.140956+0000","last_peered":"2024-04-28T13:34:05.140956+0000","last_clean":"2024-04-28T13:34:05.140956+0000","last_became_active":"2024-04-28T13:33:34.009657+0000","last_became_peered":"2024-04-28T13:33:34.009657+0000","last_unstale":"2024-04-28T13:34:05.140956+0000","last_undegraded":"2024-04-28T13:34:05.140956+0000","last_fullsized":"2024-04-28T13:34:05.140956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:18:56.809253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531828+0000","last_change":"2024-04-28T13:33:34.004233+0000","last_active":"2024-04-28T13:34:05.531828+0000","last_peered":"2024-04-28T13:34:05.531828+0000","last_clean":"2024-04-28T13:34:05.531828+0000","last_became_active":"2024-04-28T13:33:33.999567+0000","last_became_peered":"2024-04-28T13:33:33.999567+0000","last_unstale":"2024-04-28T13:34:05.531828+0000","last_undegraded":"2024-04-28T13:34:05.531828+0000","last_fullsized":"2024-04-28T13:34:05.531828+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:36.248393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344842+0000","last_change":"2024-04-28T13:33:34.008932+0000","last_active":"2024-04-28T13:34:04.344842+0000","last_peered":"2024-04-28T13:34:04.344842+0000","last_clean":"2024-04-28T13:34:04.344842+0000","last_became_active":"2024-04-28T13:33:34.008652+0000","last_became_peered":"2024-04-28T13:33:34.008652+0000","last_unstale":"2024-04-28T13:34:04.344842+0000","last_undegraded":"2024-04-28T13:34:04.344842+0000","last_fullsized":"2024-04-28T13:34:04.344842+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:43:22.000390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880044+0000","last_change":"2024-04-28T13:33:34.006521+0000","last_active":"2024-04-28T13:34:04.880044+0000","last_peered":"2024-04-28T13:34:04.880044+0000","last_clean":"2024-04-28T13:34:04.880044+0000","last_became_active":"2024-04-28T13:33:34.006227+0000","last_became_peered":"2024-04-28T13:33:34.006227+0000","last_unstale":"2024-04-28T13:34:04.880044+0000","last_undegraded":"2024-04-28T13:34:04.880044+0000","last_fullsized":"2024-04-28T13:34:04.880044+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:46.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139737+0000","last_change":"2024-04-28T13:33:33.999169+0000","last_active":"2024-04-28T13:34:05.139737+0000","last_peered":"2024-04-28T13:34:05.139737+0000","last_clean":"2024-04-28T13:34:05.139737+0000","last_became_active":"2024-04-28T13:33:33.998179+0000","last_became_peered":"2024-04-28T13:33:33.998179+0000","last_unstale":"2024-04-28T13:34:05.139737+0000","last_undegraded":"2024-04-28T13:34:05.139737+0000","last_fullsized":"2024-04-28T13:34:05.139737+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:12.546153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139678+0000","last_change":"2024-04-28T13:33:33.998034+0000","last_active":"2024-04-28T13:34:05.139678+0000","last_peered":"2024-04-28T13:34:05.139678+0000","last_clean":"2024-04-28T13:34:05.139678+0000","last_became_active":"2024-04-28T13:33:33.997648+0000","last_became_peered":"2024-04-28T13:33:33.997648+0000","last_unstale":"2024-04-28T13:34:05.139678+0000","last_undegraded":"2024-04-28T13:34:05.139678+0000","last_fullsized":"2024-04-28T13:34:05.139678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:30.570296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.141197+0000","last_change":"2024-04-28T13:33:34.007275+0000","last_active":"2024-04-28T13:34:05.141197+0000","last_peered":"2024-04-28T13:34:05.141197+0000","last_clean":"2024-04-28T13:34:05.141197+0000","last_became_active":"2024-04-28T13:33:34.005017+0000","last_became_peered":"2024-04-28T13:33:34.005017+0000","last_unstale":"2024-04-28T13:34:05.141197+0000","last_undegraded":"2024-04-28T13:34:05.141197+0000","last_fullsized":"2024-04-28T13:34:05.141197+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:37:53.261649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531787+0000","last_change":"2024-04-28T13:33:33.995804+0000","last_active":"2024-04-28T13:34:05.531787+0000","last_peered":"2024-04-28T13:34:05.531787+0000","last_clean":"2024-04-28T13:34:05.531787+0000","last_became_active":"2024-04-28T13:33:33.995502+0000","last_became_peered":"2024-04-28T13:33:33.995502+0000","last_unstale":"2024-04-28T13:34:05.531787+0000","last_undegraded":"2024-04-28T13:34:05.531787+0000","last_fullsized":"2024-04-28T13:34:05.531787+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:27:53.804713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347538+0000","last_change":"2024-04-28T13:33:34.006447+0000","last_active":"2024-04-28T13:34:04.347538+0000","last_peered":"2024-04-28T13:34:04.347538+0000","last_clean":"2024-04-28T13:34:04.347538+0000","last_became_active":"2024-04-28T13:33:34.005656+0000","last_became_peered":"2024-04-28T13:33:34.005656+0000","last_unstale":"2024-04-28T13:34:04.347538+0000","last_undegraded":"2024-04-28T13:34:04.347538+0000","last_fullsized":"2024-04-28T13:34:04.347538+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355564+0000","last_change":"2024-04-28T13:33:34.005608+0000","last_active":"2024-04-28T13:34:04.355564+0000","last_peered":"2024-04-28T13:34:04.355564+0000","last_clean":"2024-04-28T13:34:04.355564+0000","last_became_active":"2024-04-28T13:33:34.005316+0000","last_became_peered":"2024-04-28T13:33:34.005316+0000","last_unstale":"2024-04-28T13:34:04.355564+0000","last_undegraded":"2024-04-28T13:34:04.355564+0000","last_fullsized":"2024-04-28T13:34:04.355564+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139766+0000","last_change":"2024-04-28T13:33:34.002648+0000","last_active":"2024-04-28T13:34:05.139766+0000","last_peered":"2024-04-28T13:34:05.139766+0000","last_clean":"2024-04-28T13:34:05.139766+0000","last_became_active":"2024-04-28T13:33:34.002375+0000","last_became_peered":"2024-04-28T13:33:34.002375+0000","last_unstale":"2024-04-28T13:34:05.139766+0000","last_undegraded":"2024-04-28T13:34:05.139766+0000","last_fullsized":"2024-04-28T13:34:05.139766+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:42.082730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346448+0000","last_change":"2024-04-28T13:33:33.999443+0000","last_active":"2024-04-28T13:34:04.346448+0000","last_peered":"2024-04-28T13:34:04.346448+0000","last_clean":"2024-04-28T13:34:04.346448+0000","last_became_active":"2024-04-28T13:33:33.998905+0000","last_became_peered":"2024-04-28T13:33:33.998905+0000","last_unstale":"2024-04-28T13:34:04.346448+0000","last_undegraded":"2024-04-28T13:34:04.346448+0000","last_fullsized":"2024-04-28T13:34:04.346448+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344364+0000","last_change":"2024-04-28T13:33:34.008881+0000","last_active":"2024-04-28T13:34:04.344364+0000","last_peered":"2024-04-28T13:34:04.344364+0000","last_clean":"2024-04-28T13:34:04.344364+0000","last_became_active":"2024-04-28T13:33:34.008297+0000","last_became_peered":"2024-04-28T13:33:34.008297+0000","last_unstale":"2024-04-28T13:34:04.344364+0000","last_undegraded":"2024-04-28T13:34:04.344364+0000","last_fullsized":"2024-04-28T13:34:04.344364+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:41:26.988112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531605+0000","last_change":"2024-04-28T13:33:34.003176+0000","last_active":"2024-04-28T13:34:05.531605+0000","last_peered":"2024-04-28T13:34:05.531605+0000","last_clean":"2024-04-28T13:34:05.531605+0000","last_became_active":"2024-04-28T13:33:34.002915+0000","last_became_peered":"2024-04-28T13:33:34.002915+0000","last_unstale":"2024-04-28T13:34:05.531605+0000","last_undegraded":"2024-04-28T13:34:05.531605+0000","last_fullsized":"2024-04-28T13:34:05.531605+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:52:06.849498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493215+0000","last_change":"2024-04-28T13:33:34.005669+0000","last_active":"2024-04-28T13:34:04.493215+0000","last_peered":"2024-04-28T13:34:04.493215+0000","last_clean":"2024-04-28T13:34:04.493215+0000","last_became_active":"2024-04-28T13:33:34.005365+0000","last_became_peered":"2024-04-28T13:33:34.005365+0000","last_unstale":"2024-04-28T13:34:04.493215+0000","last_undegraded":"2024-04-28T13:34:04.493215+0000","last_fullsized":"2024-04-28T13:34:04.493215+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:18:56.809253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":23,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345243+0000","last_change":"2024-04-28T13:33:32.998037+0000","last_active":"2024-04-28T13:34:04.345243+0000","last_peered":"2024-04-28T13:34:04.345243+0000","last_clean":"2024-04-28T13:34:04.345243+0000","last_became_active":"2024-04-28T13:33:32.997694+0000","last_became_peered":"2024-04-28T13:33:32.997694+0000","last_unstale":"2024-04-28T13:34:04.345243+0000","last_undegraded":"2024-04-28T13:34:04.345243+0000","last_fullsized":"2024-04-28T13:34:04.345243+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:02.432193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.141337+0000","last_change":"2024-04-28T13:33:33.989048+0000","last_active":"2024-04-28T13:34:05.141337+0000","last_peered":"2024-04-28T13:34:05.141337+0000","last_clean":"2024-04-28T13:34:05.141337+0000","last_became_active":"2024-04-28T13:33:33.988758+0000","last_became_peered":"2024-04-28T13:33:33.988758+0000","last_unstale":"2024-04-28T13:34:05.141337+0000","last_undegraded":"2024-04-28T13:34:05.141337+0000","last_fullsized":"2024-04-28T13:34:05.141337+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:14.152642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347981+0000","last_change":"2024-04-28T13:33:32.999239+0000","last_active":"2024-04-28T13:34:04.347981+0000","last_peered":"2024-04-28T13:34:04.347981+0000","last_clean":"2024-04-28T13:34:04.347981+0000","last_became_active":"2024-04-28T13:33:32.998813+0000","last_became_peered":"2024-04-28T13:33:32.998813+0000","last_unstale":"2024-04-28T13:34:04.347981+0000","last_undegraded":"2024-04-28T13:34:04.347981+0000","last_fullsized":"2024-04-28T13:34:04.347981+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:22:34.444155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355706+0000","last_change":"2024-04-28T13:33:34.003176+0000","last_active":"2024-04-28T13:34:04.355706+0000","last_peered":"2024-04-28T13:34:04.355706+0000","last_clean":"2024-04-28T13:34:04.355706+0000","last_became_active":"2024-04-28T13:33:34.002904+0000","last_became_peered":"2024-04-28T13:33:34.002904+0000","last_unstale":"2024-04-28T13:34:04.355706+0000","last_undegraded":"2024-04-28T13:34:04.355706+0000","last_fullsized":"2024-04-28T13:34:04.355706+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493775+0000","last_change":"2024-04-28T13:33:32.997729+0000","last_active":"2024-04-28T13:34:04.493775+0000","last_peered":"2024-04-28T13:34:04.493775+0000","last_clean":"2024-04-28T13:34:04.493775+0000","last_became_active":"2024-04-28T13:33:32.996881+0000","last_became_peered":"2024-04-28T13:33:32.996881+0000","last_unstale":"2024-04-28T13:34:04.493775+0000","last_undegraded":"2024-04-28T13:34:04.493775+0000","last_fullsized":"2024-04-28T13:34:04.493775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493141+0000","last_change":"2024-04-28T13:33:34.004339+0000","last_active":"2024-04-28T13:34:04.493141+0000","last_peered":"2024-04-28T13:34:04.493141+0000","last_clean":"2024-04-28T13:34:04.493141+0000","last_became_active":"2024-04-28T13:33:34.002361+0000","last_became_peered":"2024-04-28T13:33:34.002361+0000","last_unstale":"2024-04-28T13:34:04.493141+0000","last_undegraded":"2024-04-28T13:34:04.493141+0000","last_fullsized":"2024-04-28T13:34:04.493141+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:31.671210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350207+0000","last_change":"2024-04-28T13:33:33.000754+0000","last_active":"2024-04-28T13:34:04.350207+0000","last_peered":"2024-04-28T13:34:04.350207+0000","last_clean":"2024-04-28T13:34:04.350207+0000","last_became_active":"2024-04-28T13:33:33.000387+0000","last_became_peered":"2024-04-28T13:33:33.000387+0000","last_unstale":"2024-04-28T13:34:04.350207+0000","last_undegraded":"2024-04-28T13:34:04.350207+0000","last_fullsized":"2024-04-28T13:34:04.350207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:28.481114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346329+0000","last_change":"2024-04-28T13:33:33.994316+0000","last_active":"2024-04-28T13:34:04.346329+0000","last_peered":"2024-04-28T13:34:04.346329+0000","last_clean":"2024-04-28T13:34:04.346329+0000","last_became_active":"2024-04-28T13:33:33.994002+0000","last_became_peered":"2024-04-28T13:33:33.994002+0000","last_unstale":"2024-04-28T13:34:04.346329+0000","last_undegraded":"2024-04-28T13:34:04.346329+0000","last_fullsized":"2024-04-28T13:34:04.346329+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:03.446736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344611+0000","last_change":"2024-04-28T13:33:32.999540+0000","last_active":"2024-04-28T13:34:04.344611+0000","last_peered":"2024-04-28T13:34:04.344611+0000","last_clean":"2024-04-28T13:34:04.344611+0000","last_became_active":"2024-04-28T13:33:32.999147+0000","last_became_peered":"2024-04-28T13:33:32.999147+0000","last_unstale":"2024-04-28T13:34:04.344611+0000","last_undegraded":"2024-04-28T13:34:04.344611+0000","last_fullsized":"2024-04-28T13:34:04.344611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:45.657949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880509+0000","last_change":"2024-04-28T13:33:34.000424+0000","last_active":"2024-04-28T13:34:04.880509+0000","last_peered":"2024-04-28T13:34:04.880509+0000","last_clean":"2024-04-28T13:34:04.880509+0000","last_became_active":"2024-04-28T13:33:33.999414+0000","last_became_peered":"2024-04-28T13:33:33.999414+0000","last_unstale":"2024-04-28T13:34:04.880509+0000","last_undegraded":"2024-04-28T13:34:04.880509+0000","last_fullsized":"2024-04-28T13:34:04.880509+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346593+0000","last_change":"2024-04-28T13:33:32.988943+0000","last_active":"2024-04-28T13:34:04.346593+0000","last_peered":"2024-04-28T13:34:04.346593+0000","last_clean":"2024-04-28T13:34:04.346593+0000","last_became_active":"2024-04-28T13:33:32.986596+0000","last_became_peered":"2024-04-28T13:33:32.986596+0000","last_unstale":"2024-04-28T13:34:04.346593+0000","last_undegraded":"2024-04-28T13:34:04.346593+0000","last_fullsized":"2024-04-28T13:34:04.346593+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879866+0000","last_change":"2024-04-28T13:33:34.002579+0000","last_active":"2024-04-28T13:34:04.879866+0000","last_peered":"2024-04-28T13:34:04.879866+0000","last_clean":"2024-04-28T13:34:04.879866+0000","last_became_active":"2024-04-28T13:33:34.002254+0000","last_became_peered":"2024-04-28T13:33:34.002254+0000","last_unstale":"2024-04-28T13:34:04.879866+0000","last_undegraded":"2024-04-28T13:34:04.879866+0000","last_fullsized":"2024-04-28T13:34:04.879866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354254+0000","last_change":"2024-04-28T13:33:32.996052+0000","last_active":"2024-04-28T13:34:04.354254+0000","last_peered":"2024-04-28T13:34:04.354254+0000","last_clean":"2024-04-28T13:34:04.354254+0000","last_became_active":"2024-04-28T13:33:32.990258+0000","last_became_peered":"2024-04-28T13:33:32.990258+0000","last_unstale":"2024-04-28T13:34:04.354254+0000","last_undegraded":"2024-04-28T13:34:04.354254+0000","last_fullsized":"2024-04-28T13:34:04.354254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139482+0000","last_change":"2024-04-28T13:33:33.998180+0000","last_active":"2024-04-28T13:34:05.139482+0000","last_peered":"2024-04-28T13:34:05.139482+0000","last_clean":"2024-04-28T13:34:05.139482+0000","last_became_active":"2024-04-28T13:33:33.997730+0000","last_became_peered":"2024-04-28T13:33:33.997730+0000","last_unstale":"2024-04-28T13:34:05.139482+0000","last_undegraded":"2024-04-28T13:34:05.139482+0000","last_fullsized":"2024-04-28T13:34:05.139482+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:22:35.458698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880139+0000","last_change":"2024-04-28T13:33:32.989759+0000","last_active":"2024-04-28T13:34:04.880139+0000","last_peered":"2024-04-28T13:34:04.880139+0000","last_clean":"2024-04-28T13:34:04.880139+0000","last_became_active":"2024-04-28T13:33:32.988565+0000","last_became_peered":"2024-04-28T13:33:32.988565+0000","last_unstale":"2024-04-28T13:34:04.880139+0000","last_undegraded":"2024-04-28T13:34:04.880139+0000","last_fullsized":"2024-04-28T13:34:04.880139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:27:52.790170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347610+0000","last_change":"2024-04-28T13:33:34.007736+0000","last_active":"2024-04-28T13:34:04.347610+0000","last_peered":"2024-04-28T13:34:04.347610+0000","last_clean":"2024-04-28T13:34:04.347610+0000","last_became_active":"2024-04-28T13:33:34.007170+0000","last_became_peered":"2024-04-28T13:33:34.007170+0000","last_unstale":"2024-04-28T13:34:04.347610+0000","last_undegraded":"2024-04-28T13:34:04.347610+0000","last_fullsized":"2024-04-28T13:34:04.347610+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:01:33.789936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347247+0000","last_change":"2024-04-28T13:33:32.992483+0000","last_active":"2024-04-28T13:34:04.347247+0000","last_peered":"2024-04-28T13:34:04.347247+0000","last_clean":"2024-04-28T13:34:04.347247+0000","last_became_active":"2024-04-28T13:33:32.992216+0000","last_became_peered":"2024-04-28T13:33:32.992216+0000","last_unstale":"2024-04-28T13:34:04.347247+0000","last_undegraded":"2024-04-28T13:34:04.347247+0000","last_fullsized":"2024-04-28T13:34:04.347247+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:11.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350567+0000","last_change":"2024-04-28T13:33:34.009420+0000","last_active":"2024-04-28T13:34:04.350567+0000","last_peered":"2024-04-28T13:34:04.350567+0000","last_clean":"2024-04-28T13:34:04.350567+0000","last_became_active":"2024-04-28T13:33:34.009020+0000","last_became_peered":"2024-04-28T13:33:34.009020+0000","last_unstale":"2024-04-28T13:34:04.350567+0000","last_undegraded":"2024-04-28T13:34:04.350567+0000","last_fullsized":"2024-04-28T13:34:04.350567+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:48:58.152999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880919+0000","last_change":"2024-04-28T13:33:32.997498+0000","last_active":"2024-04-28T13:34:04.880919+0000","last_peered":"2024-04-28T13:34:04.880919+0000","last_clean":"2024-04-28T13:34:04.880919+0000","last_became_active":"2024-04-28T13:33:32.997237+0000","last_became_peered":"2024-04-28T13:33:32.997237+0000","last_unstale":"2024-04-28T13:34:04.880919+0000","last_undegraded":"2024-04-28T13:34:04.880919+0000","last_fullsized":"2024-04-28T13:34:04.880919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:37:18.628510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344809+0000","last_change":"2024-04-28T13:33:34.002865+0000","last_active":"2024-04-28T13:34:04.344809+0000","last_peered":"2024-04-28T13:34:04.344809+0000","last_clean":"2024-04-28T13:34:04.344809+0000","last_became_active":"2024-04-28T13:33:34.002421+0000","last_became_peered":"2024-04-28T13:33:34.002421+0000","last_unstale":"2024-04-28T13:34:04.344809+0000","last_undegraded":"2024-04-28T13:34:04.344809+0000","last_fullsized":"2024-04-28T13:34:04.344809+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:51:46.332983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350650+0000","last_change":"2024-04-28T13:33:33.004559+0000","last_active":"2024-04-28T13:34:04.350650+0000","last_peered":"2024-04-28T13:34:04.350650+0000","last_clean":"2024-04-28T13:34:04.350650+0000","last_became_active":"2024-04-28T13:33:33.004153+0000","last_became_peered":"2024-04-28T13:33:33.004153+0000","last_unstale":"2024-04-28T13:34:04.350650+0000","last_undegraded":"2024-04-28T13:34:04.350650+0000","last_fullsized":"2024-04-28T13:34:04.350650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:02.432193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350366+0000","last_change":"2024-04-28T13:33:34.006854+0000","last_active":"2024-04-28T13:34:04.350366+0000","last_peered":"2024-04-28T13:34:04.350366+0000","last_clean":"2024-04-28T13:34:04.350366+0000","last_became_active":"2024-04-28T13:33:34.006561+0000","last_became_peered":"2024-04-28T13:33:34.006561+0000","last_unstale":"2024-04-28T13:34:04.350366+0000","last_undegraded":"2024-04-28T13:34:04.350366+0000","last_fullsized":"2024-04-28T13:34:04.350366+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:54.852573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'12","reported_seq":30,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:47.492938+0000","last_change":"2024-04-28T13:33:32.991367+0000","last_active":"2024-04-28T13:35:47.492938+0000","last_peered":"2024-04-28T13:35:47.492938+0000","last_clean":"2024-04-28T13:35:47.492938+0000","last_became_active":"2024-04-28T13:33:32.991060+0000","last_became_peered":"2024-04-28T13:33:32.991060+0000","last_unstale":"2024-04-28T13:35:47.492938+0000","last_undegraded":"2024-04-28T13:35:47.492938+0000","last_fullsized":"2024-04-28T13:35:47.492938+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:00:16.144423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3397,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139850+0000","last_change":"2024-04-28T13:33:33.996658+0000","last_active":"2024-04-28T13:34:05.139850+0000","last_peered":"2024-04-28T13:34:05.139850+0000","last_clean":"2024-04-28T13:34:05.139850+0000","last_became_active":"2024-04-28T13:33:33.996260+0000","last_became_peered":"2024-04-28T13:33:33.996260+0000","last_unstale":"2024-04-28T13:34:05.139850+0000","last_undegraded":"2024-04-28T13:34:05.139850+0000","last_fullsized":"2024-04-28T13:34:05.139850+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:52.292838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345011+0000","last_change":"2024-04-28T13:33:32.990968+0000","last_active":"2024-04-28T13:34:04.345011+0000","last_peered":"2024-04-28T13:34:04.345011+0000","last_clean":"2024-04-28T13:34:04.345011+0000","last_became_active":"2024-04-28T13:33:32.990718+0000","last_became_peered":"2024-04-28T13:33:32.990718+0000","last_unstale":"2024-04-28T13:34:04.345011+0000","last_undegraded":"2024-04-28T13:34:04.345011+0000","last_fullsized":"2024-04-28T13:34:04.345011+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:03.358943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880822+0000","last_change":"2024-04-28T13:33:34.003961+0000","last_active":"2024-04-28T13:34:04.880822+0000","last_peered":"2024-04-28T13:34:04.880822+0000","last_clean":"2024-04-28T13:34:04.880822+0000","last_became_active":"2024-04-28T13:33:34.003599+0000","last_became_peered":"2024-04-28T13:33:34.003599+0000","last_unstale":"2024-04-28T13:34:04.880822+0000","last_undegraded":"2024-04-28T13:34:04.880822+0000","last_fullsized":"2024-04-28T13:34:04.880822+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:32:47.409486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349522+0000","last_change":"2024-04-28T13:33:33.000521+0000","last_active":"2024-04-28T13:34:04.349522+0000","last_peered":"2024-04-28T13:34:04.349522+0000","last_clean":"2024-04-28T13:34:04.349522+0000","last_became_active":"2024-04-28T13:33:32.998867+0000","last_became_peered":"2024-04-28T13:33:32.998867+0000","last_unstale":"2024-04-28T13:34:04.349522+0000","last_undegraded":"2024-04-28T13:34:04.349522+0000","last_fullsized":"2024-04-28T13:34:04.349522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:43.562896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344089+0000","last_change":"2024-04-28T13:33:34.006368+0000","last_active":"2024-04-28T13:34:04.344089+0000","last_peered":"2024-04-28T13:34:04.344089+0000","last_clean":"2024-04-28T13:34:04.344089+0000","last_became_active":"2024-04-28T13:33:34.002579+0000","last_became_peered":"2024-04-28T13:33:34.002579+0000","last_unstale":"2024-04-28T13:34:04.344089+0000","last_undegraded":"2024-04-28T13:34:04.344089+0000","last_fullsized":"2024-04-28T13:34:04.344089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:27:51.330692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345395+0000","last_change":"2024-04-28T13:33:32.991648+0000","last_active":"2024-04-28T13:34:04.345395+0000","last_peered":"2024-04-28T13:34:04.345395+0000","last_clean":"2024-04-28T13:34:04.345395+0000","last_became_active":"2024-04-28T13:33:32.991369+0000","last_became_peered":"2024-04-28T13:33:32.991369+0000","last_unstale":"2024-04-28T13:34:04.345395+0000","last_undegraded":"2024-04-28T13:34:04.345395+0000","last_fullsized":"2024-04-28T13:34:04.345395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:43.562896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349607+0000","last_change":"2024-04-28T13:33:34.010407+0000","last_active":"2024-04-28T13:34:04.349607+0000","last_peered":"2024-04-28T13:34:04.349607+0000","last_clean":"2024-04-28T13:34:04.349607+0000","last_became_active":"2024-04-28T13:33:34.010077+0000","last_became_peered":"2024-04-28T13:33:34.010077+0000","last_unstale":"2024-04-28T13:34:04.349607+0000","last_undegraded":"2024-04-28T13:34:04.349607+0000","last_fullsized":"2024-04-28T13:34:04.349607+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:20:07.691466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880423+0000","last_change":"2024-04-28T13:33:32.989587+0000","last_active":"2024-04-28T13:34:04.880423+0000","last_peered":"2024-04-28T13:34:04.880423+0000","last_clean":"2024-04-28T13:34:04.880423+0000","last_became_active":"2024-04-28T13:33:32.988579+0000","last_became_peered":"2024-04-28T13:33:32.988579+0000","last_unstale":"2024-04-28T13:34:04.880423+0000","last_undegraded":"2024-04-28T13:34:04.880423+0000","last_fullsized":"2024-04-28T13:34:04.880423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:35.233850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.532050+0000","last_change":"2024-04-28T13:33:34.012322+0000","last_active":"2024-04-28T13:34:05.532050+0000","last_peered":"2024-04-28T13:34:05.532050+0000","last_clean":"2024-04-28T13:34:05.532050+0000","last_became_active":"2024-04-28T13:33:34.012112+0000","last_became_peered":"2024-04-28T13:33:34.012112+0000","last_unstale":"2024-04-28T13:34:05.532050+0000","last_undegraded":"2024-04-28T13:34:05.532050+0000","last_fullsized":"2024-04-28T13:34:05.532050+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:10.015862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531304+0000","last_change":"2024-04-28T13:33:32.990958+0000","last_active":"2024-04-28T13:34:05.531304+0000","last_peered":"2024-04-28T13:34:05.531304+0000","last_clean":"2024-04-28T13:34:05.531304+0000","last_became_active":"2024-04-28T13:33:32.990659+0000","last_became_peered":"2024-04-28T13:33:32.990659+0000","last_unstale":"2024-04-28T13:34:05.531304+0000","last_undegraded":"2024-04-28T13:34:05.531304+0000","last_fullsized":"2024-04-28T13:34:05.531304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:24:34.028457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.881209+0000","last_change":"2024-04-28T13:33:34.005832+0000","last_active":"2024-04-28T13:34:04.881209+0000","last_peered":"2024-04-28T13:34:04.881209+0000","last_clean":"2024-04-28T13:34:04.881209+0000","last_became_active":"2024-04-28T13:33:34.005364+0000","last_became_peered":"2024-04-28T13:33:34.005364+0000","last_unstale":"2024-04-28T13:34:04.881209+0000","last_undegraded":"2024-04-28T13:34:04.881209+0000","last_fullsized":"2024-04-28T13:34:04.881209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:28.548233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.348084+0000","last_change":"2024-04-28T13:33:32.995338+0000","last_active":"2024-04-28T13:34:04.348084+0000","last_peered":"2024-04-28T13:34:04.348084+0000","last_clean":"2024-04-28T13:34:04.348084+0000","last_became_active":"2024-04-28T13:33:32.994798+0000","last_became_peered":"2024-04-28T13:33:32.994798+0000","last_unstale":"2024-04-28T13:34:04.348084+0000","last_undegraded":"2024-04-28T13:34:04.348084+0000","last_fullsized":"2024-04-28T13:34:04.348084+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:29.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.140801+0000","last_change":"2024-04-28T13:33:34.002242+0000","last_active":"2024-04-28T13:34:05.140801+0000","last_peered":"2024-04-28T13:34:05.140801+0000","last_clean":"2024-04-28T13:34:05.140801+0000","last_became_active":"2024-04-28T13:33:34.001893+0000","last_became_peered":"2024-04-28T13:33:34.001893+0000","last_unstale":"2024-04-28T13:34:05.140801+0000","last_undegraded":"2024-04-28T13:34:05.140801+0000","last_fullsized":"2024-04-28T13:34:05.140801+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:35.969568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355718+0000","last_change":"2024-04-28T13:33:32.991441+0000","last_active":"2024-04-28T13:34:04.355718+0000","last_peered":"2024-04-28T13:34:04.355718+0000","last_clean":"2024-04-28T13:34:04.355718+0000","last_became_active":"2024-04-28T13:33:32.990745+0000","last_became_peered":"2024-04-28T13:33:32.990745+0000","last_unstale":"2024-04-28T13:34:04.355718+0000","last_undegraded":"2024-04-28T13:34:04.355718+0000","last_fullsized":"2024-04-28T13:34:04.355718+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:30.656667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.140616+0000","last_change":"2024-04-28T13:33:34.003094+0000","last_active":"2024-04-28T13:34:05.140616+0000","last_peered":"2024-04-28T13:34:05.140616+0000","last_clean":"2024-04-28T13:34:05.140616+0000","last_became_active":"2024-04-28T13:33:34.002228+0000","last_became_peered":"2024-04-28T13:33:34.002228+0000","last_unstale":"2024-04-28T13:34:05.140616+0000","last_undegraded":"2024-04-28T13:34:05.140616+0000","last_fullsized":"2024-04-28T13:34:05.140616+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:31.671210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344933+0000","last_change":"2024-04-28T13:33:33.003720+0000","last_active":"2024-04-28T13:34:04.344933+0000","last_peered":"2024-04-28T13:34:04.344933+0000","last_clean":"2024-04-28T13:34:04.344933+0000","last_became_active":"2024-04-28T13:33:33.003389+0000","last_became_peered":"2024-04-28T13:33:33.003389+0000","last_unstale":"2024-04-28T13:34:04.344933+0000","last_undegraded":"2024-04-28T13:34:04.344933+0000","last_fullsized":"2024-04-28T13:34:04.344933+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:18.814381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345100+0000","last_change":"2024-04-28T13:33:34.006579+0000","last_active":"2024-04-28T13:34:04.345100+0000","last_peered":"2024-04-28T13:34:04.345100+0000","last_clean":"2024-04-28T13:34:04.345100+0000","last_became_active":"2024-04-28T13:33:34.006282+0000","last_became_peered":"2024-04-28T13:33:34.006282+0000","last_unstale":"2024-04-28T13:34:04.345100+0000","last_undegraded":"2024-04-28T13:34:04.345100+0000","last_fullsized":"2024-04-28T13:34:04.345100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:18:15.243319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345377+0000","last_change":"2024-04-28T13:33:32.999347+0000","last_active":"2024-04-28T13:34:04.345377+0000","last_peered":"2024-04-28T13:34:04.345377+0000","last_clean":"2024-04-28T13:34:04.345377+0000","last_became_active":"2024-04-28T13:33:32.998923+0000","last_became_peered":"2024-04-28T13:33:32.998923+0000","last_unstale":"2024-04-28T13:34:04.345377+0000","last_undegraded":"2024-04-28T13:34:04.345377+0000","last_fullsized":"2024-04-28T13:34:04.345377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:46.522691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.492925+0000","last_change":"2024-04-28T13:33:34.011875+0000","last_active":"2024-04-28T13:34:04.492925+0000","last_peered":"2024-04-28T13:34:04.492925+0000","last_clean":"2024-04-28T13:34:04.492925+0000","last_became_active":"2024-04-28T13:33:34.011271+0000","last_became_peered":"2024-04-28T13:33:34.011271+0000","last_unstale":"2024-04-28T13:34:04.492925+0000","last_undegraded":"2024-04-28T13:34:04.492925+0000","last_fullsized":"2024-04-28T13:34:04.492925+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:35.969568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354592+0000","last_change":"2024-04-28T13:33:33.002216+0000","last_active":"2024-04-28T13:34:04.354592+0000","last_peered":"2024-04-28T13:34:04.354592+0000","last_clean":"2024-04-28T13:34:04.354592+0000","last_became_active":"2024-04-28T13:33:33.000335+0000","last_became_peered":"2024-04-28T13:33:33.000335+0000","last_unstale":"2024-04-28T13:34:04.354592+0000","last_undegraded":"2024-04-28T13:34:04.354592+0000","last_fullsized":"2024-04-28T13:34:04.354592+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:34.955026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531906+0000","last_change":"2024-04-28T13:33:34.000104+0000","last_active":"2024-04-28T13:34:05.531906+0000","last_peered":"2024-04-28T13:34:05.531906+0000","last_clean":"2024-04-28T13:34:05.531906+0000","last_became_active":"2024-04-28T13:33:33.999839+0000","last_became_peered":"2024-04-28T13:33:33.999839+0000","last_unstale":"2024-04-28T13:34:05.531906+0000","last_undegraded":"2024-04-28T13:34:05.531906+0000","last_fullsized":"2024-04-28T13:34:05.531906+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:03:14.657009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879925+0000","last_change":"2024-04-28T13:33:32.992149+0000","last_active":"2024-04-28T13:34:04.879925+0000","last_peered":"2024-04-28T13:34:04.879925+0000","last_clean":"2024-04-28T13:34:04.879925+0000","last_became_active":"2024-04-28T13:33:32.991800+0000","last_became_peered":"2024-04-28T13:33:32.991800+0000","last_unstale":"2024-04-28T13:34:04.879925+0000","last_undegraded":"2024-04-28T13:34:04.879925+0000","last_fullsized":"2024-04-28T13:34:04.879925+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:56:39.708282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879801+0000","last_change":"2024-04-28T13:33:34.011133+0000","last_active":"2024-04-28T13:34:04.879801+0000","last_peered":"2024-04-28T13:34:04.879801+0000","last_clean":"2024-04-28T13:34:04.879801+0000","last_became_active":"2024-04-28T13:33:34.010896+0000","last_became_peered":"2024-04-28T13:33:34.010896+0000","last_unstale":"2024-04-28T13:34:04.879801+0000","last_undegraded":"2024-04-28T13:34:04.879801+0000","last_fullsized":"2024-04-28T13:34:04.879801+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.348967+0000","last_change":"2024-04-28T13:33:33.000859+0000","last_active":"2024-04-28T13:34:04.348967+0000","last_peered":"2024-04-28T13:34:04.348967+0000","last_clean":"2024-04-28T13:34:04.348967+0000","last_became_active":"2024-04-28T13:33:33.000095+0000","last_became_peered":"2024-04-28T13:33:33.000095+0000","last_unstale":"2024-04-28T13:34:04.348967+0000","last_undegraded":"2024-04-28T13:34:04.348967+0000","last_fullsized":"2024-04-28T13:34:04.348967+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:18.221214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.141479+0000","last_change":"2024-04-28T13:33:34.008405+0000","last_active":"2024-04-28T13:34:05.141479+0000","last_peered":"2024-04-28T13:34:05.141479+0000","last_clean":"2024-04-28T13:34:05.141479+0000","last_became_active":"2024-04-28T13:33:34.007425+0000","last_became_peered":"2024-04-28T13:33:34.007425+0000","last_unstale":"2024-04-28T13:34:05.141479+0000","last_undegraded":"2024-04-28T13:34:05.141479+0000","last_fullsized":"2024-04-28T13:34:05.141479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:29.495657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"47'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:45.005269+0000","last_change":"2024-04-28T13:33:32.998594+0000","last_active":"2024-04-28T13:34:45.005269+0000","last_peered":"2024-04-28T13:34:45.005269+0000","last_clean":"2024-04-28T13:34:45.005269+0000","last_became_active":"2024-04-28T13:33:32.998328+0000","last_became_peered":"2024-04-28T13:33:32.998328+0000","last_unstale":"2024-04-28T13:34:45.005269+0000","last_undegraded":"2024-04-28T13:34:45.005269+0000","last_fullsized":"2024-04-28T13:34:45.005269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:18.221214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880058+0000","last_change":"2024-04-28T13:33:33.998134+0000","last_active":"2024-04-28T13:34:04.880058+0000","last_peered":"2024-04-28T13:34:04.880058+0000","last_clean":"2024-04-28T13:34:04.880058+0000","last_became_active":"2024-04-28T13:33:33.997834+0000","last_became_peered":"2024-04-28T13:33:33.997834+0000","last_unstale":"2024-04-28T13:34:04.880058+0000","last_undegraded":"2024-04-28T13:34:04.880058+0000","last_fullsized":"2024-04-28T13:34:04.880058+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:01:33.789936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350856+0000","last_change":"2024-04-28T13:33:33.005737+0000","last_active":"2024-04-28T13:34:04.350856+0000","last_peered":"2024-04-28T13:34:04.350856+0000","last_clean":"2024-04-28T13:34:04.350856+0000","last_became_active":"2024-04-28T13:33:33.005179+0000","last_became_peered":"2024-04-28T13:33:33.005179+0000","last_unstale":"2024-04-28T13:34:04.350856+0000","last_undegraded":"2024-04-28T13:34:04.350856+0000","last_fullsized":"2024-04-28T13:34:04.350856+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:45.657949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350592+0000","last_change":"2024-04-28T13:33:34.011725+0000","last_active":"2024-04-28T13:34:04.350592+0000","last_peered":"2024-04-28T13:34:04.350592+0000","last_clean":"2024-04-28T13:34:04.350592+0000","last_became_active":"2024-04-28T13:33:34.011422+0000","last_became_peered":"2024-04-28T13:33:34.011422+0000","last_unstale":"2024-04-28T13:34:04.350592+0000","last_undegraded":"2024-04-28T13:34:04.350592+0000","last_fullsized":"2024-04-28T13:34:04.350592+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:28.548233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.492969+0000","last_change":"2024-04-28T13:33:32.997338+0000","last_active":"2024-04-28T13:34:04.492969+0000","last_peered":"2024-04-28T13:34:04.492969+0000","last_clean":"2024-04-28T13:34:04.492969+0000","last_became_active":"2024-04-28T13:33:32.996934+0000","last_became_peered":"2024-04-28T13:33:32.996934+0000","last_unstale":"2024-04-28T13:34:04.492969+0000","last_undegraded":"2024-04-28T13:34:04.492969+0000","last_fullsized":"2024-04-28T13:34:04.492969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:29.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345593+0000","last_change":"2024-04-28T13:33:34.020082+0000","last_active":"2024-04-28T13:34:04.345593+0000","last_peered":"2024-04-28T13:34:04.345593+0000","last_clean":"2024-04-28T13:34:04.345593+0000","last_became_active":"2024-04-28T13:33:34.018922+0000","last_became_peered":"2024-04-28T13:33:34.018922+0000","last_unstale":"2024-04-28T13:34:04.345593+0000","last_undegraded":"2024-04-28T13:34:04.345593+0000","last_fullsized":"2024-04-28T13:34:04.345593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:53:12.865627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350434+0000","last_change":"2024-04-28T13:33:33.000626+0000","last_active":"2024-04-28T13:34:04.350434+0000","last_peered":"2024-04-28T13:34:04.350434+0000","last_clean":"2024-04-28T13:34:04.350434+0000","last_became_active":"2024-04-28T13:33:33.000049+0000","last_became_peered":"2024-04-28T13:33:33.000049+0000","last_unstale":"2024-04-28T13:34:04.350434+0000","last_undegraded":"2024-04-28T13:34:04.350434+0000","last_fullsized":"2024-04-28T13:34:04.350434+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:18.814381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344459+0000","last_change":"2024-04-28T13:33:34.004230+0000","last_active":"2024-04-28T13:34:04.344459+0000","last_peered":"2024-04-28T13:34:04.344459+0000","last_clean":"2024-04-28T13:34:04.344459+0000","last_became_active":"2024-04-28T13:33:34.003940+0000","last_became_peered":"2024-04-28T13:33:34.003940+0000","last_unstale":"2024-04-28T13:34:04.344459+0000","last_undegraded":"2024-04-28T13:34:04.344459+0000","last_fullsized":"2024-04-28T13:34:04.344459+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:11.732507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346733+0000","last_change":"2024-04-28T13:33:32.991869+0000","last_active":"2024-04-28T13:34:04.346733+0000","last_peered":"2024-04-28T13:34:04.346733+0000","last_clean":"2024-04-28T13:34:04.346733+0000","last_became_active":"2024-04-28T13:33:32.991435+0000","last_became_peered":"2024-04-28T13:33:32.991435+0000","last_unstale":"2024-04-28T13:34:04.346733+0000","last_undegraded":"2024-04-28T13:34:04.346733+0000","last_fullsized":"2024-04-28T13:34:04.346733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:41.068187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354423+0000","last_change":"2024-04-28T13:33:34.004178+0000","last_active":"2024-04-28T13:34:04.354423+0000","last_peered":"2024-04-28T13:34:04.354423+0000","last_clean":"2024-04-28T13:34:04.354423+0000","last_became_active":"2024-04-28T13:33:34.002092+0000","last_became_peered":"2024-04-28T13:33:34.002092+0000","last_unstale":"2024-04-28T13:34:04.354423+0000","last_undegraded":"2024-04-28T13:34:04.354423+0000","last_fullsized":"2024-04-28T13:34:04.354423+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:03.446736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493055+0000","last_change":"2024-04-28T13:33:33.003796+0000","last_active":"2024-04-28T13:34:04.493055+0000","last_peered":"2024-04-28T13:34:04.493055+0000","last_clean":"2024-04-28T13:34:04.493055+0000","last_became_active":"2024-04-28T13:33:33.003374+0000","last_became_peered":"2024-04-28T13:33:33.003374+0000","last_unstale":"2024-04-28T13:34:04.493055+0000","last_undegraded":"2024-04-28T13:34:04.493055+0000","last_fullsized":"2024-04-28T13:34:04.493055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:11.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347052+0000","last_change":"2024-04-28T13:33:34.001551+0000","last_active":"2024-04-28T13:34:04.347052+0000","last_peered":"2024-04-28T13:34:04.347052+0000","last_clean":"2024-04-28T13:34:04.347052+0000","last_became_active":"2024-04-28T13:33:33.999963+0000","last_became_peered":"2024-04-28T13:33:33.999963+0000","last_unstale":"2024-04-28T13:34:04.347052+0000","last_undegraded":"2024-04-28T13:34:04.347052+0000","last_fullsized":"2024-04-28T13:34:04.347052+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:19.828924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880581+0000","last_change":"2024-04-28T13:33:32.991033+0000","last_active":"2024-04-28T13:34:04.880581+0000","last_peered":"2024-04-28T13:34:04.880581+0000","last_clean":"2024-04-28T13:34:04.880581+0000","last_became_active":"2024-04-28T13:33:32.990726+0000","last_became_peered":"2024-04-28T13:33:32.990726+0000","last_unstale":"2024-04-28T13:34:04.880581+0000","last_undegraded":"2024-04-28T13:34:04.880581+0000","last_fullsized":"2024-04-28T13:34:04.880581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:09.001319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880728+0000","last_change":"2024-04-28T13:33:34.005421+0000","last_active":"2024-04-28T13:34:04.880728+0000","last_peered":"2024-04-28T13:34:04.880728+0000","last_clean":"2024-04-28T13:34:04.880728+0000","last_became_active":"2024-04-28T13:33:34.004776+0000","last_became_peered":"2024-04-28T13:33:34.004776+0000","last_unstale":"2024-04-28T13:34:04.880728+0000","last_undegraded":"2024-04-28T13:34:04.880728+0000","last_fullsized":"2024-04-28T13:34:04.880728+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:20:07.691466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.356034+0000","last_change":"2024-04-28T13:33:32.991462+0000","last_active":"2024-04-28T13:34:04.356034+0000","last_peered":"2024-04-28T13:34:04.356034+0000","last_clean":"2024-04-28T13:34:04.356034+0000","last_became_active":"2024-04-28T13:33:32.991149+0000","last_became_peered":"2024-04-28T13:33:32.991149+0000","last_unstale":"2024-04-28T13:34:04.356034+0000","last_undegraded":"2024-04-28T13:34:04.356034+0000","last_fullsized":"2024-04-28T13:34:04.356034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:41.068187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":220,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349073+0000","last_change":"2024-04-28T13:31:11.831538+0000","last_active":"2024-04-28T13:34:04.349073+0000","last_peered":"2024-04-28T13:34:04.349073+0000","last_clean":"2024-04-28T13:34:04.349073+0000","last_became_active":"2024-04-28T13:31:11.828959+0000","last_became_peered":"2024-04-28T13:31:11.828959+0000","last_unstale":"2024-04-28T13:34:04.349073+0000","last_undegraded":"2024-04-28T13:34:04.349073+0000","last_fullsized":"2024-04-28T13:34:04.349073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_clean_scrub_stamp":"2024-04-28T13:31:10.444388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:25:27.078911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":4195,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343692,"statfs":{"total":96636764160,"available":96607940608,"internally_reserved":0,"allocated":1150976,"data_stored":818777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":561152,"data_stored":228991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":228389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":565248,"data_stored":231575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":565248,"data_stored":228957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":569344,"data_stored":231597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":228389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1150976,"data_stored":818721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T13:35:52.670 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:35:52.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.883+0000 7f3a31200640 1 Processor -- start 2024-04-28T13:35:52.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.883+0000 7f3a31200640 1 -- start start 2024-04-28T13:35:52.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.885+0000 7f3a31200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3a2c0d1b80 0x7f3a2c0d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:52.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.885+0000 7f3a31200640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3a2c0b60a0 0x7f3a2c0b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:52.887 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.885+0000 7f3a31200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a2c0d0220 con 0x7f3a2c0cfda0 2024-04-28T13:35:52.887 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.885+0000 7f3a31200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3a2c0d0500 con 0x7f3a2c0d1b80 2024-04-28T13:35:52.887 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.885+0000 7f3a31200640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3a2c0d07e0 con 0x7f3a2c0b60a0 2024-04-28T13:35:52.887 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.886+0000 7f3a29e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3a2c0d1b80 0x7f3a2c0d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:52.887 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.886+0000 7f3a23e00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f3a2c0cfda0 0x7f3a2c0d3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:53166/0 (socket says 172.21.15.196:53166) 2024-04-28T13:35:52.887 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.886+0000 7f3a23e00640 1 -- 172.21.15.196:0/2075368067 learned_addr learned my addr 172.21.15.196:0/2075368067 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:52.887 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.886+0000 7f3a29e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3a2c0d1b80 0x7f3a2c0d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:38964/0 (socket says 172.21.15.196:38964) 2024-04-28T13:35:52.887 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.886+0000 7f3a2a800640 1 --2- 172.21.15.196:0/2075368067 >> v2:172.21.15.203:3300/0 conn(0x7f3a2c0b60a0 0x7f3a2c0b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:52.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.886+0000 7f3a29e00640 1 -- 172.21.15.196:0/2075368067 >> v2:172.21.15.203:3300/0 conn(0x7f3a2c0b60a0 msgr2=0x7f3a2c0b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:52.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.886+0000 7f3a29e00640 1 --2- 172.21.15.196:0/2075368067 >> v2:172.21.15.203:3300/0 conn(0x7f3a2c0b60a0 0x7f3a2c0b64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.886+0000 7f3a29e00640 1 -- 172.21.15.196:0/2075368067 >> v1:172.21.15.196:6789/0 conn(0x7f3a2c0cfda0 legacy=0x7f3a2c0d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:52.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.886+0000 7f3a29400640 1 -- 172.21.15.196:0/2075368067 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 305291650 0 0) 0x7f3a2c0d0220 con 0x7f3a2c0cfda0 2024-04-28T13:35:52.888 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.886+0000 7f3a29e00640 1 -- 172.21.15.196:0/2075368067 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2c0d0ac0 con 0x7f3a2c0d1b80 2024-04-28T13:35:52.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.887+0000 7f3a29e00640 1 --2- 172.21.15.196:0/2075368067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a2c0d1b80 0x7f3a2c0d1f60 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c06ce80 tx=0x7f3a1c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3356d1aa449e4c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:52.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.888+0000 7f3a29400640 1 -- 172.21.15.196:0/2075368067 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c037070 con 0x7f3a2c0d1b80 2024-04-28T13:35:52.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.888+0000 7f3a29400640 1 -- 172.21.15.196:0/2075368067 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a1c032440 con 0x7f3a2c0d1b80 2024-04-28T13:35:52.889 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.888+0000 7f3a29400640 1 -- 172.21.15.196:0/2075368067 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c032810 con 0x7f3a2c0d1b80 2024-04-28T13:35:52.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.888+0000 7f3a31200640 1 -- 172.21.15.196:0/2075368067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a2c0d1b80 msgr2=0x7f3a2c0d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:52.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.888+0000 7f3a31200640 1 --2- 172.21.15.196:0/2075368067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a2c0d1b80 0x7f3a2c0d1f60 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c06ce80 tx=0x7f3a1c02a1a0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.888+0000 7f3a31200640 1 -- 172.21.15.196:0/2075368067 shutdown_connections 2024-04-28T13:35:52.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.888+0000 7f3a31200640 1 --2- 172.21.15.196:0/2075368067 >> v2:172.21.15.203:3300/0 conn(0x7f3a2c0b60a0 0x7f3a2c0b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.888+0000 7f3a31200640 1 --2- 172.21.15.196:0/2075368067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a2c0d1b80 0x7f3a2c0d1f60 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.888+0000 7f3a31200640 1 -- 172.21.15.196:0/2075368067 >> 172.21.15.196:0/2075368067 conn(0x7f3a2c09e540 msgr2=0x7f3a2c0d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:52.890 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.888+0000 7f3a31200640 1 -- 172.21.15.196:0/2075368067 shutdown_connections 2024-04-28T13:35:52.891 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.888+0000 7f3a31200640 1 -- 172.21.15.196:0/2075368067 wait complete. 2024-04-28T13:35:52.892 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.890+0000 7f3a31200640 1 Processor -- start 2024-04-28T13:35:52.893 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.890+0000 7f3a31200640 1 -- start start 2024-04-28T13:35:52.893 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.891+0000 7f3a31200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a2c0b60a0 0x7f3a2c14dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:52.893 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.891+0000 7f3a31200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a2c0cfda0 0x7f3a2c14e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:52.893 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.891+0000 7f3a31200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a2c0d1b80 0x7f3a2c150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:52.893 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.891+0000 7f3a31200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c0d4170 con 0x7f3a2c0d1b80 2024-04-28T13:35:52.893 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.891+0000 7f3a31200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c0b6a30 con 0x7f3a2c0b60a0 2024-04-28T13:35:52.893 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.891+0000 7f3a31200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3a2c15ee50 con 0x7f3a2c0cfda0 2024-04-28T13:35:52.893 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.891+0000 7f3a2a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a2c0d1b80 0x7f3a2c150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:52.893 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.891+0000 7f3a2a800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a2c0d1b80 0x7f3a2c150a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:38966/0 (socket says 172.21.15.196:38966) 2024-04-28T13:35:52.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.891+0000 7f3a2a800640 1 -- 172.21.15.196:0/1346815338 learned_addr learned my addr 172.21.15.196:0/1346815338 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:52.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.892+0000 7f3a23e00640 1 --2- 172.21.15.196:0/1346815338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a2c0cfda0 0x7f3a2c14e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:52.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.892+0000 7f3a29e00640 1 --2- 172.21.15.196:0/1346815338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a2c0b60a0 0x7f3a2c14dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:52.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.892+0000 7f3a2a800640 1 -- 172.21.15.196:0/1346815338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a2c0cfda0 msgr2=0x7f3a2c14e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:52.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.892+0000 7f3a2a800640 1 --2- 172.21.15.196:0/1346815338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a2c0cfda0 0x7f3a2c14e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.892+0000 7f3a2a800640 1 -- 172.21.15.196:0/1346815338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a2c0b60a0 msgr2=0x7f3a2c14dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:52.894 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.892+0000 7f3a2a800640 1 --2- 172.21.15.196:0/1346815338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a2c0b60a0 0x7f3a2c14dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:52.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.892+0000 7f3a2a800640 1 -- 172.21.15.196:0/1346815338 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2c154850 con 0x7f3a2c0d1b80 2024-04-28T13:35:52.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.892+0000 7f3a2a800640 1 --2- 172.21.15.196:0/1346815338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a2c0d1b80 0x7f3a2c150a70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400cd70 tx=0x7f3a2400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:52.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.892+0000 7f3a22000640 1 -- 172.21.15.196:0/1346815338 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24003b90 con 0x7f3a2c0d1b80 2024-04-28T13:35:52.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.892+0000 7f3a22000640 1 -- 172.21.15.196:0/1346815338 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a24003d30 con 0x7f3a2c0d1b80 2024-04-28T13:35:52.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.892+0000 7f3a31200640 1 -- 172.21.15.196:0/1346815338 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2c162400 con 0x7f3a2c0d1b80 2024-04-28T13:35:52.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.893+0000 7f3a22000640 1 -- 172.21.15.196:0/1346815338 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24004240 con 0x7f3a2c0d1b80 2024-04-28T13:35:52.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.893+0000 7f3a31200640 1 -- 172.21.15.196:0/1346815338 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a2c14ebf0 con 0x7f3a2c0d1b80 2024-04-28T13:35:52.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.894+0000 7f3a31200640 1 -- 172.21.15.196:0/1346815338 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a2c14e9e0 con 0x7f3a2c0d1b80 2024-04-28T13:35:52.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.895+0000 7f3a22000640 1 -- 172.21.15.196:0/1346815338 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f3a240043e0 con 0x7f3a2c0d1b80 2024-04-28T13:35:52.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.896+0000 7f3a22000640 1 --2- 172.21.15.196:0/1346815338 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3a0806c980 0x7f3a0806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:52.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.896+0000 7f3a22000640 1 -- 172.21.15.196:0/1346815338 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f3a2408c830 con 0x7f3a2c0d1b80 2024-04-28T13:35:52.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.896+0000 7f3a29e00640 1 --2- 172.21.15.196:0/1346815338 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3a0806c980 0x7f3a0806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:52.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.898+0000 7f3a29e00640 1 --2- 172.21.15.196:0/1346815338 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3a0806c980 0x7f3a0806ee40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c034000 tx=0x7f3a1c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:52.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:52.900+0000 7f3a22000640 1 -- 172.21.15.196:0/1346815338 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3a2401d340 con 0x7f3a2c0d1b80 2024-04-28T13:35:53.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:53.087+0000 7f3a31200640 1 -- 172.21.15.196:0/1346815338 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a2c06a290 con 0x7f3a2c0d1b80 2024-04-28T13:35:53.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:53.089+0000 7f3a22000640 1 -- 172.21.15.196:0/1346815338 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16741 (secure 0 0 0) 0x7f3a2405ab20 con 0x7f3a2c0d1b80 2024-04-28T13:35:53.089 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:35:53.089 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":47,"fsid":"270aea32-2e16-4a97-85f7-aeae39bc8c76","created":"2024-04-28T13:30:57.111301+0000","modified":"2024-04-28T13:34:04.329925+0000","last_up_change":"2024-04-28T13:31:09.397149+0000","last_in_change":"2024-04-28T13:31:00.393424+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T13:31:10.214495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-28T13:33:31.193265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-28T13:33:32.415392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8953d3d4-6285-4b7b-9a65-e32dc56d87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6801","nonce":3595094355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6803","nonce":3595094355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6808","nonce":3595094355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3595094355},{"type":"v1","addr":"172.21.15.196:6805","nonce":3595094355}]},"public_addr":"172.21.15.196:6801/3595094355","cluster_addr":"172.21.15.196:6803/3595094355","heartbeat_back_addr":"172.21.15.196:6808/3595094355","heartbeat_front_addr":"172.21.15.196:6805/3595094355","state":["exists","up"]},{"osd":1,"uuid":"1f041afc-6cd2-4e7f-a1a4-eb0ffce15749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6825","nonce":2304700074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6827","nonce":2304700074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6831","nonce":2304700074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2304700074},{"type":"v1","addr":"172.21.15.196:6829","nonce":2304700074}]},"public_addr":"172.21.15.196:6825/2304700074","cluster_addr":"172.21.15.196:6827/2304700074","heartbeat_back_addr":"172.21.15.196:6831/2304700074","heartbeat_front_addr":"172.21.15.196:6829/2304700074","state":["exists","up"]},{"osd":2,"uuid":"c52ac708-974d-4c56-834b-c384bb41afac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6809","nonce":377093821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6811","nonce":377093821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6816","nonce":377093821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":377093821},{"type":"v1","addr":"172.21.15.196:6813","nonce":377093821}]},"public_addr":"172.21.15.196:6809/377093821","cluster_addr":"172.21.15.196:6811/377093821","heartbeat_back_addr":"172.21.15.196:6816/377093821","heartbeat_front_addr":"172.21.15.196:6813/377093821","state":["exists","up"]},{"osd":3,"uuid":"6cad7e98-14c8-4217-a1ad-c1c1378c0c1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6815","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6817","nonce":176565235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6819","nonce":176565235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6823","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6824","nonce":176565235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":176565235},{"type":"v1","addr":"172.21.15.196:6821","nonce":176565235}]},"public_addr":"172.21.15.196:6817/176565235","cluster_addr":"172.21.15.196:6819/176565235","heartbeat_back_addr":"172.21.15.196:6824/176565235","heartbeat_front_addr":"172.21.15.196:6821/176565235","state":["exists","up"]},{"osd":4,"uuid":"2092249e-3aa5-4a4b-a5e9-13843e365ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6801","nonce":1325487575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6806","nonce":1325487575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6816","nonce":1325487575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1325487575},{"type":"v1","addr":"172.21.15.203:6811","nonce":1325487575}]},"public_addr":"172.21.15.203:6801/1325487575","cluster_addr":"172.21.15.203:6806/1325487575","heartbeat_back_addr":"172.21.15.203:6816/1325487575","heartbeat_front_addr":"172.21.15.203:6811/1325487575","state":["exists","up"]},{"osd":5,"uuid":"2201e71b-0237-4734-818e-a6a9725d1f80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6825","nonce":2058013113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6827","nonce":2058013113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6831","nonce":2058013113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2058013113},{"type":"v1","addr":"172.21.15.203:6829","nonce":2058013113}]},"public_addr":"172.21.15.203:6825/2058013113","cluster_addr":"172.21.15.203:6827/2058013113","heartbeat_back_addr":"172.21.15.203:6831/2058013113","heartbeat_front_addr":"172.21.15.203:6829/2058013113","state":["exists","up"]},{"osd":6,"uuid":"a0a083ff-9305-48ca-b1c2-5d5f6736a47f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6807","nonce":2209636555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6812","nonce":2209636555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6821","nonce":2209636555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":2209636555},{"type":"v1","addr":"172.21.15.203:6817","nonce":2209636555}]},"public_addr":"172.21.15.203:6807/2209636555","cluster_addr":"172.21.15.203:6812/2209636555","heartbeat_back_addr":"172.21.15.203:6821/2209636555","heartbeat_front_addr":"172.21.15.203:6817/2209636555","state":["exists","up"]},{"osd":7,"uuid":"eed708bd-0a80-4194-ad72-435b034cf05d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6805","nonce":2023229662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6814","nonce":2023229662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6823","nonce":2023229662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2023229662},{"type":"v1","addr":"172.21.15.203:6820","nonce":2023229662}]},"public_addr":"172.21.15.203:6805/2023229662","cluster_addr":"172.21.15.203:6814/2023229662","heartbeat_back_addr":"172.21.15.203:6823/2023229662","heartbeat_front_addr":"172.21.15.203:6820/2023229662","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.806837+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.493802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.814047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:05.546284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.018531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.012368+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.072238+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T13:31:06.063458+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.32","mappings":[{"from":0,"to":5}]},{"pgid":"7.38","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:35:53.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:53.092+0000 7f3a31200640 1 -- 172.21.15.196:0/1346815338 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3a0806c980 msgr2=0x7f3a0806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:53.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:53.092+0000 7f3a31200640 1 --2- 172.21.15.196:0/1346815338 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3a0806c980 0x7f3a0806ee40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c034000 tx=0x7f3a1c02e000 comp rx=0 tx=0).stop 2024-04-28T13:35:53.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:53.092+0000 7f3a31200640 1 -- 172.21.15.196:0/1346815338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a2c0d1b80 msgr2=0x7f3a2c150a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:53.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:53.092+0000 7f3a31200640 1 --2- 172.21.15.196:0/1346815338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a2c0d1b80 0x7f3a2c150a70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400cd70 tx=0x7f3a2400a660 comp rx=0 tx=0).stop 2024-04-28T13:35:53.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:53.092+0000 7f3a31200640 1 -- 172.21.15.196:0/1346815338 shutdown_connections 2024-04-28T13:35:53.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:53.092+0000 7f3a31200640 1 --2- 172.21.15.196:0/1346815338 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f3a0806c980 0x7f3a0806ee40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:53.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:53.092+0000 7f3a31200640 1 --2- 172.21.15.196:0/1346815338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a2c0d1b80 0x7f3a2c150a70 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:53.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:53.092+0000 7f3a31200640 1 --2- 172.21.15.196:0/1346815338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a2c0cfda0 0x7f3a2c14e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:53.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:53.092+0000 7f3a31200640 1 --2- 172.21.15.196:0/1346815338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a2c0b60a0 0x7f3a2c14dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:53.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:53.092+0000 7f3a31200640 1 -- 172.21.15.196:0/1346815338 >> 172.21.15.196:0/1346815338 conn(0x7f3a2c09e540 msgr2=0x7f3a2c0ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:53.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:53.092+0000 7f3a31200640 1 -- 172.21.15.196:0/1346815338 shutdown_connections 2024-04-28T13:35:53.093 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:53.092+0000 7f3a31200640 1 -- 172.21.15.196:0/1346815338 wait complete. 2024-04-28T13:35:54.109 INFO:tasks.ceph:Scrubbing osd.0 2024-04-28T13:35:54.110 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-28T13:35:54.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.328+0000 7f9e5ea00640 1 Processor -- start 2024-04-28T13:35:54.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.328+0000 7f9e5ea00640 1 -- start start 2024-04-28T13:35:54.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.330+0000 7f9e5ea00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9e5815ef00 0x7f9e580b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.330+0000 7f9e5ea00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9e580d1050 0x7f9e580d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.330+0000 7f9e5ea00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e5806dc70 con 0x7f9e5815f2e0 2024-04-28T13:35:54.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.330+0000 7f9e5ea00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9e580d1a00 con 0x7f9e580d1050 2024-04-28T13:35:54.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.330+0000 7f9e5ea00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9e5806df50 con 0x7f9e5815ef00 2024-04-28T13:35:54.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.330+0000 7f9e5e000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9e580d1050 0x7f9e580d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:54.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.331+0000 7f9e5e000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9e580d1050 0x7f9e580d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:38980/0 (socket says 172.21.15.196:38980) 2024-04-28T13:35:54.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.331+0000 7f9e5e000640 1 -- 172.21.15.196:0/1372730377 learned_addr learned my addr 172.21.15.196:0/1372730377 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:54.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.331+0000 7f9e5d600640 1 --2- 172.21.15.196:0/1372730377 >> v2:172.21.15.203:3300/0 conn(0x7f9e5815ef00 0x7f9e580b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:54.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.331+0000 7f9e5e000640 1 -- 172.21.15.196:0/1372730377 >> v2:172.21.15.203:3300/0 conn(0x7f9e5815ef00 msgr2=0x7f9e580b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:54.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.331+0000 7f9e5e000640 1 --2- 172.21.15.196:0/1372730377 >> v2:172.21.15.203:3300/0 conn(0x7f9e5815ef00 0x7f9e580b5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.331+0000 7f9e5e000640 1 -- 172.21.15.196:0/1372730377 >> v1:172.21.15.203:6789/0 conn(0x7f9e5815f2e0 legacy=0x7f9e5806f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:54.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.331+0000 7f9e5e000640 1 -- 172.21.15.196:0/1372730377 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5806e230 con 0x7f9e580d1050 2024-04-28T13:35:54.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.332+0000 7f9e5e000640 1 --2- 172.21.15.196:0/1372730377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e580d1050 0x7f9e580d14c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9e58068fc0 tx=0x7f9e5002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec4d1cefc42ea3b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:54.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.332+0000 7f9e57e00640 1 -- 172.21.15.196:0/1372730377 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50036070 con 0x7f9e580d1050 2024-04-28T13:35:54.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.332+0000 7f9e57e00640 1 -- 172.21.15.196:0/1372730377 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e50031600 con 0x7f9e580d1050 2024-04-28T13:35:54.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.332+0000 7f9e57e00640 1 -- 172.21.15.196:0/1372730377 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e500319d0 con 0x7f9e580d1050 2024-04-28T13:35:54.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.333+0000 7f9e5ea00640 1 -- 172.21.15.196:0/1372730377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e580d1050 msgr2=0x7f9e580d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:54.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.333+0000 7f9e5ea00640 1 --2- 172.21.15.196:0/1372730377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e580d1050 0x7f9e580d14c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9e58068fc0 tx=0x7f9e5002a660 comp rx=0 tx=0).stop 2024-04-28T13:35:54.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.333+0000 7f9e5ea00640 1 -- 172.21.15.196:0/1372730377 shutdown_connections 2024-04-28T13:35:54.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.333+0000 7f9e5ea00640 1 --2- 172.21.15.196:0/1372730377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e580d1050 0x7f9e580d14c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.333+0000 7f9e5ea00640 1 --2- 172.21.15.196:0/1372730377 >> v2:172.21.15.203:3300/0 conn(0x7f9e5815ef00 0x7f9e580b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.333+0000 7f9e5ea00640 1 -- 172.21.15.196:0/1372730377 >> 172.21.15.196:0/1372730377 conn(0x7f9e5806c6c0 msgr2=0x7f9e5806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:54.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.333+0000 7f9e5ea00640 1 -- 172.21.15.196:0/1372730377 shutdown_connections 2024-04-28T13:35:54.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.333+0000 7f9e5ea00640 1 -- 172.21.15.196:0/1372730377 wait complete. 2024-04-28T13:35:54.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.335+0000 7f9e5ea00640 1 Processor -- start 2024-04-28T13:35:54.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.335+0000 7f9e5ea00640 1 -- start start 2024-04-28T13:35:54.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.336+0000 7f9e5ea00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e580d1050 0x7f9e581499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.336+0000 7f9e5ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e5815ef00 0x7f9e58155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.336+0000 7f9e5ea00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9e5815f2e0 0x7f9e581614d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.336+0000 7f9e5ea00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58072280 con 0x7f9e580d1050 2024-04-28T13:35:54.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.336+0000 7f9e5ea00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58072100 con 0x7f9e5815ef00 2024-04-28T13:35:54.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.336+0000 7f9e5ea00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9e58072400 con 0x7f9e5815f2e0 2024-04-28T13:35:54.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.336+0000 7f9e5d600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e580d1050 0x7f9e581499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:54.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.336+0000 7f9e5d600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e580d1050 0x7f9e581499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:38986/0 (socket says 172.21.15.196:38986) 2024-04-28T13:35:54.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.336+0000 7f9e5d600640 1 -- 172.21.15.196:0/2767642114 learned_addr learned my addr 172.21.15.196:0/2767642114 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:54.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.337+0000 7f9e5d600640 1 -- 172.21.15.196:0/2767642114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9e5815f2e0 msgr2=0x7f9e581614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:54.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.337+0000 7f9e5d600640 1 --2- 172.21.15.196:0/2767642114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9e5815f2e0 0x7f9e581614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.337+0000 7f9e5d600640 1 -- 172.21.15.196:0/2767642114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e5815ef00 msgr2=0x7f9e58155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:54.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.337+0000 7f9e5d600640 1 --2- 172.21.15.196:0/2767642114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e5815ef00 0x7f9e58155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.337+0000 7f9e5d600640 1 -- 172.21.15.196:0/2767642114 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5809bbc0 con 0x7f9e580d1050 2024-04-28T13:35:54.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.337+0000 7f9e5d600640 1 --2- 172.21.15.196:0/2767642114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e580d1050 0x7f9e581499d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00c7f0 tx=0x7f9e4c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:54.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.337+0000 7f9e55600640 1 -- 172.21.15.196:0/2767642114 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c015070 con 0x7f9e580d1050 2024-04-28T13:35:54.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.337+0000 7f9e55600640 1 -- 172.21.15.196:0/2767642114 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4c004240 con 0x7f9e580d1050 2024-04-28T13:35:54.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.337+0000 7f9e55600640 1 -- 172.21.15.196:0/2767642114 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c007620 con 0x7f9e580d1050 2024-04-28T13:35:54.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.337+0000 7f9e5ea00640 1 -- 172.21.15.196:0/2767642114 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5809bea0 con 0x7f9e580d1050 2024-04-28T13:35:54.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.337+0000 7f9e5ea00640 1 -- 172.21.15.196:0/2767642114 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5809c460 con 0x7f9e580d1050 2024-04-28T13:35:54.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.338+0000 7f9e5ea00640 1 -- 172.21.15.196:0/2767642114 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e58050730 con 0x7f9e580d1050 2024-04-28T13:35:54.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.339+0000 7f9e55600640 1 -- 172.21.15.196:0/2767642114 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f9e4c0079d0 con 0x7f9e580d1050 2024-04-28T13:35:54.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.340+0000 7f9e55600640 1 --2- 172.21.15.196:0/2767642114 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9e3806c9b0 0x7f9e3806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.340+0000 7f9e55600640 1 -- 172.21.15.196:0/2767642114 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f9e4c08bef0 con 0x7f9e580d1050 2024-04-28T13:35:54.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.341+0000 7f9e55600640 1 --2- 172.21.15.196:0/2767642114 >> [v2:172.21.15.196:6800/3595094355,v1:172.21.15.196:6801/3595094355] conn(0x7f9e38074c10 0x7f9e38077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.341+0000 7f9e55600640 1 -- 172.21.15.196:0/2767642114 --> [v2:172.21.15.196:6800/3595094355,v1:172.21.15.196:6801/3595094355] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e38077720 con 0x7f9e38074c10 2024-04-28T13:35:54.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.341+0000 7f9e55600640 1 -- 172.21.15.196:0/2767642114 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9e4c08e510 con 0x7f9e580d1050 2024-04-28T13:35:54.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.341+0000 7f9e5cc00640 1 --2- 172.21.15.196:0/2767642114 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9e3806c9b0 0x7f9e3806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:54.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.341+0000 7f9e5e000640 1 --2- 172.21.15.196:0/2767642114 >> [v2:172.21.15.196:6800/3595094355,v1:172.21.15.196:6801/3595094355] conn(0x7f9e38074c10 0x7f9e38077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:54.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.342+0000 7f9e5e000640 1 --2- 172.21.15.196:0/2767642114 >> [v2:172.21.15.196:6800/3595094355,v1:172.21.15.196:6801/3595094355] conn(0x7f9e38074c10 0x7f9e38077050 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:54.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.342+0000 7f9e5cc00640 1 --2- 172.21.15.196:0/2767642114 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9e3806c9b0 0x7f9e3806ee70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9e58162780 tx=0x7f9e44006ba0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:54.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.343+0000 7f9e55600640 1 -- 172.21.15.196:0/2767642114 <== osd.0 v2:172.21.15.196:6800/3595094355 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e38077720 con 0x7f9e38074c10 2024-04-28T13:35:54.356 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.357+0000 7f9e5ea00640 1 -- 172.21.15.196:0/2767642114 --> [v2:172.21.15.196:6800/3595094355,v1:172.21.15.196:6801/3595094355] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9e581629e0 con 0x7f9e38074c10 2024-04-28T13:35:54.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.357+0000 7f9e55600640 1 -- 172.21.15.196:0/2767642114 <== osd.0 v2:172.21.15.196:6800/3595094355 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9e581629e0 con 0x7f9e38074c10 2024-04-28T13:35:54.357 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:35:54.357 INFO:teuthology.orchestra.run.smithi196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T13:35:54.357 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:35:54.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.357+0000 7f9e5ea00640 1 -- 172.21.15.196:0/2767642114 >> [v2:172.21.15.196:6800/3595094355,v1:172.21.15.196:6801/3595094355] conn(0x7f9e38074c10 msgr2=0x7f9e38077050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:54.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.357+0000 7f9e5ea00640 1 --2- 172.21.15.196:0/2767642114 >> [v2:172.21.15.196:6800/3595094355,v1:172.21.15.196:6801/3595094355] conn(0x7f9e38074c10 0x7f9e38077050 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.358+0000 7f9e5ea00640 1 -- 172.21.15.196:0/2767642114 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9e3806c9b0 msgr2=0x7f9e3806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:54.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.358+0000 7f9e5ea00640 1 --2- 172.21.15.196:0/2767642114 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f9e3806c9b0 0x7f9e3806ee70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9e58162780 tx=0x7f9e44006ba0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.358+0000 7f9e5ea00640 1 -- 172.21.15.196:0/2767642114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e580d1050 msgr2=0x7f9e581499d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:54.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.358+0000 7f9e5ea00640 1 --2- 172.21.15.196:0/2767642114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e580d1050 0x7f9e581499d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00c7f0 tx=0x7f9e4c00ccb0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.358+0000 7f9e5e000640 1 -- 172.21.15.196:0/2767642114 reap_dead start 2024-04-28T13:35:54.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.358+0000 7f9e5ea00640 1 -- 172.21.15.196:0/2767642114 shutdown_connections 2024-04-28T13:35:54.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.358+0000 7f9e5ea00640 1 -- 172.21.15.196:0/2767642114 >> 172.21.15.196:0/2767642114 conn(0x7f9e5806c6c0 msgr2=0x7f9e5806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:54.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.358+0000 7f9e5ea00640 1 -- 172.21.15.196:0/2767642114 shutdown_connections 2024-04-28T13:35:54.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.358+0000 7f9e5ea00640 1 -- 172.21.15.196:0/2767642114 wait complete. 2024-04-28T13:35:54.374 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-28T13:35:54.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.560+0000 7fcac3e00640 1 Processor -- start 2024-04-28T13:35:54.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.560+0000 7fcac3e00640 1 -- start start 2024-04-28T13:35:54.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.561+0000 7fcac3e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcac415f2f0 0x7fcac406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.561+0000 7fcac3e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcac40cec20 0x7fcac406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.561+0000 7fcac3e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcac415f6d0 con 0x7fcac406ddc0 2024-04-28T13:35:54.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.561+0000 7fcac3e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fcac406db00 con 0x7fcac415f2f0 2024-04-28T13:35:54.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.561+0000 7fcac3e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcac40d1da0 con 0x7fcac40cec20 2024-04-28T13:35:54.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.562+0000 7fcac2000640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fcac406ddc0 0x7fcac40cf5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:53176/0 (socket says 172.21.15.196:53176) 2024-04-28T13:35:54.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.562+0000 7fcac2000640 1 -- 172.21.15.196:0/1438601271 learned_addr learned my addr 172.21.15.196:0/1438601271 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:54.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.562+0000 7fcac2a00640 1 --2- 172.21.15.196:0/1438601271 >> v2:172.21.15.203:3300/0 conn(0x7fcac415f2f0 0x7fcac406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:54.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.562+0000 7fcac1600640 1 -- 172.21.15.196:0/1438601271 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2566939672 0 0) 0x7fcac415f6d0 con 0x7fcac406ddc0 2024-04-28T13:35:54.564 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.562+0000 7fcac3400640 1 --2- 172.21.15.196:0/1438601271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcac40cec20 0x7fcac406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:54.564 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.562+0000 7fcac1600640 1 -- 172.21.15.196:0/1438601271 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcaa8003520 con 0x7fcac406ddc0 2024-04-28T13:35:54.564 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.562+0000 7fcac1600640 1 -- 172.21.15.196:0/1438601271 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3890713479 0 0) 0x7fcaa8003520 con 0x7fcac406ddc0 2024-04-28T13:35:54.564 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.562+0000 7fcac1600640 1 -- 172.21.15.196:0/1438601271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcac40cec20 msgr2=0x7fcac406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:54.564 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.562+0000 7fcac1600640 1 --2- 172.21.15.196:0/1438601271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcac40cec20 0x7fcac406e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.564 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.562+0000 7fcac1600640 1 -- 172.21.15.196:0/1438601271 >> v2:172.21.15.203:3300/0 conn(0x7fcac415f2f0 msgr2=0x7fcac406d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:54.564 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.562+0000 7fcac1600640 1 --2- 172.21.15.196:0/1438601271 >> v2:172.21.15.203:3300/0 conn(0x7fcac415f2f0 0x7fcac406d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.564 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.562+0000 7fcac1600640 1 -- 172.21.15.196:0/1438601271 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac40d2080 con 0x7fcac406ddc0 2024-04-28T13:35:54.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.562+0000 7fcac1600640 1 -- 172.21.15.196:0/1438601271 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fcab0001690 con 0x7fcac406ddc0 2024-04-28T13:35:54.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.562+0000 7fcac1600640 1 -- 172.21.15.196:0/1438601271 >> v1:172.21.15.196:6789/0 conn(0x7fcac406ddc0 legacy=0x7fcac40cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:54.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.563+0000 7fcac1600640 1 --2- 172.21.15.196:0/1438601271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaa80039b0 0x7fcaa8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.563+0000 7fcac1600640 1 --2- 172.21.15.196:0/1438601271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaa8004800 0x7fcaa8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.563+0000 7fcac1600640 1 --2- 172.21.15.196:0/1438601271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcac40cec20 0x7fcaa80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.563+0000 7fcac1600640 1 -- 172.21.15.196:0/1438601271 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcac406db00 con 0x7fcaa8004800 2024-04-28T13:35:54.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.563+0000 7fcac1600640 1 -- 172.21.15.196:0/1438601271 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcac40d1da0 con 0x7fcaa80039b0 2024-04-28T13:35:54.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.563+0000 7fcac1600640 1 -- 172.21.15.196:0/1438601271 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa800bac0 con 0x7fcac40cec20 2024-04-28T13:35:54.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.563+0000 7fcac2000640 1 --2- 172.21.15.196:0/1438601271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaa8004800 0x7fcaa8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:54.565 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.563+0000 7fcac2a00640 1 --2- 172.21.15.196:0/1438601271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaa80039b0 0x7fcaa8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:54.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.563+0000 7fcac2000640 1 -- 172.21.15.196:0/1438601271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcac40cec20 msgr2=0x7fcaa80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:54.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.563+0000 7fcac2000640 1 --2- 172.21.15.196:0/1438601271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcac40cec20 0x7fcaa80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.563+0000 7fcac2000640 1 -- 172.21.15.196:0/1438601271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaa80039b0 msgr2=0x7fcaa8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:54.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.563+0000 7fcac2000640 1 --2- 172.21.15.196:0/1438601271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaa80039b0 0x7fcaa8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.563+0000 7fcac2000640 1 -- 172.21.15.196:0/1438601271 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa8003520 con 0x7fcaa8004800 2024-04-28T13:35:54.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.564+0000 7fcac2000640 1 --2- 172.21.15.196:0/1438601271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaa8004800 0x7fcaa8004c80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fcac4069070 tx=0x7fcaa8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dfa507ae1935d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:54.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.564+0000 7fcac1600640 1 -- 172.21.15.196:0/1438601271 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0007ca0 con 0x7fcaa8004800 2024-04-28T13:35:54.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.564+0000 7fcac1600640 1 -- 172.21.15.196:0/1438601271 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcab0005ad0 con 0x7fcaa8004800 2024-04-28T13:35:54.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.564+0000 7fcac1600640 1 -- 172.21.15.196:0/1438601271 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0007070 con 0x7fcaa8004800 2024-04-28T13:35:54.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.565+0000 7fcac3e00640 1 -- 172.21.15.196:0/1438601271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaa8004800 msgr2=0x7fcaa8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:54.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.565+0000 7fcac3e00640 1 --2- 172.21.15.196:0/1438601271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaa8004800 0x7fcaa8004c80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fcac4069070 tx=0x7fcaa8005830 comp rx=0 tx=0).stop 2024-04-28T13:35:54.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.565+0000 7fcac3400640 1 -- 172.21.15.196:0/1438601271 reap_dead start 2024-04-28T13:35:54.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.565+0000 7fcac3e00640 1 -- 172.21.15.196:0/1438601271 shutdown_connections 2024-04-28T13:35:54.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.565+0000 7fcac3e00640 1 -- 172.21.15.196:0/1438601271 >> 172.21.15.196:0/1438601271 conn(0x7fcac40b4830 msgr2=0x7fcac40b4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:54.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.565+0000 7fcac3e00640 1 -- 172.21.15.196:0/1438601271 shutdown_connections 2024-04-28T13:35:54.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.565+0000 7fcac3e00640 1 -- 172.21.15.196:0/1438601271 wait complete. 2024-04-28T13:35:54.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.567+0000 7fcac3e00640 1 Processor -- start 2024-04-28T13:35:54.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.567+0000 7fcac3e00640 1 -- start start 2024-04-28T13:35:54.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.568+0000 7fcac3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaa8004800 0x7fcac40cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.568+0000 7fcac3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcac409e580 0x7fcac409cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.568+0000 7fcac3e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcac4150d50 0x7fcac4153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.568+0000 7fcac3e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcac40cd6f0 con 0x7fcaa8004800 2024-04-28T13:35:54.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.568+0000 7fcac3e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcac40cd410 con 0x7fcac409e580 2024-04-28T13:35:54.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.568+0000 7fcac3e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcac40b5550 con 0x7fcac4150d50 2024-04-28T13:35:54.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.568+0000 7fcac2a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaa8004800 0x7fcac40cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:54.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.568+0000 7fcac2a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaa8004800 0x7fcac40cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39002/0 (socket says 172.21.15.196:39002) 2024-04-28T13:35:54.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.568+0000 7fcac2a00640 1 -- 172.21.15.196:0/800858221 learned_addr learned my addr 172.21.15.196:0/800858221 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:54.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.569+0000 7fcac3400640 1 --2- 172.21.15.196:0/800858221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcac4150d50 0x7fcac4153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:54.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.569+0000 7fcac2000640 1 --2- 172.21.15.196:0/800858221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcac409e580 0x7fcac409cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:54.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.569+0000 7fcac2a00640 1 -- 172.21.15.196:0/800858221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcac4150d50 msgr2=0x7fcac4153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:54.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.569+0000 7fcac2a00640 1 --2- 172.21.15.196:0/800858221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcac4150d50 0x7fcac4153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.569+0000 7fcac2a00640 1 -- 172.21.15.196:0/800858221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcac409e580 msgr2=0x7fcac409cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:54.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.569+0000 7fcac2a00640 1 --2- 172.21.15.196:0/800858221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcac409e580 0x7fcac409cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.569+0000 7fcac2a00640 1 -- 172.21.15.196:0/800858221 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac415d9f0 con 0x7fcaa8004800 2024-04-28T13:35:54.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.569+0000 7fcac2a00640 1 --2- 172.21.15.196:0/800858221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaa8004800 0x7fcac40cc8e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcab400e810 tx=0x7fcab400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:54.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.569+0000 7fcabaa00640 1 -- 172.21.15.196:0/800858221 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4017070 con 0x7fcaa8004800 2024-04-28T13:35:54.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.569+0000 7fcabaa00640 1 -- 172.21.15.196:0/800858221 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcab400c6d0 con 0x7fcaa8004800 2024-04-28T13:35:54.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.569+0000 7fcabaa00640 1 -- 172.21.15.196:0/800858221 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab400f040 con 0x7fcaa8004800 2024-04-28T13:35:54.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.569+0000 7fcac3e00640 1 -- 172.21.15.196:0/800858221 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac4151840 con 0x7fcaa8004800 2024-04-28T13:35:54.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.569+0000 7fcac3e00640 1 -- 172.21.15.196:0/800858221 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac4151e00 con 0x7fcaa8004800 2024-04-28T13:35:54.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.571+0000 7fcac3e00640 1 -- 172.21.15.196:0/800858221 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac409e960 con 0x7fcaa8004800 2024-04-28T13:35:54.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.572+0000 7fcabaa00640 1 -- 172.21.15.196:0/800858221 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fcab401a070 con 0x7fcaa8004800 2024-04-28T13:35:54.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.572+0000 7fcabaa00640 1 --2- 172.21.15.196:0/800858221 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcaa406c9b0 0x7fcaa406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:54.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.572+0000 7fcabaa00640 1 -- 172.21.15.196:0/800858221 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fcab408b8c0 con 0x7fcaa8004800 2024-04-28T13:35:54.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.573+0000 7fcac2000640 1 --2- 172.21.15.196:0/800858221 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcaa406c9b0 0x7fcaa406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:54.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.574+0000 7fcac2000640 1 --2- 172.21.15.196:0/800858221 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcaa406c9b0 0x7fcaa406ee70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcab0032000 tx=0x7fcab0009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:54.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.577+0000 7fcabaa00640 1 -- 172.21.15.196:0/800858221 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcac415d9f0 con 0x7fcaa8004800 2024-04-28T13:35:54.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.774+0000 7fcac3e00640 1 -- 172.21.15.196:0/800858221 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fcac406a290 con 0x7fcaa406c9b0 2024-04-28T13:35:54.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.776+0000 7fcabaa00640 1 -- 172.21.15.196:0/800858221 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcac406a290 con 0x7fcaa406c9b0 2024-04-28T13:35:54.775 INFO:teuthology.orchestra.run.smithi196.stderr:instructed osd(s) 0 to deep-scrub 2024-04-28T13:35:54.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.778+0000 7fcac3e00640 1 -- 172.21.15.196:0/800858221 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcaa406c9b0 msgr2=0x7fcaa406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:54.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.778+0000 7fcac3e00640 1 --2- 172.21.15.196:0/800858221 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcaa406c9b0 0x7fcaa406ee70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcab0032000 tx=0x7fcab0009030 comp rx=0 tx=0).stop 2024-04-28T13:35:54.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.778+0000 7fcac3e00640 1 -- 172.21.15.196:0/800858221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaa8004800 msgr2=0x7fcac40cc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:54.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.778+0000 7fcac3e00640 1 --2- 172.21.15.196:0/800858221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaa8004800 0x7fcac40cc8e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcab400e810 tx=0x7fcab400ecd0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.778+0000 7fcac3e00640 1 -- 172.21.15.196:0/800858221 shutdown_connections 2024-04-28T13:35:54.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.778+0000 7fcac3e00640 1 --2- 172.21.15.196:0/800858221 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fcaa406c9b0 0x7fcaa406ee70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.778+0000 7fcac3e00640 1 --2- 172.21.15.196:0/800858221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcac4150d50 0x7fcac4153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.778+0000 7fcac3e00640 1 --2- 172.21.15.196:0/800858221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcac409e580 0x7fcac409cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.778+0000 7fcac3e00640 1 --2- 172.21.15.196:0/800858221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaa8004800 0x7fcac40cc8e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:54.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.778+0000 7fcac3e00640 1 -- 172.21.15.196:0/800858221 >> 172.21.15.196:0/800858221 conn(0x7fcac40b4830 msgr2=0x7fcac40b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:54.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.779+0000 7fcac3e00640 1 -- 172.21.15.196:0/800858221 shutdown_connections 2024-04-28T13:35:54.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.779+0000 7fcac3e00640 1 -- 172.21.15.196:0/800858221 wait complete. 2024-04-28T13:35:54.796 INFO:tasks.ceph:Scrubbing osd.1 2024-04-28T13:35:54.796 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-28T13:35:54.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:54.999+0000 7fa53b600640 1 Processor -- start 2024-04-28T13:35:54.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.000+0000 7fa53b600640 1 -- start start 2024-04-28T13:35:55.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.001+0000 7fa53b600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa53406f600 0x7fa534160e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.001+0000 7fa53b600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa5340d1190 0x7fa5340d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.001+0000 7fa53b600640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa534161570 con 0x7fa5341437f0 2024-04-28T13:35:55.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.001+0000 7fa53b600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa534073440 con 0x7fa53406f600 2024-04-28T13:35:55.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.001+0000 7fa53b600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa534161850 con 0x7fa5340d1190 2024-04-28T13:35:55.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.002+0000 7fa533e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa53406f600 0x7fa534160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.002+0000 7fa533e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa53406f600 0x7fa534160e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39018/0 (socket says 172.21.15.196:39018) 2024-04-28T13:35:55.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.002+0000 7fa533e00640 1 -- 172.21.15.196:0/3016749940 learned_addr learned my addr 172.21.15.196:0/3016749940 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:55.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.002+0000 7fa539600640 1 --2- 172.21.15.196:0/3016749940 >> v2:172.21.15.203:3300/0 conn(0x7fa5340d1190 0x7fa5340d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.002+0000 7fa533e00640 1 -- 172.21.15.196:0/3016749940 >> v2:172.21.15.203:3300/0 conn(0x7fa5340d1190 msgr2=0x7fa5340d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:55.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.002+0000 7fa533e00640 1 --2- 172.21.15.196:0/3016749940 >> v2:172.21.15.203:3300/0 conn(0x7fa5340d1190 0x7fa5340d1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.002+0000 7fa533e00640 1 -- 172.21.15.196:0/3016749940 >> v1:172.21.15.203:6789/0 conn(0x7fa5341437f0 legacy=0x7fa534144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:55.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.002+0000 7fa533e00640 1 -- 172.21.15.196:0/3016749940 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa534161b30 con 0x7fa53406f600 2024-04-28T13:35:55.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.003+0000 7fa533e00640 1 --2- 172.21.15.196:0/3016749940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa53406f600 0x7fa534160e70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa5340b0a90 tx=0x7fa52402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c0ce9c9b7c941 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:55.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.003+0000 7fa533400640 1 -- 172.21.15.196:0/3016749940 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524036070 con 0x7fa53406f600 2024-04-28T13:35:55.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.003+0000 7fa533400640 1 -- 172.21.15.196:0/3016749940 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa524031600 con 0x7fa53406f600 2024-04-28T13:35:55.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.003+0000 7fa533400640 1 -- 172.21.15.196:0/3016749940 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240319d0 con 0x7fa53406f600 2024-04-28T13:35:55.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.004+0000 7fa53b600640 1 -- 172.21.15.196:0/3016749940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa53406f600 msgr2=0x7fa534160e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.004+0000 7fa53b600640 1 --2- 172.21.15.196:0/3016749940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa53406f600 0x7fa534160e70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa5340b0a90 tx=0x7fa52402a6f0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.004+0000 7fa53b600640 1 -- 172.21.15.196:0/3016749940 shutdown_connections 2024-04-28T13:35:55.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.004+0000 7fa53b600640 1 --2- 172.21.15.196:0/3016749940 >> v2:172.21.15.203:3300/0 conn(0x7fa5340d1190 0x7fa5340d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.004+0000 7fa53b600640 1 --2- 172.21.15.196:0/3016749940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa53406f600 0x7fa534160e70 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.004+0000 7fa53b600640 1 -- 172.21.15.196:0/3016749940 >> 172.21.15.196:0/3016749940 conn(0x7fa53406c690 msgr2=0x7fa53406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:55.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.005+0000 7fa53b600640 1 -- 172.21.15.196:0/3016749940 shutdown_connections 2024-04-28T13:35:55.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.005+0000 7fa53b600640 1 -- 172.21.15.196:0/3016749940 wait complete. 2024-04-28T13:35:55.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.007+0000 7fa53b600640 1 Processor -- start 2024-04-28T13:35:55.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.007+0000 7fa53b600640 1 -- start start 2024-04-28T13:35:55.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.008+0000 7fa53b600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa53406f600 0x7fa534142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.008+0000 7fa53b600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5340d1190 0x7fa53415a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.008+0000 7fa53b600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5341437f0 0x7fa5340af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.008+0000 7fa53b600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa534143d50 con 0x7fa5340d1190 2024-04-28T13:35:55.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.008+0000 7fa53b600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa534143bd0 con 0x7fa5341437f0 2024-04-28T13:35:55.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.008+0000 7fa53b600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa534073bc0 con 0x7fa53406f600 2024-04-28T13:35:55.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.009+0000 7fa533e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5340d1190 0x7fa53415a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.009+0000 7fa533e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5340d1190 0x7fa53415a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39026/0 (socket says 172.21.15.196:39026) 2024-04-28T13:35:55.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.009+0000 7fa533e00640 1 -- 172.21.15.196:0/1802799506 learned_addr learned my addr 172.21.15.196:0/1802799506 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:55.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.009+0000 7fa539600640 1 --2- 172.21.15.196:0/1802799506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5341437f0 0x7fa5340af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.009+0000 7fa538c00640 1 --2- 172.21.15.196:0/1802799506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa53406f600 0x7fa534142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.009+0000 7fa533e00640 1 -- 172.21.15.196:0/1802799506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa53406f600 msgr2=0x7fa534142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.009+0000 7fa533e00640 1 --2- 172.21.15.196:0/1802799506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa53406f600 0x7fa534142140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.009+0000 7fa533e00640 1 -- 172.21.15.196:0/1802799506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5341437f0 msgr2=0x7fa5340af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.009+0000 7fa533e00640 1 --2- 172.21.15.196:0/1802799506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5341437f0 0x7fa5340af2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.009+0000 7fa533e00640 1 -- 172.21.15.196:0/1802799506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53415b3d0 con 0x7fa5340d1190 2024-04-28T13:35:55.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.010+0000 7fa533e00640 1 --2- 172.21.15.196:0/1802799506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5340d1190 0x7fa53415a8f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa524009f40 tx=0x7fa524004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:55.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.010+0000 7fa530c00640 1 -- 172.21.15.196:0/1802799506 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524036050 con 0x7fa5340d1190 2024-04-28T13:35:55.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.010+0000 7fa530c00640 1 -- 172.21.15.196:0/1802799506 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa524042070 con 0x7fa5340d1190 2024-04-28T13:35:55.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.010+0000 7fa530c00640 1 -- 172.21.15.196:0/1802799506 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524009510 con 0x7fa5340d1190 2024-04-28T13:35:55.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.010+0000 7fa53b600640 1 -- 172.21.15.196:0/1802799506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53415b6b0 con 0x7fa5340d1190 2024-04-28T13:35:55.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.010+0000 7fa53b600640 1 -- 172.21.15.196:0/1802799506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa53415bc70 con 0x7fa5340d1190 2024-04-28T13:35:55.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.011+0000 7fa53b600640 1 -- 172.21.15.196:0/1802799506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa53406a290 con 0x7fa5340d1190 2024-04-28T13:35:55.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.012+0000 7fa530c00640 1 -- 172.21.15.196:0/1802799506 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa524007c50 con 0x7fa5340d1190 2024-04-28T13:35:55.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.013+0000 7fa530c00640 1 --2- 172.21.15.196:0/1802799506 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa51006c980 0x7fa51006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.013+0000 7fa530c00640 1 -- 172.21.15.196:0/1802799506 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fa5240acce0 con 0x7fa5340d1190 2024-04-28T13:35:55.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.013+0000 7fa530c00640 1 --2- 172.21.15.196:0/1802799506 >> [v2:172.21.15.196:6822/2304700074,v1:172.21.15.196:6825/2304700074] conn(0x7fa510074be0 0x7fa510077020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.013+0000 7fa530c00640 1 -- 172.21.15.196:0/1802799506 --> [v2:172.21.15.196:6822/2304700074,v1:172.21.15.196:6825/2304700074] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa52404ae80 con 0x7fa510074be0 2024-04-28T13:35:55.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.013+0000 7fa530c00640 1 -- 172.21.15.196:0/1802799506 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa5240af3d0 con 0x7fa5340d1190 2024-04-28T13:35:55.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.014+0000 7fa538c00640 1 --2- 172.21.15.196:0/1802799506 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa51006c980 0x7fa51006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.014+0000 7fa539600640 1 --2- 172.21.15.196:0/1802799506 >> [v2:172.21.15.196:6822/2304700074,v1:172.21.15.196:6825/2304700074] conn(0x7fa510074be0 0x7fa510077020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.014+0000 7fa539600640 1 --2- 172.21.15.196:0/1802799506 >> [v2:172.21.15.196:6822/2304700074,v1:172.21.15.196:6825/2304700074] conn(0x7fa510074be0 0x7fa510077020 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:55.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.015+0000 7fa538c00640 1 --2- 172.21.15.196:0/1802799506 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa51006c980 0x7fa51006ee40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa5341f9040 tx=0x7fa51c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:55.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.015+0000 7fa530c00640 1 -- 172.21.15.196:0/1802799506 <== osd.1 v2:172.21.15.196:6822/2304700074 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa52404ae80 con 0x7fa510074be0 2024-04-28T13:35:55.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.030+0000 7fa53b600640 1 -- 172.21.15.196:0/1802799506 --> [v2:172.21.15.196:6822/2304700074,v1:172.21.15.196:6825/2304700074] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa534160ba0 con 0x7fa510074be0 2024-04-28T13:35:55.030 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.030+0000 7fa530c00640 1 -- 172.21.15.196:0/1802799506 <== osd.1 v2:172.21.15.196:6822/2304700074 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa534160ba0 con 0x7fa510074be0 2024-04-28T13:35:55.030 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:35:55.031 INFO:teuthology.orchestra.run.smithi196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T13:35:55.031 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:35:55.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.031+0000 7fa53b600640 1 -- 172.21.15.196:0/1802799506 >> [v2:172.21.15.196:6822/2304700074,v1:172.21.15.196:6825/2304700074] conn(0x7fa510074be0 msgr2=0x7fa510077020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.031+0000 7fa53b600640 1 --2- 172.21.15.196:0/1802799506 >> [v2:172.21.15.196:6822/2304700074,v1:172.21.15.196:6825/2304700074] conn(0x7fa510074be0 0x7fa510077020 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.031+0000 7fa53b600640 1 -- 172.21.15.196:0/1802799506 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa51006c980 msgr2=0x7fa51006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.031+0000 7fa53b600640 1 --2- 172.21.15.196:0/1802799506 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa51006c980 0x7fa51006ee40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa5341f9040 tx=0x7fa51c008040 comp rx=0 tx=0).stop 2024-04-28T13:35:55.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.031+0000 7fa53b600640 1 -- 172.21.15.196:0/1802799506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5340d1190 msgr2=0x7fa53415a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.031+0000 7fa53b600640 1 --2- 172.21.15.196:0/1802799506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5340d1190 0x7fa53415a8f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa524009f40 tx=0x7fa524004a40 comp rx=0 tx=0).stop 2024-04-28T13:35:55.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.031+0000 7fa539600640 1 -- 172.21.15.196:0/1802799506 reap_dead start 2024-04-28T13:35:55.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.032+0000 7fa53b600640 1 -- 172.21.15.196:0/1802799506 shutdown_connections 2024-04-28T13:35:55.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.032+0000 7fa53b600640 1 -- 172.21.15.196:0/1802799506 >> 172.21.15.196:0/1802799506 conn(0x7fa53406c690 msgr2=0x7fa5340b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:55.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.032+0000 7fa53b600640 1 -- 172.21.15.196:0/1802799506 shutdown_connections 2024-04-28T13:35:55.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.032+0000 7fa53b600640 1 -- 172.21.15.196:0/1802799506 wait complete. 2024-04-28T13:35:55.051 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-28T13:35:55.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.253+0000 7fba10a00640 1 Processor -- start 2024-04-28T13:35:55.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.254+0000 7fba10a00640 1 -- start start 2024-04-28T13:35:55.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.255+0000 7fba10a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fba0c163bf0 0x7fba0c163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.255+0000 7fba10a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba0c163570 0x7fba0c0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.255+0000 7fba10a00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba0c162da0 con 0x7fba0c1629c0 2024-04-28T13:35:55.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.255+0000 7fba10a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fba0c0d3e20 con 0x7fba0c163bf0 2024-04-28T13:35:55.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.255+0000 7fba10a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fba0c0b6190 con 0x7fba0c163570 2024-04-28T13:35:55.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.256+0000 7fba0b200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba0c163570 0x7fba0c0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.256+0000 7fba0b200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba0c163570 0x7fba0c0b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.196:33086/0 (socket says 172.21.15.196:33086) 2024-04-28T13:35:55.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.256+0000 7fba0b200640 1 -- 172.21.15.196:0/3578188896 learned_addr learned my addr 172.21.15.196:0/3578188896 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:55.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.256+0000 7fba09e00640 1 --2- 172.21.15.196:0/3578188896 >> v2:172.21.15.203:3300/0 conn(0x7fba0c163bf0 0x7fba0c163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.257+0000 7fba0b200640 1 -- 172.21.15.196:0/3578188896 >> v2:172.21.15.203:3300/0 conn(0x7fba0c163bf0 msgr2=0x7fba0c163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:55.257 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.257+0000 7fba0b200640 1 --2- 172.21.15.196:0/3578188896 >> v2:172.21.15.203:3300/0 conn(0x7fba0c163bf0 0x7fba0c163030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.257+0000 7fba0b200640 1 -- 172.21.15.196:0/3578188896 >> v1:172.21.15.203:6789/0 conn(0x7fba0c1629c0 legacy=0x7fba0c0d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.257+0000 7fba0b200640 1 -- 172.21.15.196:0/3578188896 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0c0b6470 con 0x7fba0c163570 2024-04-28T13:35:55.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.257+0000 7fba09e00640 1 --2- 172.21.15.196:0/3578188896 >> v2:172.21.15.203:3300/0 conn(0x7fba0c163bf0 0x7fba0c163030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:35:55.258 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.259+0000 7fba0b200640 1 --2- 172.21.15.196:0/3578188896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba0c163570 0x7fba0c0b2380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fba04009870 tx=0x7fba0402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13f74e1bd2753e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:55.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.259+0000 7fba09400640 1 -- 172.21.15.196:0/3578188896 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04037070 con 0x7fba0c163570 2024-04-28T13:35:55.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.259+0000 7fba09400640 1 -- 172.21.15.196:0/3578188896 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba04032440 con 0x7fba0c163570 2024-04-28T13:35:55.259 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.259+0000 7fba09400640 1 -- 172.21.15.196:0/3578188896 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04032810 con 0x7fba0c163570 2024-04-28T13:35:55.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.260+0000 7fba10a00640 1 -- 172.21.15.196:0/3578188896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba0c163570 msgr2=0x7fba0c0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.260+0000 7fba10a00640 1 --2- 172.21.15.196:0/3578188896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba0c163570 0x7fba0c0b2380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fba04009870 tx=0x7fba0402a250 comp rx=0 tx=0).stop 2024-04-28T13:35:55.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.260+0000 7fba10a00640 1 -- 172.21.15.196:0/3578188896 shutdown_connections 2024-04-28T13:35:55.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.260+0000 7fba10a00640 1 --2- 172.21.15.196:0/3578188896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba0c163570 0x7fba0c0b2380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.260+0000 7fba10a00640 1 --2- 172.21.15.196:0/3578188896 >> v2:172.21.15.203:3300/0 conn(0x7fba0c163bf0 0x7fba0c163030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.260+0000 7fba10a00640 1 -- 172.21.15.196:0/3578188896 >> 172.21.15.196:0/3578188896 conn(0x7fba0c09e570 msgr2=0x7fba0c0d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:55.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.260+0000 7fba10a00640 1 -- 172.21.15.196:0/3578188896 shutdown_connections 2024-04-28T13:35:55.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.260+0000 7fba10a00640 1 -- 172.21.15.196:0/3578188896 wait complete. 2024-04-28T13:35:55.261 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.262+0000 7fba10a00640 1 Processor -- start 2024-04-28T13:35:55.262 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.262+0000 7fba10a00640 1 -- start start 2024-04-28T13:35:55.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.263+0000 7fba10a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba0c1629c0 0x7fba0c157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.263+0000 7fba10a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba0c163570 0x7fba0c0d1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.263+0000 7fba10a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba0c163bf0 0x7fba0c0d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.263+0000 7fba10a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c0b1e60 con 0x7fba0c163bf0 2024-04-28T13:35:55.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.263+0000 7fba10a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c0b6a30 con 0x7fba0c163570 2024-04-28T13:35:55.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.263+0000 7fba10a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fba0c0b28c0 con 0x7fba0c1629c0 2024-04-28T13:35:55.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.263+0000 7fba0b200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba0c163bf0 0x7fba0c0d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.264+0000 7fba0b200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba0c163bf0 0x7fba0c0d1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39036/0 (socket says 172.21.15.196:39036) 2024-04-28T13:35:55.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.264+0000 7fba0b200640 1 -- 172.21.15.196:0/650022705 learned_addr learned my addr 172.21.15.196:0/650022705 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:55.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.264+0000 7fba09e00640 1 --2- 172.21.15.196:0/650022705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba0c163570 0x7fba0c0d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.264+0000 7fba0a800640 1 --2- 172.21.15.196:0/650022705 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba0c1629c0 0x7fba0c157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.264+0000 7fba0b200640 1 -- 172.21.15.196:0/650022705 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba0c1629c0 msgr2=0x7fba0c157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.264+0000 7fba0b200640 1 --2- 172.21.15.196:0/650022705 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba0c1629c0 0x7fba0c157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.264+0000 7fba0b200640 1 -- 172.21.15.196:0/650022705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba0c163570 msgr2=0x7fba0c0d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.264+0000 7fba0b200640 1 --2- 172.21.15.196:0/650022705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba0c163570 0x7fba0c0d1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.264+0000 7fba0b200640 1 -- 172.21.15.196:0/650022705 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0c0c99c0 con 0x7fba0c163bf0 2024-04-28T13:35:55.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.264+0000 7fba0b200640 1 --2- 172.21.15.196:0/650022705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba0c163bf0 0x7fba0c0d1ab0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fba04009870 tx=0x7fba040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:55.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.265+0000 7fba02a00640 1 -- 172.21.15.196:0/650022705 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04041020 con 0x7fba0c163bf0 2024-04-28T13:35:55.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.265+0000 7fba02a00640 1 -- 172.21.15.196:0/650022705 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba0403d640 con 0x7fba0c163bf0 2024-04-28T13:35:55.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.265+0000 7fba02a00640 1 -- 172.21.15.196:0/650022705 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04037040 con 0x7fba0c163bf0 2024-04-28T13:35:55.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.265+0000 7fba10a00640 1 -- 172.21.15.196:0/650022705 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0c0c9ca0 con 0x7fba0c163bf0 2024-04-28T13:35:55.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.265+0000 7fba10a00640 1 -- 172.21.15.196:0/650022705 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba0c0ca260 con 0x7fba0c163bf0 2024-04-28T13:35:55.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.266+0000 7fba10a00640 1 -- 172.21.15.196:0/650022705 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba0c06a290 con 0x7fba0c163bf0 2024-04-28T13:35:55.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.267+0000 7fba02a00640 1 -- 172.21.15.196:0/650022705 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fba04044020 con 0x7fba0c163bf0 2024-04-28T13:35:55.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.268+0000 7fba02a00640 1 --2- 172.21.15.196:0/650022705 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb9e806c980 0x7fb9e806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.268+0000 7fba02a00640 1 -- 172.21.15.196:0/650022705 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fba040ac510 con 0x7fba0c163bf0 2024-04-28T13:35:55.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.269+0000 7fba0a800640 1 --2- 172.21.15.196:0/650022705 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb9e806c980 0x7fb9e806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.270+0000 7fba0a800640 1 --2- 172.21.15.196:0/650022705 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb9e806c980 0x7fb9e806ee40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc001640 tx=0x7fb9fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:55.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.273+0000 7fba02a00640 1 -- 172.21.15.196:0/650022705 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fba0404b030 con 0x7fba0c163bf0 2024-04-28T13:35:55.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.473+0000 7fba10a00640 1 -- 172.21.15.196:0/650022705 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fba0c0b2520 con 0x7fb9e806c980 2024-04-28T13:35:55.474 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.474+0000 7fba02a00640 1 -- 172.21.15.196:0/650022705 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fba0c0b2520 con 0x7fb9e806c980 2024-04-28T13:35:55.474 INFO:teuthology.orchestra.run.smithi196.stderr:instructed osd(s) 1 to deep-scrub 2024-04-28T13:35:55.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.476+0000 7fba10a00640 1 -- 172.21.15.196:0/650022705 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb9e806c980 msgr2=0x7fb9e806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.476+0000 7fba10a00640 1 --2- 172.21.15.196:0/650022705 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb9e806c980 0x7fb9e806ee40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc001640 tx=0x7fb9fc009040 comp rx=0 tx=0).stop 2024-04-28T13:35:55.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.477+0000 7fba10a00640 1 -- 172.21.15.196:0/650022705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba0c163bf0 msgr2=0x7fba0c0d1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.477+0000 7fba10a00640 1 --2- 172.21.15.196:0/650022705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba0c163bf0 0x7fba0c0d1ab0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fba04009870 tx=0x7fba040046c0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.477+0000 7fba10a00640 1 -- 172.21.15.196:0/650022705 shutdown_connections 2024-04-28T13:35:55.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.477+0000 7fba10a00640 1 --2- 172.21.15.196:0/650022705 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb9e806c980 0x7fb9e806ee40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.477+0000 7fba10a00640 1 --2- 172.21.15.196:0/650022705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba0c163bf0 0x7fba0c0d1ab0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.477+0000 7fba10a00640 1 --2- 172.21.15.196:0/650022705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba0c163570 0x7fba0c0d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.477+0000 7fba10a00640 1 --2- 172.21.15.196:0/650022705 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba0c1629c0 0x7fba0c157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.477+0000 7fba10a00640 1 -- 172.21.15.196:0/650022705 >> 172.21.15.196:0/650022705 conn(0x7fba0c09e570 msgr2=0x7fba0c161cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:55.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.477+0000 7fba10a00640 1 -- 172.21.15.196:0/650022705 shutdown_connections 2024-04-28T13:35:55.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.477+0000 7fba10a00640 1 -- 172.21.15.196:0/650022705 wait complete. 2024-04-28T13:35:55.496 INFO:tasks.ceph:Scrubbing osd.2 2024-04-28T13:35:55.496 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-28T13:35:55.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.700+0000 7f191be00640 1 Processor -- start 2024-04-28T13:35:55.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.701+0000 7f191be00640 1 -- start start 2024-04-28T13:35:55.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.702+0000 7f191be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f191c15ef00 0x7f191c0b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.702+0000 7f191be00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f191c15f2e0 0x7f191c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.702+0000 7f191be00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f191c06fce0 con 0x7f191c0d1050 2024-04-28T13:35:55.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.702+0000 7f191be00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f191c06dc70 con 0x7f191c15ef00 2024-04-28T13:35:55.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.702+0000 7f191be00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f191c06df50 con 0x7f191c15f2e0 2024-04-28T13:35:55.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.703+0000 7f191aa00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f191c15ef00 0x7f191c0b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.703+0000 7f191aa00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f191c15ef00 0x7f191c0b5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39038/0 (socket says 172.21.15.196:39038) 2024-04-28T13:35:55.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.703+0000 7f191aa00640 1 -- 172.21.15.196:0/1918172179 learned_addr learned my addr 172.21.15.196:0/1918172179 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:55.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.703+0000 7f191aa00640 1 -- 172.21.15.196:0/1918172179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f191c15f2e0 msgr2=0x7f191c06f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:55.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.703+0000 7f191aa00640 1 --2- 172.21.15.196:0/1918172179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f191c15f2e0 0x7f191c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.703+0000 7f191aa00640 1 -- 172.21.15.196:0/1918172179 >> v1:172.21.15.203:6789/0 conn(0x7f191c0d1050 legacy=0x7f191c162c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:55.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.703+0000 7f191aa00640 1 -- 172.21.15.196:0/1918172179 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c06e230 con 0x7f191c15ef00 2024-04-28T13:35:55.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.704+0000 7f191aa00640 1 --2- 172.21.15.196:0/1918172179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f191c15ef00 0x7f191c0b5c10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f190c009870 tx=0x7f190c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568ac542870f6c17 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:55.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.704+0000 7f191a000640 1 -- 172.21.15.196:0/1918172179 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c037070 con 0x7f191c15ef00 2024-04-28T13:35:55.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.704+0000 7f191a000640 1 -- 172.21.15.196:0/1918172179 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f190c032440 con 0x7f191c15ef00 2024-04-28T13:35:55.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.704+0000 7f191a000640 1 -- 172.21.15.196:0/1918172179 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c032810 con 0x7f191c15ef00 2024-04-28T13:35:55.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.705+0000 7f191be00640 1 -- 172.21.15.196:0/1918172179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f191c15ef00 msgr2=0x7f191c0b5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.705+0000 7f191be00640 1 --2- 172.21.15.196:0/1918172179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f191c15ef00 0x7f191c0b5c10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f190c009870 tx=0x7f190c02a180 comp rx=0 tx=0).stop 2024-04-28T13:35:55.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.705+0000 7f191be00640 1 -- 172.21.15.196:0/1918172179 shutdown_connections 2024-04-28T13:35:55.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.705+0000 7f191be00640 1 --2- 172.21.15.196:0/1918172179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f191c15f2e0 0x7f191c06f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.705+0000 7f191be00640 1 --2- 172.21.15.196:0/1918172179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f191c15ef00 0x7f191c0b5c10 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.705+0000 7f191be00640 1 -- 172.21.15.196:0/1918172179 >> 172.21.15.196:0/1918172179 conn(0x7f191c06c6c0 msgr2=0x7f191c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:55.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.705+0000 7f191be00640 1 -- 172.21.15.196:0/1918172179 shutdown_connections 2024-04-28T13:35:55.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.705+0000 7f191be00640 1 -- 172.21.15.196:0/1918172179 wait complete. 2024-04-28T13:35:55.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.707+0000 7f191be00640 1 Processor -- start 2024-04-28T13:35:55.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.708+0000 7f191be00640 1 -- start start 2024-04-28T13:35:55.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.708+0000 7f191be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f191c0d1050 0x7f191c160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.709+0000 7f191be00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f191c15ef00 0x7f191c15c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.709+0000 7f191be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f191c15f2e0 0x7f191c15cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.709+0000 7f191be00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f191c073290 con 0x7f191c0d1050 2024-04-28T13:35:55.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.709+0000 7f191be00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f191c073110 con 0x7f191c15f2e0 2024-04-28T13:35:55.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.709+0000 7f191be00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f191c0d1500 con 0x7f191c15ef00 2024-04-28T13:35:55.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.709+0000 7f191aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f191c0d1050 0x7f191c160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.709+0000 7f191aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f191c0d1050 0x7f191c160c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39040/0 (socket says 172.21.15.196:39040) 2024-04-28T13:35:55.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.709+0000 7f191aa00640 1 -- 172.21.15.196:0/1632880944 learned_addr learned my addr 172.21.15.196:0/1632880944 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:55.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.709+0000 7f191aa00640 1 -- 172.21.15.196:0/1632880944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f191c15ef00 msgr2=0x7f191c15c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.709+0000 7f191aa00640 1 --2- 172.21.15.196:0/1632880944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f191c15ef00 0x7f191c15c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.709+0000 7f191aa00640 1 -- 172.21.15.196:0/1632880944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f191c15f2e0 msgr2=0x7f191c15cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.709+0000 7f191aa00640 1 --2- 172.21.15.196:0/1632880944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f191c15f2e0 0x7f191c15cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.709+0000 7f191aa00640 1 -- 172.21.15.196:0/1632880944 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c15fbb0 con 0x7f191c0d1050 2024-04-28T13:35:55.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.710+0000 7f191aa00640 1 --2- 172.21.15.196:0/1632880944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f191c0d1050 0x7f191c160c50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f190c009870 tx=0x7f190c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:55.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.710+0000 7f1912a00640 1 -- 172.21.15.196:0/1632880944 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c041070 con 0x7f191c0d1050 2024-04-28T13:35:55.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.710+0000 7f1912a00640 1 -- 172.21.15.196:0/1632880944 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f190c03c930 con 0x7f191c0d1050 2024-04-28T13:35:55.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.710+0000 7f1912a00640 1 -- 172.21.15.196:0/1632880944 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c037040 con 0x7f191c0d1050 2024-04-28T13:35:55.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.710+0000 7f191be00640 1 -- 172.21.15.196:0/1632880944 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191c15fe90 con 0x7f191c0d1050 2024-04-28T13:35:55.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.710+0000 7f191be00640 1 -- 172.21.15.196:0/1632880944 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191c160450 con 0x7f191c0d1050 2024-04-28T13:35:55.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.711+0000 7f191be00640 1 -- 172.21.15.196:0/1632880944 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f191c050730 con 0x7f191c0d1050 2024-04-28T13:35:55.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.712+0000 7f1912a00640 1 -- 172.21.15.196:0/1632880944 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f190c044070 con 0x7f191c0d1050 2024-04-28T13:35:55.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.713+0000 7f1912a00640 1 --2- 172.21.15.196:0/1632880944 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f18f806c980 0x7f18f806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.713+0000 7f1912a00640 1 -- 172.21.15.196:0/1632880944 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f190c0ac850 con 0x7f191c0d1050 2024-04-28T13:35:55.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.713+0000 7f1912a00640 1 --2- 172.21.15.196:0/1632880944 >> [v2:172.21.15.196:6807/377093821,v1:172.21.15.196:6809/377093821] conn(0x7f18f8074be0 0x7f18f8077020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.713+0000 7f1912a00640 1 -- 172.21.15.196:0/1632880944 --> [v2:172.21.15.196:6807/377093821,v1:172.21.15.196:6809/377093821] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f190c00af30 con 0x7f18f8074be0 2024-04-28T13:35:55.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.713+0000 7f1912a00640 1 -- 172.21.15.196:0/1632880944 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f190c0b12a0 con 0x7f191c0d1050 2024-04-28T13:35:55.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.714+0000 7f1913e00640 1 --2- 172.21.15.196:0/1632880944 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f18f806c980 0x7f18f806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.714+0000 7f191b400640 1 --2- 172.21.15.196:0/1632880944 >> [v2:172.21.15.196:6807/377093821,v1:172.21.15.196:6809/377093821] conn(0x7f18f8074be0 0x7f18f8077020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.714+0000 7f191b400640 1 --2- 172.21.15.196:0/1632880944 >> [v2:172.21.15.196:6807/377093821,v1:172.21.15.196:6809/377093821] conn(0x7f18f8074be0 0x7f18f8077020 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:55.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.715+0000 7f1913e00640 1 --2- 172.21.15.196:0/1632880944 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f18f806c980 0x7f18f806ee40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f191c1ea580 tx=0x7f1904006bf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:55.715 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.715+0000 7f1912a00640 1 -- 172.21.15.196:0/1632880944 <== osd.2 v2:172.21.15.196:6807/377093821 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f190c00af30 con 0x7f18f8074be0 2024-04-28T13:35:55.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.730+0000 7f191be00640 1 -- 172.21.15.196:0/1632880944 --> [v2:172.21.15.196:6807/377093821,v1:172.21.15.196:6809/377093821] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f191c0b01a0 con 0x7f18f8074be0 2024-04-28T13:35:55.730 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.731+0000 7f1912a00640 1 -- 172.21.15.196:0/1632880944 <== osd.2 v2:172.21.15.196:6807/377093821 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f191c0b01a0 con 0x7f18f8074be0 2024-04-28T13:35:55.731 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:35:55.731 INFO:teuthology.orchestra.run.smithi196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T13:35:55.731 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:35:55.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.731+0000 7f191be00640 1 -- 172.21.15.196:0/1632880944 >> [v2:172.21.15.196:6807/377093821,v1:172.21.15.196:6809/377093821] conn(0x7f18f8074be0 msgr2=0x7f18f8077020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.731+0000 7f191be00640 1 --2- 172.21.15.196:0/1632880944 >> [v2:172.21.15.196:6807/377093821,v1:172.21.15.196:6809/377093821] conn(0x7f18f8074be0 0x7f18f8077020 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.732+0000 7f191be00640 1 -- 172.21.15.196:0/1632880944 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f18f806c980 msgr2=0x7f18f806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.732+0000 7f191be00640 1 --2- 172.21.15.196:0/1632880944 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f18f806c980 0x7f18f806ee40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f191c1ea580 tx=0x7f1904006bf0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.732+0000 7f191be00640 1 -- 172.21.15.196:0/1632880944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f191c0d1050 msgr2=0x7f191c160c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.732+0000 7f191be00640 1 --2- 172.21.15.196:0/1632880944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f191c0d1050 0x7f191c160c50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f190c009870 tx=0x7f190c0046c0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.732+0000 7f191b400640 1 -- 172.21.15.196:0/1632880944 reap_dead start 2024-04-28T13:35:55.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.732+0000 7f191be00640 1 -- 172.21.15.196:0/1632880944 shutdown_connections 2024-04-28T13:35:55.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.732+0000 7f191be00640 1 -- 172.21.15.196:0/1632880944 >> 172.21.15.196:0/1632880944 conn(0x7f191c06c6c0 msgr2=0x7f191c0b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:55.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.732+0000 7f191be00640 1 -- 172.21.15.196:0/1632880944 shutdown_connections 2024-04-28T13:35:55.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.732+0000 7f191be00640 1 -- 172.21.15.196:0/1632880944 wait complete. 2024-04-28T13:35:55.748 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-28T13:35:55.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.939+0000 7ff161000640 1 Processor -- start 2024-04-28T13:35:55.939 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.940+0000 7ff161000640 1 -- start start 2024-04-28T13:35:55.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.941+0000 7ff161000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff15c0b3c20 0x7ff15c0b4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.941+0000 7ff161000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff15c0b47a0 con 0x7ff15c0d0540 2024-04-28T13:35:55.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.941+0000 7ff161000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff15c0738e0 con 0x7ff15c073370 2024-04-28T13:35:55.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.941+0000 7ff161000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff15c0aea80 con 0x7ff15c0b3c20 2024-04-28T13:35:55.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.941+0000 7ff15be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff15c0b3c20 0x7ff15c0b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.941 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.941+0000 7ff15b400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ff15c0d0540 0x7ff15c0af560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:53180/0 (socket says 172.21.15.196:53180) 2024-04-28T13:35:55.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.941+0000 7ff15be00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff15c0b3c20 0x7ff15c0b4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39044/0 (socket says 172.21.15.196:39044) 2024-04-28T13:35:55.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.941+0000 7ff15b400640 1 -- 172.21.15.196:0/4110572911 learned_addr learned my addr 172.21.15.196:0/4110572911 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:55.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.942+0000 7ff15be00640 1 -- 172.21.15.196:0/4110572911 >> v1:172.21.15.203:6789/0 conn(0x7ff15c073370 legacy=0x7ff15c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:55.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.942+0000 7ff15be00640 1 -- 172.21.15.196:0/4110572911 >> v1:172.21.15.196:6789/0 conn(0x7ff15c0d0540 legacy=0x7ff15c0af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.942+0000 7ff15be00640 1 -- 172.21.15.196:0/4110572911 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15c0aed60 con 0x7ff15c0b3c20 2024-04-28T13:35:55.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.943+0000 7ff15be00640 1 --2- 172.21.15.196:0/4110572911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff15c0b3c20 0x7ff15c0b4070 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff1540097a0 tx=0x7ff15402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9c11b0762b320a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:55.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.943+0000 7ff15a000640 1 -- 172.21.15.196:0/4110572911 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154037070 con 0x7ff15c0b3c20 2024-04-28T13:35:55.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.943+0000 7ff15a000640 1 -- 172.21.15.196:0/4110572911 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff154032440 con 0x7ff15c0b3c20 2024-04-28T13:35:55.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.943+0000 7ff15a000640 1 -- 172.21.15.196:0/4110572911 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154032810 con 0x7ff15c0b3c20 2024-04-28T13:35:55.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.944+0000 7ff161000640 1 -- 172.21.15.196:0/4110572911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff15c0b3c20 msgr2=0x7ff15c0b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.944+0000 7ff161000640 1 --2- 172.21.15.196:0/4110572911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff15c0b3c20 0x7ff15c0b4070 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff1540097a0 tx=0x7ff15402a1e0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.944+0000 7ff161000640 1 -- 172.21.15.196:0/4110572911 shutdown_connections 2024-04-28T13:35:55.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.944+0000 7ff161000640 1 --2- 172.21.15.196:0/4110572911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff15c0b3c20 0x7ff15c0b4070 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.944+0000 7ff161000640 1 -- 172.21.15.196:0/4110572911 >> 172.21.15.196:0/4110572911 conn(0x7ff15c06c6c0 msgr2=0x7ff15c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:55.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.944+0000 7ff161000640 1 -- 172.21.15.196:0/4110572911 shutdown_connections 2024-04-28T13:35:55.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.944+0000 7ff161000640 1 -- 172.21.15.196:0/4110572911 wait complete. 2024-04-28T13:35:55.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.945+0000 7ff161000640 1 Processor -- start 2024-04-28T13:35:55.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.946+0000 7ff161000640 1 -- start start 2024-04-28T13:35:55.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.946+0000 7ff161000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff15c073370 0x7ff15c1404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.947+0000 7ff161000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff15c0b3c20 0x7ff15c142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.947+0000 7ff161000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c0d0540 0x7ff15c159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.947+0000 7ff161000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c0d0920 con 0x7ff15c0b3c20 2024-04-28T13:35:55.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.947+0000 7ff161000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c0ae6c0 con 0x7ff15c0d0540 2024-04-28T13:35:55.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.947+0000 7ff161000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff15c158b60 con 0x7ff15c073370 2024-04-28T13:35:55.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.947+0000 7ff15aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff15c0b3c20 0x7ff15c142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.947+0000 7ff15aa00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff15c0b3c20 0x7ff15c142db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39056/0 (socket says 172.21.15.196:39056) 2024-04-28T13:35:55.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.947+0000 7ff15aa00640 1 -- 172.21.15.196:0/3124852078 learned_addr learned my addr 172.21.15.196:0/3124852078 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:55.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.947+0000 7ff15b400640 1 --2- 172.21.15.196:0/3124852078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff15c073370 0x7ff15c1404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.947+0000 7ff15be00640 1 --2- 172.21.15.196:0/3124852078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c0d0540 0x7ff15c159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.947+0000 7ff15aa00640 1 -- 172.21.15.196:0/3124852078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff15c073370 msgr2=0x7ff15c1404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.947+0000 7ff15aa00640 1 --2- 172.21.15.196:0/3124852078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff15c073370 0x7ff15c1404a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.947+0000 7ff15aa00640 1 -- 172.21.15.196:0/3124852078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c0d0540 msgr2=0x7ff15c159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:55.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.947+0000 7ff15aa00640 1 --2- 172.21.15.196:0/3124852078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c0d0540 0x7ff15c159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:55.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.947+0000 7ff15aa00640 1 -- 172.21.15.196:0/3124852078 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15c143760 con 0x7ff15c0b3c20 2024-04-28T13:35:55.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.948+0000 7ff15aa00640 1 --2- 172.21.15.196:0/3124852078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff15c0b3c20 0x7ff15c142db0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff14800b420 tx=0x7ff14800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:55.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.948+0000 7ff153400640 1 -- 172.21.15.196:0/3124852078 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148011830 con 0x7ff15c0b3c20 2024-04-28T13:35:55.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.948+0000 7ff153400640 1 -- 172.21.15.196:0/3124852078 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff148016070 con 0x7ff15c0b3c20 2024-04-28T13:35:55.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.948+0000 7ff153400640 1 -- 172.21.15.196:0/3124852078 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148011c10 con 0x7ff15c0b3c20 2024-04-28T13:35:55.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.948+0000 7ff161000640 1 -- 172.21.15.196:0/3124852078 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15c143a40 con 0x7ff15c0b3c20 2024-04-28T13:35:55.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.948+0000 7ff161000640 1 -- 172.21.15.196:0/3124852078 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff15c1409e0 con 0x7ff15c0b3c20 2024-04-28T13:35:55.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.949+0000 7ff161000640 1 -- 172.21.15.196:0/3124852078 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff15c06a290 con 0x7ff15c0b3c20 2024-04-28T13:35:55.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.950+0000 7ff153400640 1 -- 172.21.15.196:0/3124852078 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff14801b460 con 0x7ff15c0b3c20 2024-04-28T13:35:55.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.951+0000 7ff153400640 1 --2- 172.21.15.196:0/3124852078 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff13c06c9b0 0x7ff13c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:55.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.951+0000 7ff153400640 1 -- 172.21.15.196:0/3124852078 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7ff14808b830 con 0x7ff15c0b3c20 2024-04-28T13:35:55.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.952+0000 7ff15b400640 1 --2- 172.21.15.196:0/3124852078 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff13c06c9b0 0x7ff13c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:55.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.953+0000 7ff15b400640 1 --2- 172.21.15.196:0/3124852078 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff13c06c9b0 0x7ff13c06ee70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0012e0 tx=0x7ff14c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:55.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:55.956+0000 7ff153400640 1 -- 172.21.15.196:0/3124852078 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff148020020 con 0x7ff15c0b3c20 2024-04-28T13:35:56.156 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.156+0000 7ff161000640 1 -- 172.21.15.196:0/3124852078 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7ff15c0694f0 con 0x7ff13c06c9b0 2024-04-28T13:35:56.157 INFO:teuthology.orchestra.run.smithi196.stderr:instructed osd(s) 2 to deep-scrub 2024-04-28T13:35:56.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.158+0000 7ff153400640 1 -- 172.21.15.196:0/3124852078 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff15c0694f0 con 0x7ff13c06c9b0 2024-04-28T13:35:56.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.161+0000 7ff161000640 1 -- 172.21.15.196:0/3124852078 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff13c06c9b0 msgr2=0x7ff13c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:56.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.161+0000 7ff161000640 1 --2- 172.21.15.196:0/3124852078 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff13c06c9b0 0x7ff13c06ee70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0012e0 tx=0x7ff14c009040 comp rx=0 tx=0).stop 2024-04-28T13:35:56.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.161+0000 7ff161000640 1 -- 172.21.15.196:0/3124852078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff15c0b3c20 msgr2=0x7ff15c142db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:56.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.161+0000 7ff161000640 1 --2- 172.21.15.196:0/3124852078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff15c0b3c20 0x7ff15c142db0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff14800b420 tx=0x7ff14800b8e0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.161+0000 7ff161000640 1 -- 172.21.15.196:0/3124852078 shutdown_connections 2024-04-28T13:35:56.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.161+0000 7ff161000640 1 --2- 172.21.15.196:0/3124852078 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7ff13c06c9b0 0x7ff13c06ee70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.161+0000 7ff161000640 1 --2- 172.21.15.196:0/3124852078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c0d0540 0x7ff15c159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.161+0000 7ff161000640 1 --2- 172.21.15.196:0/3124852078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff15c0b3c20 0x7ff15c142db0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.161+0000 7ff161000640 1 --2- 172.21.15.196:0/3124852078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff15c073370 0x7ff15c1404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.161+0000 7ff161000640 1 -- 172.21.15.196:0/3124852078 >> 172.21.15.196:0/3124852078 conn(0x7ff15c06c6c0 msgr2=0x7ff15c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:56.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.161+0000 7ff161000640 1 -- 172.21.15.196:0/3124852078 shutdown_connections 2024-04-28T13:35:56.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.161+0000 7ff161000640 1 -- 172.21.15.196:0/3124852078 wait complete. 2024-04-28T13:35:56.178 INFO:tasks.ceph:Scrubbing osd.3 2024-04-28T13:35:56.178 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-28T13:35:56.374 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.374+0000 7f8d74600640 1 Processor -- start 2024-04-28T13:35:56.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.375+0000 7f8d74600640 1 -- start start 2024-04-28T13:35:56.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.376+0000 7f8d74600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d6c1437f0 0x7f8d6c06f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.376+0000 7f8d74600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8d6c160e70 0x7f8d6c144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.376+0000 7f8d74600640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8d6c0d1190 0x7f8d6c0d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.376+0000 7f8d74600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8d6c0d1b30 con 0x7f8d6c160e70 2024-04-28T13:35:56.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.376+0000 7f8d74600640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f8d6c073440 con 0x7f8d6c0d1190 2024-04-28T13:35:56.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.376+0000 7f8d74600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8d6c1612b0 con 0x7f8d6c1437f0 2024-04-28T13:35:56.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.376+0000 7f8d71200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8d6c160e70 0x7f8d6c144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.376+0000 7f8d71200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8d6c160e70 0x7f8d6c144120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39060/0 (socket says 172.21.15.196:39060) 2024-04-28T13:35:56.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.376+0000 7f8d71200640 1 -- 172.21.15.196:0/4172287766 learned_addr learned my addr 172.21.15.196:0/4172287766 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:56.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.377+0000 7f8d71c00640 1 --2- 172.21.15.196:0/4172287766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d6c1437f0 0x7f8d6c06f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.377+0000 7f8d72600640 1 --2- 172.21.15.196:0/4172287766 >> v2:172.21.15.203:3300/0 conn(0x7f8d6c0d1190 0x7f8d6c0d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.377+0000 7f8d71200640 1 -- 172.21.15.196:0/4172287766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d6c1437f0 msgr2=0x7f8d6c06f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:56.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.377+0000 7f8d71200640 1 --2- 172.21.15.196:0/4172287766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d6c1437f0 0x7f8d6c06f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.377+0000 7f8d71200640 1 -- 172.21.15.196:0/4172287766 >> v2:172.21.15.203:3300/0 conn(0x7f8d6c0d1190 msgr2=0x7f8d6c0d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:56.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.377+0000 7f8d71200640 1 --2- 172.21.15.196:0/4172287766 >> v2:172.21.15.203:3300/0 conn(0x7f8d6c0d1190 0x7f8d6c0d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.379 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.377+0000 7f8d71200640 1 -- 172.21.15.196:0/4172287766 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c161570 con 0x7f8d6c160e70 2024-04-28T13:35:56.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.377+0000 7f8d72600640 1 --2- 172.21.15.196:0/4172287766 >> v2:172.21.15.203:3300/0 conn(0x7f8d6c0d1190 0x7f8d6c0d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:35:56.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.378+0000 7f8d71200640 1 --2- 172.21.15.196:0/4172287766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d6c160e70 0x7f8d6c144120 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0704b0 tx=0x7f8d5802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d1583024a778a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:56.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.378+0000 7f8d6be00640 1 -- 172.21.15.196:0/4172287766 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58037070 con 0x7f8d6c160e70 2024-04-28T13:35:56.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.378+0000 7f8d6be00640 1 -- 172.21.15.196:0/4172287766 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d58032440 con 0x7f8d6c160e70 2024-04-28T13:35:56.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.378+0000 7f8d6be00640 1 -- 172.21.15.196:0/4172287766 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58032810 con 0x7f8d6c160e70 2024-04-28T13:35:56.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.379+0000 7f8d74600640 1 -- 172.21.15.196:0/4172287766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d6c160e70 msgr2=0x7f8d6c144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:56.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.379+0000 7f8d74600640 1 --2- 172.21.15.196:0/4172287766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d6c160e70 0x7f8d6c144120 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0704b0 tx=0x7f8d5802a210 comp rx=0 tx=0).stop 2024-04-28T13:35:56.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.379+0000 7f8d74600640 1 -- 172.21.15.196:0/4172287766 shutdown_connections 2024-04-28T13:35:56.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.379+0000 7f8d74600640 1 --2- 172.21.15.196:0/4172287766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d6c160e70 0x7f8d6c144120 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.379+0000 7f8d74600640 1 --2- 172.21.15.196:0/4172287766 >> v2:172.21.15.203:3300/0 conn(0x7f8d6c0d1190 0x7f8d6c0d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.379+0000 7f8d74600640 1 --2- 172.21.15.196:0/4172287766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d6c1437f0 0x7f8d6c06f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.379+0000 7f8d74600640 1 -- 172.21.15.196:0/4172287766 >> 172.21.15.196:0/4172287766 conn(0x7f8d6c06c690 msgr2=0x7f8d6c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:56.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.379+0000 7f8d74600640 1 -- 172.21.15.196:0/4172287766 shutdown_connections 2024-04-28T13:35:56.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.379+0000 7f8d74600640 1 -- 172.21.15.196:0/4172287766 wait complete. 2024-04-28T13:35:56.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.381+0000 7f8d74600640 1 Processor -- start 2024-04-28T13:35:56.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.381+0000 7f8d74600640 1 -- start start 2024-04-28T13:35:56.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.382+0000 7f8d74600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d6c0d1190 0x7f8d6c142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.382+0000 7f8d74600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d6c1437f0 0x7f8d6c13b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.382+0000 7f8d74600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d6c160e70 0x7f8d6c15e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.382+0000 7f8d74600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c0736a0 con 0x7f8d6c0d1190 2024-04-28T13:35:56.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.382+0000 7f8d74600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c161b30 con 0x7f8d6c1437f0 2024-04-28T13:35:56.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.382+0000 7f8d74600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8d6c144660 con 0x7f8d6c160e70 2024-04-28T13:35:56.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.382+0000 7f8d71c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d6c0d1190 0x7f8d6c142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.382+0000 7f8d71c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d6c0d1190 0x7f8d6c142560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39070/0 (socket says 172.21.15.196:39070) 2024-04-28T13:35:56.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.382+0000 7f8d71c00640 1 -- 172.21.15.196:0/1447367100 learned_addr learned my addr 172.21.15.196:0/1447367100 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:56.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.383+0000 7f8d71200640 1 --2- 172.21.15.196:0/1447367100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d6c1437f0 0x7f8d6c13b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.383+0000 7f8d72600640 1 --2- 172.21.15.196:0/1447367100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d6c160e70 0x7f8d6c15e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.383+0000 7f8d71c00640 1 -- 172.21.15.196:0/1447367100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d6c160e70 msgr2=0x7f8d6c15e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:56.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.383+0000 7f8d71c00640 1 --2- 172.21.15.196:0/1447367100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d6c160e70 0x7f8d6c15e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.383+0000 7f8d71c00640 1 -- 172.21.15.196:0/1447367100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d6c1437f0 msgr2=0x7f8d6c13b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:56.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.383+0000 7f8d71c00640 1 --2- 172.21.15.196:0/1447367100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d6c1437f0 0x7f8d6c13b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.385 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.383+0000 7f8d71c00640 1 -- 172.21.15.196:0/1447367100 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c13f350 con 0x7f8d6c0d1190 2024-04-28T13:35:56.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.383+0000 7f8d71c00640 1 --2- 172.21.15.196:0/1447367100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d6c0d1190 0x7f8d6c142560 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000d800 tx=0x7f8d6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:56.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.383+0000 7f8d69600640 1 -- 172.21.15.196:0/1447367100 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6000b6c0 con 0x7f8d6c0d1190 2024-04-28T13:35:56.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.383+0000 7f8d69600640 1 -- 172.21.15.196:0/1447367100 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d60019070 con 0x7f8d6c0d1190 2024-04-28T13:35:56.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.383+0000 7f8d69600640 1 -- 172.21.15.196:0/1447367100 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6000ba80 con 0x7f8d6c0d1190 2024-04-28T13:35:56.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.383+0000 7f8d74600640 1 -- 172.21.15.196:0/1447367100 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6c13fb50 con 0x7f8d6c0d1190 2024-04-28T13:35:56.386 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.384+0000 7f8d74600640 1 -- 172.21.15.196:0/1447367100 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d6c13f070 con 0x7f8d6c0d1190 2024-04-28T13:35:56.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.384+0000 7f8d74600640 1 -- 172.21.15.196:0/1447367100 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8d6c13be40 con 0x7f8d6c0d1190 2024-04-28T13:35:56.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.386+0000 7f8d69600640 1 -- 172.21.15.196:0/1447367100 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8d6000bc20 con 0x7f8d6c0d1190 2024-04-28T13:35:56.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.387+0000 7f8d69600640 1 --2- 172.21.15.196:0/1447367100 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8d4c06c980 0x7f8d4c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.387+0000 7f8d69600640 1 -- 172.21.15.196:0/1447367100 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f8d6008c0b0 con 0x7f8d6c0d1190 2024-04-28T13:35:56.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.387+0000 7f8d69600640 1 --2- 172.21.15.196:0/1447367100 >> [v2:172.21.15.196:6815/176565235,v1:172.21.15.196:6817/176565235] conn(0x7f8d4c074be0 0x7f8d4c077020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.387+0000 7f8d69600640 1 -- 172.21.15.196:0/1447367100 --> [v2:172.21.15.196:6815/176565235,v1:172.21.15.196:6817/176565235] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d4c0776f0 con 0x7f8d4c074be0 2024-04-28T13:35:56.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.387+0000 7f8d69600640 1 -- 172.21.15.196:0/1447367100 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8d60093050 con 0x7f8d6c0d1190 2024-04-28T13:35:56.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.387+0000 7f8d72600640 1 --2- 172.21.15.196:0/1447367100 >> [v2:172.21.15.196:6815/176565235,v1:172.21.15.196:6817/176565235] conn(0x7f8d4c074be0 0x7f8d4c077020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.387+0000 7f8d71200640 1 --2- 172.21.15.196:0/1447367100 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8d4c06c980 0x7f8d4c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.388+0000 7f8d72600640 1 --2- 172.21.15.196:0/1447367100 >> [v2:172.21.15.196:6815/176565235,v1:172.21.15.196:6817/176565235] conn(0x7f8d4c074be0 0x7f8d4c077020 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:56.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.389+0000 7f8d71200640 1 --2- 172.21.15.196:0/1447367100 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8d4c06c980 0x7f8d4c06ee40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8d58034000 tx=0x7f8d5802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:56.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.389+0000 7f8d69600640 1 -- 172.21.15.196:0/1447367100 <== osd.3 v2:172.21.15.196:6815/176565235 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d4c0776f0 con 0x7f8d4c074be0 2024-04-28T13:35:56.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.403+0000 7f8d74600640 1 -- 172.21.15.196:0/1447367100 --> [v2:172.21.15.196:6815/176565235,v1:172.21.15.196:6817/176565235] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8d6c06a290 con 0x7f8d4c074be0 2024-04-28T13:35:56.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.404+0000 7f8d69600640 1 -- 172.21.15.196:0/1447367100 <== osd.3 v2:172.21.15.196:6815/176565235 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8d6c06a290 con 0x7f8d4c074be0 2024-04-28T13:35:56.404 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:35:56.404 INFO:teuthology.orchestra.run.smithi196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T13:35:56.404 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:35:56.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.404+0000 7f8d74600640 1 -- 172.21.15.196:0/1447367100 >> [v2:172.21.15.196:6815/176565235,v1:172.21.15.196:6817/176565235] conn(0x7f8d4c074be0 msgr2=0x7f8d4c077020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:56.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.404+0000 7f8d74600640 1 --2- 172.21.15.196:0/1447367100 >> [v2:172.21.15.196:6815/176565235,v1:172.21.15.196:6817/176565235] conn(0x7f8d4c074be0 0x7f8d4c077020 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.404+0000 7f8d74600640 1 -- 172.21.15.196:0/1447367100 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8d4c06c980 msgr2=0x7f8d4c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:56.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.404+0000 7f8d74600640 1 --2- 172.21.15.196:0/1447367100 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f8d4c06c980 0x7f8d4c06ee40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8d58034000 tx=0x7f8d5802e000 comp rx=0 tx=0).stop 2024-04-28T13:35:56.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.404+0000 7f8d74600640 1 -- 172.21.15.196:0/1447367100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d6c0d1190 msgr2=0x7f8d6c142560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:56.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.404+0000 7f8d74600640 1 --2- 172.21.15.196:0/1447367100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d6c0d1190 0x7f8d6c142560 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000d800 tx=0x7f8d6000dcc0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.405+0000 7f8d72600640 1 -- 172.21.15.196:0/1447367100 reap_dead start 2024-04-28T13:35:56.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.405+0000 7f8d74600640 1 -- 172.21.15.196:0/1447367100 shutdown_connections 2024-04-28T13:35:56.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.405+0000 7f8d74600640 1 -- 172.21.15.196:0/1447367100 >> 172.21.15.196:0/1447367100 conn(0x7f8d6c06c690 msgr2=0x7f8d6c06ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:56.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.405+0000 7f8d74600640 1 -- 172.21.15.196:0/1447367100 shutdown_connections 2024-04-28T13:35:56.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.405+0000 7f8d74600640 1 -- 172.21.15.196:0/1447367100 wait complete. 2024-04-28T13:35:56.421 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-28T13:35:56.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.618+0000 7f4318e00640 1 Processor -- start 2024-04-28T13:35:56.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.618+0000 7f4318e00640 1 -- start start 2024-04-28T13:35:56.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.620+0000 7f4318e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4314145870 0x7f431406f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.620+0000 7f4318e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f431406f980 0x7f4314072d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.620+0000 7f4318e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4314073a40 con 0x7f4314073250 2024-04-28T13:35:56.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.620+0000 7f4318e00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f431406fde0 con 0x7f4314145870 2024-04-28T13:35:56.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.620+0000 7f4318e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f431415d8d0 con 0x7f431406f980 2024-04-28T13:35:56.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.620+0000 7f4312800640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f4314073250 0x7f431415fab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:53196/0 (socket says 172.21.15.196:53196) 2024-04-28T13:35:56.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.620+0000 7f4312800640 1 -- 172.21.15.196:0/2317355183 learned_addr learned my addr 172.21.15.196:0/2317355183 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:56.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.621+0000 7f4309400640 1 --2- 172.21.15.196:0/2317355183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f431406f980 0x7f4314072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.621+0000 7f4311e00640 1 --2- 172.21.15.196:0/2317355183 >> v2:172.21.15.203:3300/0 conn(0x7f4314145870 0x7f431406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.621+0000 7f4311400640 1 -- 172.21.15.196:0/2317355183 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2282410249 0 0) 0x7f4314073a40 con 0x7f4314073250 2024-04-28T13:35:56.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.621+0000 7f4311400640 1 -- 172.21.15.196:0/2317355183 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42f8003520 con 0x7f4314073250 2024-04-28T13:35:56.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.621+0000 7f4311400640 1 -- 172.21.15.196:0/2317355183 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 276055810 0 0) 0x7f42f8003520 con 0x7f4314073250 2024-04-28T13:35:56.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.621+0000 7f4311400640 1 -- 172.21.15.196:0/2317355183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f431406f980 msgr2=0x7f4314072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:56.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.621+0000 7f4311400640 1 --2- 172.21.15.196:0/2317355183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f431406f980 0x7f4314072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.621+0000 7f4311400640 1 -- 172.21.15.196:0/2317355183 >> v2:172.21.15.203:3300/0 conn(0x7f4314145870 msgr2=0x7f431406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:56.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.621+0000 7f4311400640 1 --2- 172.21.15.196:0/2317355183 >> v2:172.21.15.203:3300/0 conn(0x7f4314145870 0x7f431406f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.621+0000 7f4311400640 1 -- 172.21.15.196:0/2317355183 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431415dbb0 con 0x7f4314073250 2024-04-28T13:35:56.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.621+0000 7f4311400640 1 -- 172.21.15.196:0/2317355183 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f430c0019f0 con 0x7f4314073250 2024-04-28T13:35:56.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.621+0000 7f4311400640 1 -- 172.21.15.196:0/2317355183 >> v1:172.21.15.196:6789/0 conn(0x7f4314073250 legacy=0x7f431415fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:56.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.621+0000 7f4311400640 1 --2- 172.21.15.196:0/2317355183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f431406f980 0x7f42f80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.621+0000 7f4311400640 1 --2- 172.21.15.196:0/2317355183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42f8003f30 0x7f42f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.621+0000 7f4311400640 1 --2- 172.21.15.196:0/2317355183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42f8004d40 0x7f42f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.622+0000 7f4311400640 1 -- 172.21.15.196:0/2317355183 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f431415d8d0 con 0x7f42f8004d40 2024-04-28T13:35:56.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.622+0000 7f4311400640 1 -- 172.21.15.196:0/2317355183 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f42f800b9c0 con 0x7f42f8003f30 2024-04-28T13:35:56.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.622+0000 7f4311400640 1 -- 172.21.15.196:0/2317355183 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f42f800bb40 con 0x7f431406f980 2024-04-28T13:35:56.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.622+0000 7f4312800640 1 --2- 172.21.15.196:0/2317355183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42f8004d40 0x7f42f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.622+0000 7f4309400640 1 --2- 172.21.15.196:0/2317355183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42f8003f30 0x7f42f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.622+0000 7f4311e00640 1 --2- 172.21.15.196:0/2317355183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f431406f980 0x7f42f80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.622+0000 7f4312800640 1 -- 172.21.15.196:0/2317355183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f431406f980 msgr2=0x7f42f80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:56.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.622+0000 7f4312800640 1 --2- 172.21.15.196:0/2317355183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f431406f980 0x7f42f80039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.622+0000 7f4312800640 1 -- 172.21.15.196:0/2317355183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42f8003f30 msgr2=0x7f42f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:56.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.622+0000 7f4312800640 1 --2- 172.21.15.196:0/2317355183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42f8003f30 0x7f42f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.622+0000 7f4312800640 1 -- 172.21.15.196:0/2317355183 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f8003520 con 0x7f42f8004d40 2024-04-28T13:35:56.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.623+0000 7f4312800640 1 --2- 172.21.15.196:0/2317355183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42f8004d40 0x7f42f80051c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f430c00a8a0 tx=0x7f42f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f89a00b0aae87d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:56.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.624+0000 7f4311400640 1 -- 172.21.15.196:0/2317355183 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c00a4c0 con 0x7f42f8004d40 2024-04-28T13:35:56.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.624+0000 7f4311400640 1 -- 172.21.15.196:0/2317355183 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430c02f070 con 0x7f42f8004d40 2024-04-28T13:35:56.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.624+0000 7f4311400640 1 -- 172.21.15.196:0/2317355183 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c02bd50 con 0x7f42f8004d40 2024-04-28T13:35:56.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.624+0000 7f4318e00640 1 -- 172.21.15.196:0/2317355183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42f8004d40 msgr2=0x7f42f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:56.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.624+0000 7f4318e00640 1 --2- 172.21.15.196:0/2317355183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42f8004d40 0x7f42f80051c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f430c00a8a0 tx=0x7f42f8005830 comp rx=0 tx=0).stop 2024-04-28T13:35:56.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.624+0000 7f4312800640 1 -- 172.21.15.196:0/2317355183 reap_dead start 2024-04-28T13:35:56.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.624+0000 7f4318e00640 1 -- 172.21.15.196:0/2317355183 shutdown_connections 2024-04-28T13:35:56.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.624+0000 7f4318e00640 1 -- 172.21.15.196:0/2317355183 >> 172.21.15.196:0/2317355183 conn(0x7f43140cd830 msgr2=0x7f43140cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:56.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.624+0000 7f4318e00640 1 -- 172.21.15.196:0/2317355183 shutdown_connections 2024-04-28T13:35:56.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.624+0000 7f4318e00640 1 -- 172.21.15.196:0/2317355183 wait complete. 2024-04-28T13:35:56.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.626+0000 7f4318e00640 1 Processor -- start 2024-04-28T13:35:56.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.626+0000 7f4318e00640 1 -- start start 2024-04-28T13:35:56.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.627+0000 7f4318e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42f8004d40 0x7f43140b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.627+0000 7f4318e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f431413e2d0 0x7f431413e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.627+0000 7f4318e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43141417c0 0x7f4314141c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.627+0000 7f4318e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4314145c50 con 0x7f43141417c0 2024-04-28T13:35:56.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.627+0000 7f4318e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f431415d5d0 con 0x7f431413e2d0 2024-04-28T13:35:56.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.627+0000 7f4318e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f43140736e0 con 0x7f42f8004d40 2024-04-28T13:35:56.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.627+0000 7f4312800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43141417c0 0x7f4314141c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.627+0000 7f4312800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43141417c0 0x7f4314141c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39086/0 (socket says 172.21.15.196:39086) 2024-04-28T13:35:56.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.628+0000 7f4312800640 1 -- 172.21.15.196:0/2914060915 learned_addr learned my addr 172.21.15.196:0/2914060915 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:56.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.628+0000 7f4311e00640 1 --2- 172.21.15.196:0/2914060915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42f8004d40 0x7f43140b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.628+0000 7f4309400640 1 --2- 172.21.15.196:0/2914060915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f431413e2d0 0x7f431413e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.628+0000 7f4312800640 1 -- 172.21.15.196:0/2914060915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42f8004d40 msgr2=0x7f43140b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:56.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.628+0000 7f4312800640 1 --2- 172.21.15.196:0/2914060915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42f8004d40 0x7f43140b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.628+0000 7f4312800640 1 -- 172.21.15.196:0/2914060915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f431413e2d0 msgr2=0x7f431413e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:56.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.628+0000 7f4312800640 1 --2- 172.21.15.196:0/2914060915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f431413e2d0 0x7f431413e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.628+0000 7f4312800640 1 -- 172.21.15.196:0/2914060915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4314157a90 con 0x7f43141417c0 2024-04-28T13:35:56.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.628+0000 7f4312800640 1 --2- 172.21.15.196:0/2914060915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43141417c0 0x7f4314141c40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f430c005c20 tx=0x7f430c001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:56.634 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.628+0000 7f430aa00640 1 -- 172.21.15.196:0/2914060915 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c044070 con 0x7f43141417c0 2024-04-28T13:35:56.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.629+0000 7f430aa00640 1 -- 172.21.15.196:0/2914060915 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430c034470 con 0x7f43141417c0 2024-04-28T13:35:56.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.629+0000 7f430aa00640 1 -- 172.21.15.196:0/2914060915 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c00a4c0 con 0x7f43141417c0 2024-04-28T13:35:56.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.629+0000 7f4318e00640 1 -- 172.21.15.196:0/2914060915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4314157d70 con 0x7f43141417c0 2024-04-28T13:35:56.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.629+0000 7f4318e00640 1 -- 172.21.15.196:0/2914060915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4314158330 con 0x7f43141417c0 2024-04-28T13:35:56.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.630+0000 7f4318e00640 1 -- 172.21.15.196:0/2914060915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f431406a290 con 0x7f43141417c0 2024-04-28T13:35:56.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.631+0000 7f430aa00640 1 -- 172.21.15.196:0/2914060915 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f430c034820 con 0x7f43141417c0 2024-04-28T13:35:56.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.632+0000 7f430aa00640 1 --2- 172.21.15.196:0/2914060915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f42f006c980 0x7f42f006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:56.635 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.632+0000 7f430aa00640 1 -- 172.21.15.196:0/2914060915 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f430c0abc80 con 0x7f43141417c0 2024-04-28T13:35:56.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.632+0000 7f4311e00640 1 --2- 172.21.15.196:0/2914060915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f42f006c980 0x7f42f006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:56.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.634+0000 7f4311e00640 1 --2- 172.21.15.196:0/2914060915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f42f006c980 0x7f42f006ee40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4304009a90 tx=0x7f4304008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:56.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.636+0000 7f430aa00640 1 -- 172.21.15.196:0/2914060915 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f430c079a30 con 0x7f43141417c0 2024-04-28T13:35:56.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.841+0000 7f4318e00640 1 -- 172.21.15.196:0/2914060915 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f4314072eb0 con 0x7f42f006c980 2024-04-28T13:35:56.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.842+0000 7f430aa00640 1 -- 172.21.15.196:0/2914060915 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4314072eb0 con 0x7f42f006c980 2024-04-28T13:35:56.842 INFO:teuthology.orchestra.run.smithi196.stderr:instructed osd(s) 3 to deep-scrub 2024-04-28T13:35:56.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.845+0000 7f4318e00640 1 -- 172.21.15.196:0/2914060915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f42f006c980 msgr2=0x7f42f006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:56.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.845+0000 7f4318e00640 1 --2- 172.21.15.196:0/2914060915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f42f006c980 0x7f42f006ee40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4304009a90 tx=0x7f4304008040 comp rx=0 tx=0).stop 2024-04-28T13:35:56.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.845+0000 7f4318e00640 1 -- 172.21.15.196:0/2914060915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43141417c0 msgr2=0x7f4314141c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:56.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.845+0000 7f4318e00640 1 --2- 172.21.15.196:0/2914060915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43141417c0 0x7f4314141c40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f430c005c20 tx=0x7f430c001dc0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.845+0000 7f4318e00640 1 -- 172.21.15.196:0/2914060915 shutdown_connections 2024-04-28T13:35:56.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.845+0000 7f4318e00640 1 --2- 172.21.15.196:0/2914060915 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f42f006c980 0x7f42f006ee40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.845+0000 7f4318e00640 1 --2- 172.21.15.196:0/2914060915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43141417c0 0x7f4314141c40 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.845+0000 7f4318e00640 1 --2- 172.21.15.196:0/2914060915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f431413e2d0 0x7f431413e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.845+0000 7f4318e00640 1 --2- 172.21.15.196:0/2914060915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42f8004d40 0x7f43140b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:56.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.845+0000 7f4318e00640 1 -- 172.21.15.196:0/2914060915 >> 172.21.15.196:0/2914060915 conn(0x7f43140cd830 msgr2=0x7f431406e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:56.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.845+0000 7f4318e00640 1 -- 172.21.15.196:0/2914060915 shutdown_connections 2024-04-28T13:35:56.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:56.845+0000 7f4318e00640 1 -- 172.21.15.196:0/2914060915 wait complete. 2024-04-28T13:35:56.864 INFO:tasks.ceph:Scrubbing osd.4 2024-04-28T13:35:56.865 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-28T13:35:57.060 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.060+0000 7fe857200640 1 Processor -- start 2024-04-28T13:35:57.061 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.060+0000 7fe857200640 1 -- start start 2024-04-28T13:35:57.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.062+0000 7fe857200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8580b5ee0 0x7fe8580b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.062+0000 7fe857200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe8580d1760 0x7fe85806df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.062+0000 7fe857200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe85815e380 con 0x7fe85809ec80 2024-04-28T13:35:57.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.062+0000 7fe857200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe8580b4b20 con 0x7fe8580d1760 2024-04-28T13:35:57.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.062+0000 7fe857200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe8580b3f30 con 0x7fe8580b5ee0 2024-04-28T13:35:57.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.062+0000 7fe856800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe8580d1760 0x7fe85806df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.062+0000 7fe856800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe8580d1760 0x7fe85806df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39092/0 (socket says 172.21.15.196:39092) 2024-04-28T13:35:57.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.062+0000 7fe856800640 1 -- 172.21.15.196:0/2415512719 learned_addr learned my addr 172.21.15.196:0/2415512719 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:57.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.063+0000 7fe855e00640 1 --2- 172.21.15.196:0/2415512719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8580b5ee0 0x7fe8580b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.063+0000 7fe856800640 1 -- 172.21.15.196:0/2415512719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8580b5ee0 msgr2=0x7fe8580b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:57.065 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.063+0000 7fe856800640 1 --2- 172.21.15.196:0/2415512719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8580b5ee0 0x7fe8580b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.065 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.063+0000 7fe856800640 1 -- 172.21.15.196:0/2415512719 >> v1:172.21.15.203:6789/0 conn(0x7fe85809ec80 legacy=0x7fe8580d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.065 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.063+0000 7fe856800640 1 -- 172.21.15.196:0/2415512719 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8580b41f0 con 0x7fe8580d1760 2024-04-28T13:35:57.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.064+0000 7fe856800640 1 --2- 172.21.15.196:0/2415512719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8580d1760 0x7fe85806df50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe850009e80 tx=0x7fe85002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a8c04c292f21d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:57.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.064+0000 7fe855400640 1 -- 172.21.15.196:0/2415512719 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850036070 con 0x7fe8580d1760 2024-04-28T13:35:57.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.064+0000 7fe855400640 1 -- 172.21.15.196:0/2415512719 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe850031600 con 0x7fe8580d1760 2024-04-28T13:35:57.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.064+0000 7fe855400640 1 -- 172.21.15.196:0/2415512719 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8500319d0 con 0x7fe8580d1760 2024-04-28T13:35:57.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.065+0000 7fe857200640 1 -- 172.21.15.196:0/2415512719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8580d1760 msgr2=0x7fe85806df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.065+0000 7fe857200640 1 --2- 172.21.15.196:0/2415512719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8580d1760 0x7fe85806df50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe850009e80 tx=0x7fe85002a750 comp rx=0 tx=0).stop 2024-04-28T13:35:57.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.065+0000 7fe857200640 1 -- 172.21.15.196:0/2415512719 shutdown_connections 2024-04-28T13:35:57.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.065+0000 7fe857200640 1 --2- 172.21.15.196:0/2415512719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8580d1760 0x7fe85806df50 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.065+0000 7fe857200640 1 --2- 172.21.15.196:0/2415512719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8580b5ee0 0x7fe8580b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.065+0000 7fe857200640 1 -- 172.21.15.196:0/2415512719 >> 172.21.15.196:0/2415512719 conn(0x7fe85806c6c0 msgr2=0x7fe85806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:57.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.065+0000 7fe857200640 1 -- 172.21.15.196:0/2415512719 shutdown_connections 2024-04-28T13:35:57.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.065+0000 7fe857200640 1 -- 172.21.15.196:0/2415512719 wait complete. 2024-04-28T13:35:57.068 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.068+0000 7fe857200640 1 Processor -- start 2024-04-28T13:35:57.068 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.068+0000 7fe857200640 1 -- start start 2024-04-28T13:35:57.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.069+0000 7fe857200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe85809ec80 0x7fe858150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.069+0000 7fe857200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8580b5ee0 0x7fe8580b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.069+0000 7fe857200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8580d1760 0x7fe858073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.069+0000 7fe857200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8580b3c20 con 0x7fe8580d1760 2024-04-28T13:35:57.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.069+0000 7fe857200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe8580b47b0 con 0x7fe8580b5ee0 2024-04-28T13:35:57.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.069+0000 7fe857200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe8580add70 con 0x7fe85809ec80 2024-04-28T13:35:57.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.069+0000 7fe856800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8580d1760 0x7fe858073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.069+0000 7fe856800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8580d1760 0x7fe858073230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39096/0 (socket says 172.21.15.196:39096) 2024-04-28T13:35:57.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.069+0000 7fe856800640 1 -- 172.21.15.196:0/3710253143 learned_addr learned my addr 172.21.15.196:0/3710253143 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:57.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.070+0000 7fe855e00640 1 --2- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe85809ec80 0x7fe858150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.070+0000 7fe84fe00640 1 --2- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8580b5ee0 0x7fe8580b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.070+0000 7fe856800640 1 -- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe85809ec80 msgr2=0x7fe858150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.070+0000 7fe856800640 1 --2- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe85809ec80 0x7fe858150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.070+0000 7fe856800640 1 -- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8580b5ee0 msgr2=0x7fe8580b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.070+0000 7fe856800640 1 --2- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8580b5ee0 0x7fe8580b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.070+0000 7fe856800640 1 -- 172.21.15.196:0/3710253143 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8581510d0 con 0x7fe8580d1760 2024-04-28T13:35:57.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.070+0000 7fe856800640 1 --2- 172.21.15.196:0/3710253143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8580d1760 0x7fe858073230 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe850009fa0 tx=0x7fe850004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:57.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.071+0000 7fe84e000640 1 -- 172.21.15.196:0/3710253143 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85003f020 con 0x7fe8580d1760 2024-04-28T13:35:57.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.071+0000 7fe84e000640 1 -- 172.21.15.196:0/3710253143 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe850037040 con 0x7fe8580d1760 2024-04-28T13:35:57.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.071+0000 7fe84e000640 1 -- 172.21.15.196:0/3710253143 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850036070 con 0x7fe8580d1760 2024-04-28T13:35:57.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.071+0000 7fe857200640 1 -- 172.21.15.196:0/3710253143 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe858071ef0 con 0x7fe8580d1760 2024-04-28T13:35:57.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.071+0000 7fe857200640 1 -- 172.21.15.196:0/3710253143 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe858072470 con 0x7fe8580d1760 2024-04-28T13:35:57.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.072+0000 7fe857200640 1 -- 172.21.15.196:0/3710253143 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe858050730 con 0x7fe8580d1760 2024-04-28T13:35:57.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.073+0000 7fe84e000640 1 -- 172.21.15.196:0/3710253143 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe850042020 con 0x7fe8580d1760 2024-04-28T13:35:57.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.074+0000 7fe84e000640 1 --2- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe83406c9d0 0x7fe83406ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.074+0000 7fe84e000640 1 -- 172.21.15.196:0/3710253143 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fe8500ac070 con 0x7fe8580d1760 2024-04-28T13:35:57.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.074+0000 7fe84e000640 1 --2- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:6800/1325487575,v1:172.21.15.203:6801/1325487575] conn(0x7fe834074d10 0x7fe834077150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.074+0000 7fe84e000640 1 -- 172.21.15.196:0/3710253143 --> [v2:172.21.15.203:6800/1325487575,v1:172.21.15.203:6801/1325487575] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe834077820 con 0x7fe834074d10 2024-04-28T13:35:57.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.074+0000 7fe84e000640 1 -- 172.21.15.196:0/3710253143 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe8500ae5d0 con 0x7fe8580d1760 2024-04-28T13:35:57.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.074+0000 7fe855e00640 1 --2- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe83406c9d0 0x7fe83406ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.075+0000 7fe84fe00640 1 --2- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:6800/1325487575,v1:172.21.15.203:6801/1325487575] conn(0x7fe834074d10 0x7fe834077150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.076+0000 7fe855e00640 1 --2- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe83406c9d0 0x7fe83406ee90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe848009ad0 tx=0x7fe848008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:57.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.076+0000 7fe84fe00640 1 --2- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:6800/1325487575,v1:172.21.15.203:6801/1325487575] conn(0x7fe834074d10 0x7fe834077150 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:57.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.078+0000 7fe84e000640 1 -- 172.21.15.196:0/3710253143 <== osd.4 v2:172.21.15.203:6800/1325487575 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe834077820 con 0x7fe834074d10 2024-04-28T13:35:57.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.093+0000 7fe857200640 1 -- 172.21.15.196:0/3710253143 --> [v2:172.21.15.203:6800/1325487575,v1:172.21.15.203:6801/1325487575] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe85806ee60 con 0x7fe834074d10 2024-04-28T13:35:57.094 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.094+0000 7fe84e000640 1 -- 172.21.15.196:0/3710253143 <== osd.4 v2:172.21.15.203:6800/1325487575 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe85806ee60 con 0x7fe834074d10 2024-04-28T13:35:57.094 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:35:57.094 INFO:teuthology.orchestra.run.smithi196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T13:35:57.094 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:35:57.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.095+0000 7fe857200640 1 -- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:6800/1325487575,v1:172.21.15.203:6801/1325487575] conn(0x7fe834074d10 msgr2=0x7fe834077150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.095+0000 7fe857200640 1 --2- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:6800/1325487575,v1:172.21.15.203:6801/1325487575] conn(0x7fe834074d10 0x7fe834077150 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.095+0000 7fe857200640 1 -- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe83406c9d0 msgr2=0x7fe83406ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.095+0000 7fe857200640 1 --2- 172.21.15.196:0/3710253143 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fe83406c9d0 0x7fe83406ee90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe848009ad0 tx=0x7fe848008040 comp rx=0 tx=0).stop 2024-04-28T13:35:57.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.095+0000 7fe857200640 1 -- 172.21.15.196:0/3710253143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8580d1760 msgr2=0x7fe858073230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.095+0000 7fe857200640 1 --2- 172.21.15.196:0/3710253143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8580d1760 0x7fe858073230 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe850009fa0 tx=0x7fe850004a40 comp rx=0 tx=0).stop 2024-04-28T13:35:57.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.095+0000 7fe856800640 1 -- 172.21.15.196:0/3710253143 reap_dead start 2024-04-28T13:35:57.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.095+0000 7fe857200640 1 -- 172.21.15.196:0/3710253143 shutdown_connections 2024-04-28T13:35:57.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.095+0000 7fe857200640 1 -- 172.21.15.196:0/3710253143 >> 172.21.15.196:0/3710253143 conn(0x7fe85806c6c0 msgr2=0x7fe85815fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:57.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.095+0000 7fe857200640 1 -- 172.21.15.196:0/3710253143 shutdown_connections 2024-04-28T13:35:57.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.095+0000 7fe857200640 1 -- 172.21.15.196:0/3710253143 wait complete. 2024-04-28T13:35:57.114 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-28T13:35:57.316 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.316+0000 7fb58d600640 1 Processor -- start 2024-04-28T13:35:57.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.316+0000 7fb58d600640 1 -- start start 2024-04-28T13:35:57.387 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.318+0000 7fb58d600640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb5880b5ee0 0x7fb5880b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.318+0000 7fb58d600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58809ec80 0x7fb5880d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.318+0000 7fb58d600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb58815e380 con 0x7fb5880add70 2024-04-28T13:35:57.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.318+0000 7fb58d600640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb5880b4b20 con 0x7fb5880b5ee0 2024-04-28T13:35:57.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.318+0000 7fb58d600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb5880b3f30 con 0x7fb58809ec80 2024-04-28T13:35:57.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.318+0000 7fb587e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb5880b5ee0 0x7fb5880b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.318+0000 7fb58cc00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fb5880add70 0x7fb58806df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:53204/0 (socket says 172.21.15.196:53204) 2024-04-28T13:35:57.388 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.318+0000 7fb58cc00640 1 -- 172.21.15.196:0/3869584177 learned_addr learned my addr 172.21.15.196:0/3869584177 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:57.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.318+0000 7fb587400640 1 --2- 172.21.15.196:0/3869584177 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58809ec80 0x7fb5880d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.319+0000 7fb587e00640 1 -- 172.21.15.196:0/3869584177 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58809ec80 msgr2=0x7fb5880d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:57.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.319+0000 7fb587e00640 1 --2- 172.21.15.196:0/3869584177 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58809ec80 0x7fb5880d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.319+0000 7fb587e00640 1 -- 172.21.15.196:0/3869584177 >> v1:172.21.15.196:6789/0 conn(0x7fb5880add70 legacy=0x7fb58806df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.319+0000 7fb587e00640 1 -- 172.21.15.196:0/3869584177 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5880b41f0 con 0x7fb5880b5ee0 2024-04-28T13:35:57.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.320+0000 7fb587e00640 1 --2- 172.21.15.196:0/3869584177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880b5ee0 0x7fb5880b4f40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb5780097a0 tx=0x7fb57802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11adaf60fcf6fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:57.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.320+0000 7fb586a00640 1 -- 172.21.15.196:0/3869584177 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578037070 con 0x7fb5880b5ee0 2024-04-28T13:35:57.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.320+0000 7fb586a00640 1 -- 172.21.15.196:0/3869584177 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb578032440 con 0x7fb5880b5ee0 2024-04-28T13:35:57.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.320+0000 7fb586a00640 1 -- 172.21.15.196:0/3869584177 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578032810 con 0x7fb5880b5ee0 2024-04-28T13:35:57.389 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.321+0000 7fb58d600640 1 -- 172.21.15.196:0/3869584177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880b5ee0 msgr2=0x7fb5880b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.321+0000 7fb58d600640 1 --2- 172.21.15.196:0/3869584177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880b5ee0 0x7fb5880b4f40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb5780097a0 tx=0x7fb57802a180 comp rx=0 tx=0).stop 2024-04-28T13:35:57.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.321+0000 7fb58d600640 1 -- 172.21.15.196:0/3869584177 shutdown_connections 2024-04-28T13:35:57.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.321+0000 7fb58d600640 1 --2- 172.21.15.196:0/3869584177 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58809ec80 0x7fb5880d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.321+0000 7fb58d600640 1 --2- 172.21.15.196:0/3869584177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880b5ee0 0x7fb5880b4f40 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.321+0000 7fb58d600640 1 -- 172.21.15.196:0/3869584177 >> 172.21.15.196:0/3869584177 conn(0x7fb58806c6c0 msgr2=0x7fb58806cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:57.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.321+0000 7fb58d600640 1 -- 172.21.15.196:0/3869584177 shutdown_connections 2024-04-28T13:35:57.390 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.321+0000 7fb58d600640 1 -- 172.21.15.196:0/3869584177 wait complete. 2024-04-28T13:35:57.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.323+0000 7fb58d600640 1 Processor -- start 2024-04-28T13:35:57.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.323+0000 7fb58d600640 1 -- start start 2024-04-28T13:35:57.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.324+0000 7fb58d600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb58809ec80 0x7fb58814f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.324+0000 7fb58d600640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5880add70 0x7fb58809d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.324+0000 7fb58d600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880b5ee0 0x7fb58809d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.324+0000 7fb58d600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb5880b3c20 con 0x7fb5880b5ee0 2024-04-28T13:35:57.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.324+0000 7fb58d600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5880b47b0 con 0x7fb58809ec80 2024-04-28T13:35:57.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.324+0000 7fb58d600640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb5880d1590 con 0x7fb5880add70 2024-04-28T13:35:57.391 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.324+0000 7fb58cc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880b5ee0 0x7fb58809d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.324+0000 7fb58cc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880b5ee0 0x7fb58809d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39102/0 (socket says 172.21.15.196:39102) 2024-04-28T13:35:57.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.324+0000 7fb58cc00640 1 -- 172.21.15.196:0/3722023662 learned_addr learned my addr 172.21.15.196:0/3722023662 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:57.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.324+0000 7fb58cc00640 1 -- 172.21.15.196:0/3722023662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5880add70 msgr2=0x7fb58809d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.324+0000 7fb58cc00640 1 --2- 172.21.15.196:0/3722023662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5880add70 0x7fb58809d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.325+0000 7fb58cc00640 1 -- 172.21.15.196:0/3722023662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb58809ec80 msgr2=0x7fb58814f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.325+0000 7fb58cc00640 1 --2- 172.21.15.196:0/3722023662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb58809ec80 0x7fb58814f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.325+0000 7fb58cc00640 1 -- 172.21.15.196:0/3722023662 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5880ccfb0 con 0x7fb5880b5ee0 2024-04-28T13:35:57.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.325+0000 7fb58cc00640 1 --2- 172.21.15.196:0/3722023662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880b5ee0 0x7fb58809d9d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb58000cb20 tx=0x7fb58000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:57.392 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.325+0000 7fb57fe00640 1 -- 172.21.15.196:0/3722023662 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580017070 con 0x7fb5880b5ee0 2024-04-28T13:35:57.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.325+0000 7fb57fe00640 1 -- 172.21.15.196:0/3722023662 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb58000aca0 con 0x7fb5880b5ee0 2024-04-28T13:35:57.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.325+0000 7fb57fe00640 1 -- 172.21.15.196:0/3722023662 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580012600 con 0x7fb5880b5ee0 2024-04-28T13:35:57.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.325+0000 7fb58d600640 1 -- 172.21.15.196:0/3722023662 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5880cd290 con 0x7fb5880b5ee0 2024-04-28T13:35:57.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.325+0000 7fb58d600640 1 -- 172.21.15.196:0/3722023662 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5880afbd0 con 0x7fb5880b5ee0 2024-04-28T13:35:57.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.326+0000 7fb58d600640 1 -- 172.21.15.196:0/3722023662 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5880b3a10 con 0x7fb5880b5ee0 2024-04-28T13:35:57.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.327+0000 7fb57fe00640 1 -- 172.21.15.196:0/3722023662 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb58001a070 con 0x7fb5880b5ee0 2024-04-28T13:35:57.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.328+0000 7fb57fe00640 1 --2- 172.21.15.196:0/3722023662 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb56806c980 0x7fb56806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.328+0000 7fb57fe00640 1 -- 172.21.15.196:0/3722023662 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fb58008b870 con 0x7fb5880b5ee0 2024-04-28T13:35:57.393 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.329+0000 7fb587e00640 1 --2- 172.21.15.196:0/3722023662 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb56806c980 0x7fb56806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.330+0000 7fb587e00640 1 --2- 172.21.15.196:0/3722023662 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb56806c980 0x7fb56806ee40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb578034000 tx=0x7fb57802c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:57.394 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.333+0000 7fb57fe00640 1 -- 172.21.15.196:0/3722023662 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb5880b3a10 con 0x7fb5880b5ee0 2024-04-28T13:35:57.533 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.533+0000 7fb58d600640 1 -- 172.21.15.196:0/3722023662 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fb58806a290 con 0x7fb56806c980 2024-04-28T13:35:57.534 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.535+0000 7fb57fe00640 1 -- 172.21.15.196:0/3722023662 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb58806a290 con 0x7fb56806c980 2024-04-28T13:35:57.535 INFO:teuthology.orchestra.run.smithi196.stderr:instructed osd(s) 4 to deep-scrub 2024-04-28T13:35:57.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.537+0000 7fb58d600640 1 -- 172.21.15.196:0/3722023662 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb56806c980 msgr2=0x7fb56806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.537+0000 7fb58d600640 1 --2- 172.21.15.196:0/3722023662 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb56806c980 0x7fb56806ee40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb578034000 tx=0x7fb57802c000 comp rx=0 tx=0).stop 2024-04-28T13:35:57.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.537+0000 7fb58d600640 1 -- 172.21.15.196:0/3722023662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880b5ee0 msgr2=0x7fb58809d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.537+0000 7fb58d600640 1 --2- 172.21.15.196:0/3722023662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880b5ee0 0x7fb58809d9d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb58000cb20 tx=0x7fb58000a060 comp rx=0 tx=0).stop 2024-04-28T13:35:57.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.538+0000 7fb58d600640 1 -- 172.21.15.196:0/3722023662 shutdown_connections 2024-04-28T13:35:57.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.538+0000 7fb58d600640 1 --2- 172.21.15.196:0/3722023662 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb56806c980 0x7fb56806ee40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.538+0000 7fb58d600640 1 --2- 172.21.15.196:0/3722023662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880b5ee0 0x7fb58809d9d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.538 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.538+0000 7fb58d600640 1 --2- 172.21.15.196:0/3722023662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5880add70 0x7fb58809d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.538+0000 7fb58d600640 1 --2- 172.21.15.196:0/3722023662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb58809ec80 0x7fb58814f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.538+0000 7fb58d600640 1 -- 172.21.15.196:0/3722023662 >> 172.21.15.196:0/3722023662 conn(0x7fb58806c6c0 msgr2=0x7fb58815d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:57.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.538+0000 7fb58d600640 1 -- 172.21.15.196:0/3722023662 shutdown_connections 2024-04-28T13:35:57.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.538+0000 7fb58d600640 1 -- 172.21.15.196:0/3722023662 wait complete. 2024-04-28T13:35:57.555 INFO:tasks.ceph:Scrubbing osd.5 2024-04-28T13:35:57.556 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-28T13:35:57.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.750+0000 7fc305000640 1 Processor -- start 2024-04-28T13:35:57.750 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.750+0000 7fc305000640 1 -- start start 2024-04-28T13:35:57.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.752+0000 7fc305000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc3000b50c0 0x7fc3000d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.752+0000 7fc305000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3000cfb40 0x7fc3000b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.752+0000 7fc305000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3000d3900 con 0x7fc3001629c0 2024-04-28T13:35:57.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.752+0000 7fc305000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc3000d3be0 con 0x7fc3000b50c0 2024-04-28T13:35:57.752 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.752+0000 7fc305000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc3000b5580 con 0x7fc3000cfb40 2024-04-28T13:35:57.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.753+0000 7fc2f7e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc3000b50c0 0x7fc3000d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.753+0000 7fc2ffe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3000cfb40 0x7fc3000b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.753+0000 7fc2f7e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc3000b50c0 0x7fc3000d5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.196:55136/0 (socket says 172.21.15.196:55136) 2024-04-28T13:35:57.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.753+0000 7fc2f7e00640 1 -- 172.21.15.196:0/2205124740 learned_addr learned my addr 172.21.15.196:0/2205124740 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:57.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.754+0000 7fc2f7e00640 1 -- 172.21.15.196:0/2205124740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3000cfb40 msgr2=0x7fc3000b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:57.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.754+0000 7fc2f7e00640 1 --2- 172.21.15.196:0/2205124740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3000cfb40 0x7fc3000b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.754+0000 7fc2f7e00640 1 -- 172.21.15.196:0/2205124740 >> v1:172.21.15.203:6789/0 conn(0x7fc3001629c0 legacy=0x7fc3000b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.754 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.754+0000 7fc2f7e00640 1 -- 172.21.15.196:0/2205124740 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3000b4140 con 0x7fc3000b50c0 2024-04-28T13:35:57.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.755+0000 7fc2f7e00640 1 --2- 172.21.15.196:0/2205124740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3000b50c0 0x7fc3000d5e00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009870 tx=0x7fc2e802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40d23dbad25d907f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:57.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.756+0000 7fc2fea00640 1 -- 172.21.15.196:0/2205124740 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8037070 con 0x7fc3000b50c0 2024-04-28T13:35:57.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.756+0000 7fc2fea00640 1 -- 172.21.15.196:0/2205124740 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e8032440 con 0x7fc3000b50c0 2024-04-28T13:35:57.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.756+0000 7fc2fea00640 1 -- 172.21.15.196:0/2205124740 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8032810 con 0x7fc3000b50c0 2024-04-28T13:35:57.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.756+0000 7fc305000640 1 -- 172.21.15.196:0/2205124740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3000b50c0 msgr2=0x7fc3000d5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.756 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.756+0000 7fc305000640 1 --2- 172.21.15.196:0/2205124740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3000b50c0 0x7fc3000d5e00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009870 tx=0x7fc2e802a250 comp rx=0 tx=0).stop 2024-04-28T13:35:57.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.757+0000 7fc305000640 1 -- 172.21.15.196:0/2205124740 shutdown_connections 2024-04-28T13:35:57.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.757+0000 7fc305000640 1 --2- 172.21.15.196:0/2205124740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3000cfb40 0x7fc3000b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.757+0000 7fc305000640 1 --2- 172.21.15.196:0/2205124740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3000b50c0 0x7fc3000d5e00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.757+0000 7fc305000640 1 -- 172.21.15.196:0/2205124740 >> 172.21.15.196:0/2205124740 conn(0x7fc30009e570 msgr2=0x7fc3000d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:57.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.757+0000 7fc305000640 1 -- 172.21.15.196:0/2205124740 shutdown_connections 2024-04-28T13:35:57.757 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.757+0000 7fc305000640 1 -- 172.21.15.196:0/2205124740 wait complete. 2024-04-28T13:35:57.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.759+0000 7fc305000640 1 Processor -- start 2024-04-28T13:35:57.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.759+0000 7fc305000640 1 -- start start 2024-04-28T13:35:57.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.760+0000 7fc305000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3000b50c0 0x7fc3001f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.760+0000 7fc305000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3000cfb40 0x7fc3000c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.760+0000 7fc305000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3001629c0 0x7fc3000cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.760+0000 7fc305000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc3000d4040 con 0x7fc3000b50c0 2024-04-28T13:35:57.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.760+0000 7fc305000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc3000d3ec0 con 0x7fc3000cfb40 2024-04-28T13:35:57.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.760+0000 7fc305000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc3000b3880 con 0x7fc3001629c0 2024-04-28T13:35:57.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.761+0000 7fc2ff400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3000b50c0 0x7fc3001f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.761+0000 7fc2ff400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3000b50c0 0x7fc3001f4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39112/0 (socket says 172.21.15.196:39112) 2024-04-28T13:35:57.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.761+0000 7fc2ff400640 1 -- 172.21.15.196:0/674007172 learned_addr learned my addr 172.21.15.196:0/674007172 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:57.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.761+0000 7fc2f7e00640 1 --2- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3000cfb40 0x7fc3000c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.761+0000 7fc2ffe00640 1 --2- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3001629c0 0x7fc3000cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.761+0000 7fc2ff400640 1 -- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3001629c0 msgr2=0x7fc3000cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.761+0000 7fc2ff400640 1 --2- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3001629c0 0x7fc3000cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.761+0000 7fc2ff400640 1 -- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3000cfb40 msgr2=0x7fc3000c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.761+0000 7fc2ff400640 1 --2- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3000cfb40 0x7fc3000c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.761+0000 7fc2ff400640 1 -- 172.21.15.196:0/674007172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3001f53c0 con 0x7fc3000b50c0 2024-04-28T13:35:57.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.761+0000 7fc2ff400640 1 --2- 172.21.15.196:0/674007172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3000b50c0 0x7fc3001f4d70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000c730 tx=0x7fc2f000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:57.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.762+0000 7fc2f7400640 1 -- 172.21.15.196:0/674007172 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0016020 con 0x7fc3000b50c0 2024-04-28T13:35:57.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.762+0000 7fc2f7400640 1 -- 172.21.15.196:0/674007172 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f0006d50 con 0x7fc3000b50c0 2024-04-28T13:35:57.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.762+0000 7fc305000640 1 -- 172.21.15.196:0/674007172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3001f56a0 con 0x7fc3000b50c0 2024-04-28T13:35:57.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.762+0000 7fc2f7400640 1 -- 172.21.15.196:0/674007172 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0003bb0 con 0x7fc3000b50c0 2024-04-28T13:35:57.764 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.762+0000 7fc305000640 1 -- 172.21.15.196:0/674007172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3001f5c60 con 0x7fc3000b50c0 2024-04-28T13:35:57.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.763+0000 7fc305000640 1 -- 172.21.15.196:0/674007172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc300050730 con 0x7fc3000b50c0 2024-04-28T13:35:57.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.764+0000 7fc2f7400640 1 -- 172.21.15.196:0/674007172 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc2f000a850 con 0x7fc3000b50c0 2024-04-28T13:35:57.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.765+0000 7fc2f7400640 1 --2- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc2dc06c980 0x7fc2dc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.765 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.765+0000 7fc2f7400640 1 -- 172.21.15.196:0/674007172 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fc2f008b8e0 con 0x7fc3000b50c0 2024-04-28T13:35:57.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.765+0000 7fc2f7400640 1 --2- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:6824/2058013113,v1:172.21.15.203:6825/2058013113] conn(0x7fc2dc074be0 0x7fc2dc077020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.765+0000 7fc2f7400640 1 -- 172.21.15.196:0/674007172 --> [v2:172.21.15.203:6824/2058013113,v1:172.21.15.203:6825/2058013113] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2f001eec0 con 0x7fc2dc074be0 2024-04-28T13:35:57.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.765+0000 7fc2f7400640 1 -- 172.21.15.196:0/674007172 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc2f00902a0 con 0x7fc3000b50c0 2024-04-28T13:35:57.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.766+0000 7fc2f7e00640 1 --2- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc2dc06c980 0x7fc2dc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.766+0000 7fc2ffe00640 1 --2- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:6824/2058013113,v1:172.21.15.203:6825/2058013113] conn(0x7fc2dc074be0 0x7fc2dc077020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:57.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.767+0000 7fc2ffe00640 1 --2- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:6824/2058013113,v1:172.21.15.203:6825/2058013113] conn(0x7fc2dc074be0 0x7fc2dc077020 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:57.767 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.767+0000 7fc2f7e00640 1 --2- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc2dc06c980 0x7fc2dc06ee40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8034000 tx=0x7fc2e802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:57.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.769+0000 7fc2f7400640 1 -- 172.21.15.196:0/674007172 <== osd.5 v2:172.21.15.203:6824/2058013113 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2f001eec0 con 0x7fc2dc074be0 2024-04-28T13:35:57.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.784+0000 7fc305000640 1 -- 172.21.15.196:0/674007172 --> [v2:172.21.15.203:6824/2058013113,v1:172.21.15.203:6825/2058013113] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc3001610f0 con 0x7fc2dc074be0 2024-04-28T13:35:57.784 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.785+0000 7fc2f7400640 1 -- 172.21.15.196:0/674007172 <== osd.5 v2:172.21.15.203:6824/2058013113 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc3001610f0 con 0x7fc2dc074be0 2024-04-28T13:35:57.785 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:35:57.785 INFO:teuthology.orchestra.run.smithi196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T13:35:57.785 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:35:57.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.785+0000 7fc305000640 1 -- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:6824/2058013113,v1:172.21.15.203:6825/2058013113] conn(0x7fc2dc074be0 msgr2=0x7fc2dc077020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.785+0000 7fc305000640 1 --2- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:6824/2058013113,v1:172.21.15.203:6825/2058013113] conn(0x7fc2dc074be0 0x7fc2dc077020 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.785+0000 7fc305000640 1 -- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc2dc06c980 msgr2=0x7fc2dc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.785+0000 7fc305000640 1 --2- 172.21.15.196:0/674007172 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fc2dc06c980 0x7fc2dc06ee40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8034000 tx=0x7fc2e802e000 comp rx=0 tx=0).stop 2024-04-28T13:35:57.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.785+0000 7fc305000640 1 -- 172.21.15.196:0/674007172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3000b50c0 msgr2=0x7fc3001f4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:57.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.785+0000 7fc305000640 1 --2- 172.21.15.196:0/674007172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3000b50c0 0x7fc3001f4d70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000c730 tx=0x7fc2f000cbf0 comp rx=0 tx=0).stop 2024-04-28T13:35:57.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.785+0000 7fc2ffe00640 1 -- 172.21.15.196:0/674007172 reap_dead start 2024-04-28T13:35:57.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.786+0000 7fc305000640 1 -- 172.21.15.196:0/674007172 shutdown_connections 2024-04-28T13:35:57.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.786+0000 7fc305000640 1 -- 172.21.15.196:0/674007172 >> 172.21.15.196:0/674007172 conn(0x7fc30009e570 msgr2=0x7fc30006d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:57.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.786+0000 7fc305000640 1 -- 172.21.15.196:0/674007172 shutdown_connections 2024-04-28T13:35:57.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.786+0000 7fc305000640 1 -- 172.21.15.196:0/674007172 wait complete. 2024-04-28T13:35:57.803 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-28T13:35:57.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.997+0000 7fa0a6000640 1 Processor -- start 2024-04-28T13:35:57.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.997+0000 7fa0a6000640 1 -- start start 2024-04-28T13:35:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.999+0000 7fa0a6000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa0a00d0540 0x7fa0a00af560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.999+0000 7fa0a6000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a00afaa0 con 0x7fa0a00b3c20 2024-04-28T13:35:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.999+0000 7fa0a6000640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a00b47a0 con 0x7fa0a0073370 2024-04-28T13:35:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.999+0000 7fa0a6000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa0a00738e0 con 0x7fa0a00d0540 2024-04-28T13:35:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.999+0000 7fa0a5600640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa0a00b3c20 0x7fa0a006ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:53212/0 (socket says 172.21.15.196:53212) 2024-04-28T13:35:57.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:57.999+0000 7fa0a5600640 1 -- 172.21.15.196:0/3600043207 learned_addr learned my addr 172.21.15.196:0/3600043207 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:58.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa0a4c00640 1 --2- 172.21.15.196:0/3600043207 >> v2:172.21.15.203:3300/0 conn(0x7fa0a00d0540 0x7fa0a00af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa09f400640 1 -- 172.21.15.196:0/3600043207 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2560096070 0 0) 0x7fa0a00afaa0 con 0x7fa0a00b3c20 2024-04-28T13:35:58.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa09f400640 1 -- 172.21.15.196:0/3600043207 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa088003520 con 0x7fa0a00b3c20 2024-04-28T13:35:58.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa09f400640 1 -- 172.21.15.196:0/3600043207 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3846485161 0 0) 0x7fa088003520 con 0x7fa0a00b3c20 2024-04-28T13:35:58.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa09f400640 1 -- 172.21.15.196:0/3600043207 >> v2:172.21.15.203:3300/0 conn(0x7fa0a00d0540 msgr2=0x7fa0a00af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:58.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa09f400640 1 --2- 172.21.15.196:0/3600043207 >> v2:172.21.15.203:3300/0 conn(0x7fa0a00d0540 0x7fa0a00af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa09f400640 1 -- 172.21.15.196:0/3600043207 >> v1:172.21.15.203:6789/0 conn(0x7fa0a0073370 legacy=0x7fa0a0072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa09f400640 1 -- 172.21.15.196:0/3600043207 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a00aea80 con 0x7fa0a00b3c20 2024-04-28T13:35:58.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa09f400640 1 -- 172.21.15.196:0/3600043207 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7fa0980019f0 con 0x7fa0a00b3c20 2024-04-28T13:35:58.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa09f400640 1 -- 172.21.15.196:0/3600043207 >> v1:172.21.15.196:6789/0 conn(0x7fa0a00b3c20 legacy=0x7fa0a006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa09f400640 1 --2- 172.21.15.196:0/3600043207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0880039b0 0x7fa088003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa09f400640 1 --2- 172.21.15.196:0/3600043207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa088004800 0x7fa088004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa09f400640 1 --2- 172.21.15.196:0/3600043207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0880051c0 0x7fa08800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa09f400640 1 -- 172.21.15.196:0/3600043207 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a00738e0 con 0x7fa0880051c0 2024-04-28T13:35:58.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa09f400640 1 -- 172.21.15.196:0/3600043207 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa08800c060 con 0x7fa0880039b0 2024-04-28T13:35:58.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.000+0000 7fa09f400640 1 -- 172.21.15.196:0/3600043207 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa08800c260 con 0x7fa088004800 2024-04-28T13:35:58.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.001+0000 7fa0a5600640 1 --2- 172.21.15.196:0/3600043207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0880051c0 0x7fa08800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.001+0000 7fa0a4c00640 1 --2- 172.21.15.196:0/3600043207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0880039b0 0x7fa088003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.001+0000 7fa09fe00640 1 --2- 172.21.15.196:0/3600043207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa088004800 0x7fa088004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.001+0000 7fa0a5600640 1 -- 172.21.15.196:0/3600043207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa088004800 msgr2=0x7fa088004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.001+0000 7fa0a5600640 1 --2- 172.21.15.196:0/3600043207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa088004800 0x7fa088004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.001+0000 7fa0a5600640 1 -- 172.21.15.196:0/3600043207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0880039b0 msgr2=0x7fa088003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.001+0000 7fa0a5600640 1 --2- 172.21.15.196:0/3600043207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0880039b0 0x7fa088003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.001+0000 7fa0a5600640 1 -- 172.21.15.196:0/3600043207 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa088003520 con 0x7fa0880051c0 2024-04-28T13:35:58.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.001+0000 7fa09fe00640 1 --2- 172.21.15.196:0/3600043207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa088004800 0x7fa088004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:35:58.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.002+0000 7fa0a5600640 1 -- 172.21.15.196:0/3600043207 reap_dead start 2024-04-28T13:35:58.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.003+0000 7fa0a5600640 1 --2- 172.21.15.196:0/3600043207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0880051c0 0x7fa08800b9c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa09800a860 tx=0x7fa088005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a24a2638dc7f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:58.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.003+0000 7fa09f400640 1 -- 172.21.15.196:0/3600043207 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09800a480 con 0x7fa0880051c0 2024-04-28T13:35:58.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.003+0000 7fa09f400640 1 -- 172.21.15.196:0/3600043207 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa09802f070 con 0x7fa0880051c0 2024-04-28T13:35:58.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.003+0000 7fa09f400640 1 -- 172.21.15.196:0/3600043207 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09802bca0 con 0x7fa0880051c0 2024-04-28T13:35:58.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.003+0000 7fa0a6000640 1 -- 172.21.15.196:0/3600043207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0880051c0 msgr2=0x7fa08800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.003+0000 7fa0a6000640 1 --2- 172.21.15.196:0/3600043207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0880051c0 0x7fa08800b9c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa09800a860 tx=0x7fa088005830 comp rx=0 tx=0).stop 2024-04-28T13:35:58.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.004+0000 7fa0a6000640 1 -- 172.21.15.196:0/3600043207 shutdown_connections 2024-04-28T13:35:58.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.004+0000 7fa0a6000640 1 --2- 172.21.15.196:0/3600043207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0880051c0 0x7fa08800b9c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.004+0000 7fa0a6000640 1 -- 172.21.15.196:0/3600043207 >> 172.21.15.196:0/3600043207 conn(0x7fa0a006c6c0 msgr2=0x7fa0a006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:58.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.004+0000 7fa0a6000640 1 -- 172.21.15.196:0/3600043207 shutdown_connections 2024-04-28T13:35:58.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.004+0000 7fa0a6000640 1 -- 172.21.15.196:0/3600043207 wait complete. 2024-04-28T13:35:58.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.006+0000 7fa0a6000640 1 Processor -- start 2024-04-28T13:35:58.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.006+0000 7fa0a6000640 1 -- start start 2024-04-28T13:35:58.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.007+0000 7fa0a6000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0880051c0 0x7fa0a015ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.007+0000 7fa0a6000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0a009d670 0x7fa0a014f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.007+0000 7fa0a6000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0a0157c30 0x7fa0a01580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.007+0000 7fa0a6000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a00d0920 con 0x7fa0880051c0 2024-04-28T13:35:58.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.007+0000 7fa0a6000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a00af040 con 0x7fa0a0157c30 2024-04-28T13:35:58.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.007+0000 7fa0a6000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a015e1d0 con 0x7fa0a009d670 2024-04-28T13:35:58.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.007+0000 7fa0a4c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0880051c0 0x7fa0a015ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.007+0000 7fa0a4c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0880051c0 0x7fa0a015ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39128/0 (socket says 172.21.15.196:39128) 2024-04-28T13:35:58.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.007+0000 7fa0a4c00640 1 -- 172.21.15.196:0/2307384706 learned_addr learned my addr 172.21.15.196:0/2307384706 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:58.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.008+0000 7fa0a4c00640 1 -- 172.21.15.196:0/2307384706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0a009d670 msgr2=0x7fa0a014f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.008+0000 7fa09fe00640 1 --2- 172.21.15.196:0/2307384706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0a009d670 0x7fa0a014f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.008+0000 7fa0a4c00640 1 --2- 172.21.15.196:0/2307384706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0a009d670 0x7fa0a014f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.008+0000 7fa0a4c00640 1 -- 172.21.15.196:0/2307384706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0a0157c30 msgr2=0x7fa0a01580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.008+0000 7fa0a4c00640 1 --2- 172.21.15.196:0/2307384706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0a0157c30 0x7fa0a01580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.008+0000 7fa0a4c00640 1 -- 172.21.15.196:0/2307384706 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a014fea0 con 0x7fa0880051c0 2024-04-28T13:35:58.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.008+0000 7fa0a4c00640 1 --2- 172.21.15.196:0/2307384706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0880051c0 0x7fa0a015ce70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa094011540 tx=0x7fa094011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:58.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.008+0000 7fa09cc00640 1 -- 172.21.15.196:0/2307384706 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094015930 con 0x7fa0880051c0 2024-04-28T13:35:58.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.008+0000 7fa09cc00640 1 -- 172.21.15.196:0/2307384706 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa094015ad0 con 0x7fa0880051c0 2024-04-28T13:35:58.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.009+0000 7fa0a6000640 1 -- 172.21.15.196:0/2307384706 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a015d5c0 con 0x7fa0880051c0 2024-04-28T13:35:58.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.009+0000 7fa09cc00640 1 -- 172.21.15.196:0/2307384706 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09400f460 con 0x7fa0880051c0 2024-04-28T13:35:58.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.009+0000 7fa0a6000640 1 -- 172.21.15.196:0/2307384706 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a015db80 con 0x7fa0880051c0 2024-04-28T13:35:58.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.010+0000 7fa0a6000640 1 -- 172.21.15.196:0/2307384706 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a006a290 con 0x7fa0880051c0 2024-04-28T13:35:58.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.011+0000 7fa09cc00640 1 -- 172.21.15.196:0/2307384706 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa09400f600 con 0x7fa0880051c0 2024-04-28T13:35:58.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.012+0000 7fa09cc00640 1 --2- 172.21.15.196:0/2307384706 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa08006c9b0 0x7fa08006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.013+0000 7fa09cc00640 1 -- 172.21.15.196:0/2307384706 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fa09408c490 con 0x7fa0880051c0 2024-04-28T13:35:58.013 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.013+0000 7fa09fe00640 1 --2- 172.21.15.196:0/2307384706 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa08006c9b0 0x7fa08006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.015+0000 7fa09fe00640 1 --2- 172.21.15.196:0/2307384706 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa08006c9b0 0x7fa08006ee70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa08c000f60 tx=0x7fa08c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:58.018 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.018+0000 7fa09cc00640 1 -- 172.21.15.196:0/2307384706 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa094019030 con 0x7fa0880051c0 2024-04-28T13:35:58.223 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.223+0000 7fa0a6000640 1 -- 172.21.15.196:0/2307384706 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fa0a01585f0 con 0x7fa08006c9b0 2024-04-28T13:35:58.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.224+0000 7fa09cc00640 1 -- 172.21.15.196:0/2307384706 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa08c00e4c0 con 0x7fa08006c9b0 2024-04-28T13:35:58.224 INFO:teuthology.orchestra.run.smithi196.stderr:instructed osd(s) 5 to deep-scrub 2024-04-28T13:35:58.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.227+0000 7fa0a6000640 1 -- 172.21.15.196:0/2307384706 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa08006c9b0 msgr2=0x7fa08006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.227+0000 7fa0a6000640 1 --2- 172.21.15.196:0/2307384706 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa08006c9b0 0x7fa08006ee70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa08c000f60 tx=0x7fa08c009040 comp rx=0 tx=0).stop 2024-04-28T13:35:58.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.227+0000 7fa0a6000640 1 -- 172.21.15.196:0/2307384706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0880051c0 msgr2=0x7fa0a015ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.227+0000 7fa0a6000640 1 --2- 172.21.15.196:0/2307384706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0880051c0 0x7fa0a015ce70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa094011540 tx=0x7fa094011a00 comp rx=0 tx=0).stop 2024-04-28T13:35:58.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.227+0000 7fa0a6000640 1 -- 172.21.15.196:0/2307384706 shutdown_connections 2024-04-28T13:35:58.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.227+0000 7fa0a6000640 1 --2- 172.21.15.196:0/2307384706 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fa08006c9b0 0x7fa08006ee70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.227+0000 7fa0a6000640 1 --2- 172.21.15.196:0/2307384706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0a0157c30 0x7fa0a01580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.227+0000 7fa0a6000640 1 --2- 172.21.15.196:0/2307384706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0a009d670 0x7fa0a014f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.227+0000 7fa0a6000640 1 --2- 172.21.15.196:0/2307384706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0880051c0 0x7fa0a015ce70 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.227+0000 7fa0a6000640 1 -- 172.21.15.196:0/2307384706 >> 172.21.15.196:0/2307384706 conn(0x7fa0a006c6c0 msgr2=0x7fa0a01603d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:58.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.227+0000 7fa0a6000640 1 -- 172.21.15.196:0/2307384706 shutdown_connections 2024-04-28T13:35:58.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.227+0000 7fa0a6000640 1 -- 172.21.15.196:0/2307384706 wait complete. 2024-04-28T13:35:58.244 INFO:tasks.ceph:Scrubbing osd.6 2024-04-28T13:35:58.244 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-28T13:35:58.454 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.455+0000 7fb98fe00640 1 Processor -- start 2024-04-28T13:35:58.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.455+0000 7fb98fe00640 1 -- start start 2024-04-28T13:35:58.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.456+0000 7fb98fe00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb9900b5ee0 0x7fb9900b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.456+0000 7fb98fe00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb9900d1760 0x7fb99006df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.457+0000 7fb98fe00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb99015e380 con 0x7fb99009ec80 2024-04-28T13:35:58.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.457+0000 7fb98fe00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb9900b4b20 con 0x7fb9900d1760 2024-04-28T13:35:58.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.457+0000 7fb98fe00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb9900b3f30 con 0x7fb9900b5ee0 2024-04-28T13:35:58.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.457+0000 7fb98f400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb9900d1760 0x7fb99006df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.457+0000 7fb98f400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb9900d1760 0x7fb99006df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39132/0 (socket says 172.21.15.196:39132) 2024-04-28T13:35:58.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.457+0000 7fb98f400640 1 -- 172.21.15.196:0/103132979 learned_addr learned my addr 172.21.15.196:0/103132979 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:58.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.457+0000 7fb98ea00640 1 --2- 172.21.15.196:0/103132979 >> v2:172.21.15.203:3300/0 conn(0x7fb9900b5ee0 0x7fb9900b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.457+0000 7fb98f400640 1 -- 172.21.15.196:0/103132979 >> v2:172.21.15.203:3300/0 conn(0x7fb9900b5ee0 msgr2=0x7fb9900b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:58.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.457+0000 7fb98f400640 1 --2- 172.21.15.196:0/103132979 >> v2:172.21.15.203:3300/0 conn(0x7fb9900b5ee0 0x7fb9900b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.457+0000 7fb98f400640 1 -- 172.21.15.196:0/103132979 >> v1:172.21.15.203:6789/0 conn(0x7fb99009ec80 legacy=0x7fb9900d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:58.458 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.457+0000 7fb98f400640 1 -- 172.21.15.196:0/103132979 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9900b41f0 con 0x7fb9900d1760 2024-04-28T13:35:58.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.458+0000 7fb98f400640 1 --2- 172.21.15.196:0/103132979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9900d1760 0x7fb99006df50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb99006eda0 tx=0x7fb98802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5177abd5cb6275 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:58.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.459+0000 7fb98d600640 1 -- 172.21.15.196:0/103132979 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988036070 con 0x7fb9900d1760 2024-04-28T13:35:58.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.459+0000 7fb98d600640 1 -- 172.21.15.196:0/103132979 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb988031600 con 0x7fb9900d1760 2024-04-28T13:35:58.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.459+0000 7fb98d600640 1 -- 172.21.15.196:0/103132979 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9880319d0 con 0x7fb9900d1760 2024-04-28T13:35:58.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.459+0000 7fb98fe00640 1 -- 172.21.15.196:0/103132979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9900d1760 msgr2=0x7fb99006df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.459+0000 7fb98fe00640 1 --2- 172.21.15.196:0/103132979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9900d1760 0x7fb99006df50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb99006eda0 tx=0x7fb98802a6f0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.459+0000 7fb98fe00640 1 -- 172.21.15.196:0/103132979 shutdown_connections 2024-04-28T13:35:58.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.459+0000 7fb98fe00640 1 --2- 172.21.15.196:0/103132979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9900d1760 0x7fb99006df50 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.459+0000 7fb98fe00640 1 --2- 172.21.15.196:0/103132979 >> v2:172.21.15.203:3300/0 conn(0x7fb9900b5ee0 0x7fb9900b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.459+0000 7fb98fe00640 1 -- 172.21.15.196:0/103132979 >> 172.21.15.196:0/103132979 conn(0x7fb99006c6c0 msgr2=0x7fb99006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:58.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.460+0000 7fb98fe00640 1 -- 172.21.15.196:0/103132979 shutdown_connections 2024-04-28T13:35:58.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.460+0000 7fb98fe00640 1 -- 172.21.15.196:0/103132979 wait complete. 2024-04-28T13:35:58.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.461+0000 7fb98fe00640 1 Processor -- start 2024-04-28T13:35:58.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.461+0000 7fb98fe00640 1 -- start start 2024-04-28T13:35:58.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.462+0000 7fb98fe00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb99009ec80 0x7fb9900cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.462+0000 7fb98fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9900b5ee0 0x7fb99014da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.462+0000 7fb98fe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9900d1760 0x7fb990148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.462+0000 7fb98fe00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9900b3c20 con 0x7fb9900b5ee0 2024-04-28T13:35:58.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.462+0000 7fb98fe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb9900b47b0 con 0x7fb9900d1760 2024-04-28T13:35:58.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.462+0000 7fb98fe00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb9900add70 con 0x7fb99009ec80 2024-04-28T13:35:58.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.462+0000 7fb98e000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9900b5ee0 0x7fb99014da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.463+0000 7fb98e000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9900b5ee0 0x7fb99014da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39140/0 (socket says 172.21.15.196:39140) 2024-04-28T13:35:58.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.463+0000 7fb98e000640 1 -- 172.21.15.196:0/7214278 learned_addr learned my addr 172.21.15.196:0/7214278 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:58.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.463+0000 7fb98e000640 1 -- 172.21.15.196:0/7214278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb99009ec80 msgr2=0x7fb9900cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.463+0000 7fb98e000640 1 --2- 172.21.15.196:0/7214278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb99009ec80 0x7fb9900cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.463+0000 7fb98e000640 1 -- 172.21.15.196:0/7214278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9900d1760 msgr2=0x7fb990148dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.463+0000 7fb98e000640 1 --2- 172.21.15.196:0/7214278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9900d1760 0x7fb990148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.463+0000 7fb98e000640 1 -- 172.21.15.196:0/7214278 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9900ce140 con 0x7fb9900b5ee0 2024-04-28T13:35:58.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.463+0000 7fb98e000640 1 --2- 172.21.15.196:0/7214278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9900b5ee0 0x7fb99014da80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00c400 tx=0x7fb97c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:58.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.463+0000 7fb986a00640 1 -- 172.21.15.196:0/7214278 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c017070 con 0x7fb9900b5ee0 2024-04-28T13:35:58.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.463+0000 7fb986a00640 1 -- 172.21.15.196:0/7214278 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97c0035f0 con 0x7fb9900b5ee0 2024-04-28T13:35:58.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.463+0000 7fb986a00640 1 -- 172.21.15.196:0/7214278 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c0039e0 con 0x7fb9900b5ee0 2024-04-28T13:35:58.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.463+0000 7fb98fe00640 1 -- 172.21.15.196:0/7214278 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9900ce420 con 0x7fb9900b5ee0 2024-04-28T13:35:58.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.464+0000 7fb98fe00640 1 -- 172.21.15.196:0/7214278 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9900cef00 con 0x7fb9900b5ee0 2024-04-28T13:35:58.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.464+0000 7fb98fe00640 1 -- 172.21.15.196:0/7214278 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb990150730 con 0x7fb9900b5ee0 2024-04-28T13:35:58.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.466+0000 7fb986a00640 1 -- 172.21.15.196:0/7214278 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb97c003b80 con 0x7fb9900b5ee0 2024-04-28T13:35:58.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.467+0000 7fb986a00640 1 --2- 172.21.15.196:0/7214278 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb97006c9b0 0x7fb97006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.467+0000 7fb986a00640 1 -- 172.21.15.196:0/7214278 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fb97c08af10 con 0x7fb9900b5ee0 2024-04-28T13:35:58.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.467+0000 7fb986a00640 1 --2- 172.21.15.196:0/7214278 >> [v2:172.21.15.203:6804/2209636555,v1:172.21.15.203:6807/2209636555] conn(0x7fb970074c10 0x7fb970077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.467+0000 7fb986a00640 1 -- 172.21.15.196:0/7214278 --> [v2:172.21.15.203:6804/2209636555,v1:172.21.15.203:6807/2209636555] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb970077720 con 0x7fb970074c10 2024-04-28T13:35:58.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.467+0000 7fb986a00640 1 -- 172.21.15.196:0/7214278 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb97c08d530 con 0x7fb9900b5ee0 2024-04-28T13:35:58.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.467+0000 7fb98ea00640 1 --2- 172.21.15.196:0/7214278 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb97006c9b0 0x7fb97006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.468+0000 7fb98f400640 1 --2- 172.21.15.196:0/7214278 >> [v2:172.21.15.203:6804/2209636555,v1:172.21.15.203:6807/2209636555] conn(0x7fb970074c10 0x7fb970077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.469+0000 7fb98f400640 1 --2- 172.21.15.196:0/7214278 >> [v2:172.21.15.203:6804/2209636555,v1:172.21.15.203:6807/2209636555] conn(0x7fb970074c10 0x7fb970077050 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:58.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.469+0000 7fb98ea00640 1 --2- 172.21.15.196:0/7214278 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb97006c9b0 0x7fb97006ee70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb980006fd0 tx=0x7fb980008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:58.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.471+0000 7fb986a00640 1 -- 172.21.15.196:0/7214278 <== osd.6 v2:172.21.15.203:6804/2209636555 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb970077720 con 0x7fb970074c10 2024-04-28T13:35:58.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.486+0000 7fb98fe00640 1 -- 172.21.15.196:0/7214278 --> [v2:172.21.15.203:6804/2209636555,v1:172.21.15.203:6807/2209636555] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb990050730 con 0x7fb970074c10 2024-04-28T13:35:58.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.487+0000 7fb986a00640 1 -- 172.21.15.196:0/7214278 <== osd.6 v2:172.21.15.203:6804/2209636555 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb990050730 con 0x7fb970074c10 2024-04-28T13:35:58.486 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:35:58.487 INFO:teuthology.orchestra.run.smithi196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T13:35:58.487 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:35:58.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.487+0000 7fb98fe00640 1 -- 172.21.15.196:0/7214278 >> [v2:172.21.15.203:6804/2209636555,v1:172.21.15.203:6807/2209636555] conn(0x7fb970074c10 msgr2=0x7fb970077050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.487+0000 7fb98fe00640 1 --2- 172.21.15.196:0/7214278 >> [v2:172.21.15.203:6804/2209636555,v1:172.21.15.203:6807/2209636555] conn(0x7fb970074c10 0x7fb970077050 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.488+0000 7fb98fe00640 1 -- 172.21.15.196:0/7214278 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb97006c9b0 msgr2=0x7fb97006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.488+0000 7fb98fe00640 1 --2- 172.21.15.196:0/7214278 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7fb97006c9b0 0x7fb97006ee70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb980006fd0 tx=0x7fb980008040 comp rx=0 tx=0).stop 2024-04-28T13:35:58.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.488+0000 7fb98fe00640 1 -- 172.21.15.196:0/7214278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9900b5ee0 msgr2=0x7fb99014da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.488+0000 7fb98fe00640 1 --2- 172.21.15.196:0/7214278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9900b5ee0 0x7fb99014da80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00c400 tx=0x7fb97c00c8c0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.488+0000 7fb98f400640 1 -- 172.21.15.196:0/7214278 reap_dead start 2024-04-28T13:35:58.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.488+0000 7fb98fe00640 1 -- 172.21.15.196:0/7214278 shutdown_connections 2024-04-28T13:35:58.488 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.488+0000 7fb98fe00640 1 -- 172.21.15.196:0/7214278 >> 172.21.15.196:0/7214278 conn(0x7fb99006c6c0 msgr2=0x7fb9900d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:58.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.488+0000 7fb98fe00640 1 -- 172.21.15.196:0/7214278 shutdown_connections 2024-04-28T13:35:58.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.488+0000 7fb98fe00640 1 -- 172.21.15.196:0/7214278 wait complete. 2024-04-28T13:35:58.505 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-28T13:35:58.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.708+0000 7f5d0f200640 1 Processor -- start 2024-04-28T13:35:58.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.708+0000 7f5d0f200640 1 -- start start 2024-04-28T13:35:58.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.709+0000 7f5d0f200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5d081629c0 0x7f5d080d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.710+0000 7f5d0f200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d08163bf0 0x7f5d08163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.710+0000 7f5d0f200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d08162da0 con 0x7f5d08163570 2024-04-28T13:35:58.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.710+0000 7f5d0f200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5d080b6190 con 0x7f5d081629c0 2024-04-28T13:35:58.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.710+0000 7f5d0f200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5d080b6470 con 0x7f5d08163bf0 2024-04-28T13:35:58.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.710+0000 7f5d0de00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5d081629c0 0x7f5d080d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.710+0000 7f5d0de00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5d081629c0 0x7f5d080d3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39144/0 (socket says 172.21.15.196:39144) 2024-04-28T13:35:58.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.710+0000 7f5d0de00640 1 -- 172.21.15.196:0/586530802 learned_addr learned my addr 172.21.15.196:0/586530802 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:58.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.710+0000 7f5d0d400640 1 --2- 172.21.15.196:0/586530802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d08163bf0 0x7f5d08163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.711+0000 7f5d0de00640 1 -- 172.21.15.196:0/586530802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d08163bf0 msgr2=0x7f5d08163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:58.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.711+0000 7f5d0de00640 1 --2- 172.21.15.196:0/586530802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d08163bf0 0x7f5d08163030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.711+0000 7f5d0de00640 1 -- 172.21.15.196:0/586530802 >> v1:172.21.15.203:6789/0 conn(0x7f5d08163570 legacy=0x7f5d080b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:58.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.711+0000 7f5d0de00640 1 -- 172.21.15.196:0/586530802 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d080b6750 con 0x7f5d081629c0 2024-04-28T13:35:58.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.712+0000 7f5d0de00640 1 --2- 172.21.15.196:0/586530802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d081629c0 0x7f5d080d3710 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009e50 tx=0x7f5d0002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0581bdf2fbe813f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:58.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.712+0000 7f5d0ca00640 1 -- 172.21.15.196:0/586530802 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00036070 con 0x7f5d081629c0 2024-04-28T13:35:58.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.712+0000 7f5d0ca00640 1 -- 172.21.15.196:0/586530802 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d00031600 con 0x7f5d081629c0 2024-04-28T13:35:58.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.712+0000 7f5d0ca00640 1 -- 172.21.15.196:0/586530802 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d000319d0 con 0x7f5d081629c0 2024-04-28T13:35:58.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.712+0000 7f5d0f200640 1 -- 172.21.15.196:0/586530802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d081629c0 msgr2=0x7f5d080d3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.712+0000 7f5d0f200640 1 --2- 172.21.15.196:0/586530802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d081629c0 0x7f5d080d3710 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009e50 tx=0x7f5d0002a6c0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.713+0000 7f5d0f200640 1 -- 172.21.15.196:0/586530802 shutdown_connections 2024-04-28T13:35:58.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.713+0000 7f5d0f200640 1 --2- 172.21.15.196:0/586530802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d08163bf0 0x7f5d08163030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.713+0000 7f5d0f200640 1 --2- 172.21.15.196:0/586530802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d081629c0 0x7f5d080d3710 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.713+0000 7f5d0f200640 1 -- 172.21.15.196:0/586530802 >> 172.21.15.196:0/586530802 conn(0x7f5d0809e570 msgr2=0x7f5d080d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:58.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.713+0000 7f5d0f200640 1 -- 172.21.15.196:0/586530802 shutdown_connections 2024-04-28T13:35:58.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.713+0000 7f5d0f200640 1 -- 172.21.15.196:0/586530802 wait complete. 2024-04-28T13:35:58.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.714+0000 7f5d0f200640 1 Processor -- start 2024-04-28T13:35:58.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.715+0000 7f5d0f200640 1 -- start start 2024-04-28T13:35:58.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.715+0000 7f5d0f200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d081629c0 0x7f5d080b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.715+0000 7f5d0f200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d08163570 0x7f5d0815c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.715+0000 7f5d0f200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d08163bf0 0x7f5d080ba540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.715+0000 7f5d0f200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d080d3c50 con 0x7f5d08163570 2024-04-28T13:35:58.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.715+0000 7f5d0f200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5d080b1e60 con 0x7f5d081629c0 2024-04-28T13:35:58.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.715+0000 7f5d0f200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5d080d3dd0 con 0x7f5d08163bf0 2024-04-28T13:35:58.716 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.716+0000 7f5d0d400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d08163570 0x7f5d0815c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.716+0000 7f5d0d400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d08163570 0x7f5d0815c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39158/0 (socket says 172.21.15.196:39158) 2024-04-28T13:35:58.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.716+0000 7f5d0d400640 1 -- 172.21.15.196:0/3912872708 learned_addr learned my addr 172.21.15.196:0/3912872708 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:58.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.716+0000 7f5d0d400640 1 -- 172.21.15.196:0/3912872708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d08163bf0 msgr2=0x7f5d080ba540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.716+0000 7f5d0de00640 1 --2- 172.21.15.196:0/3912872708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d081629c0 0x7f5d080b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.716+0000 7f5d0d400640 1 --2- 172.21.15.196:0/3912872708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d08163bf0 0x7f5d080ba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.716+0000 7f5d0d400640 1 -- 172.21.15.196:0/3912872708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d081629c0 msgr2=0x7f5d080b7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.716+0000 7f5d0d400640 1 --2- 172.21.15.196:0/3912872708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d081629c0 0x7f5d080b7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.717 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.716+0000 7f5d0d400640 1 -- 172.21.15.196:0/3912872708 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d080b87f0 con 0x7f5d08163570 2024-04-28T13:35:58.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.717+0000 7f5d0d400640 1 --2- 172.21.15.196:0/3912872708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d08163570 0x7f5d0815c8d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800d7d0 tx=0x7f5cf800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:58.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.717+0000 7f5cfe000640 1 -- 172.21.15.196:0/3912872708 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf800b6c0 con 0x7f5d08163570 2024-04-28T13:35:58.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.717+0000 7f5cfe000640 1 -- 172.21.15.196:0/3912872708 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf8019070 con 0x7f5d08163570 2024-04-28T13:35:58.718 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.717+0000 7f5cfe000640 1 -- 172.21.15.196:0/3912872708 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf800ba80 con 0x7f5d08163570 2024-04-28T13:35:58.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.717+0000 7f5d0f200640 1 -- 172.21.15.196:0/3912872708 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d080b8ab0 con 0x7f5d08163570 2024-04-28T13:35:58.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.717+0000 7f5d0f200640 1 -- 172.21.15.196:0/3912872708 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d080d0b50 con 0x7f5d08163570 2024-04-28T13:35:58.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.718+0000 7f5d0f200640 1 -- 172.21.15.196:0/3912872708 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d0806a290 con 0x7f5d08163570 2024-04-28T13:35:58.719 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.719+0000 7f5cfe000640 1 -- 172.21.15.196:0/3912872708 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f5cf800bc20 con 0x7f5d08163570 2024-04-28T13:35:58.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.720+0000 7f5cfe000640 1 --2- 172.21.15.196:0/3912872708 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5cec06c980 0x7f5cec06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:58.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.720+0000 7f5cfe000640 1 -- 172.21.15.196:0/3912872708 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f5cf808c190 con 0x7f5d08163570 2024-04-28T13:35:58.720 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.721+0000 7f5d0de00640 1 --2- 172.21.15.196:0/3912872708 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5cec06c980 0x7f5cec06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:58.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.722+0000 7f5d0de00640 1 --2- 172.21.15.196:0/3912872708 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5cec06c980 0x7f5cec06ee40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5d00033000 tx=0x7f5d00009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:58.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.724+0000 7f5cfe000640 1 -- 172.21.15.196:0/3912872708 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5cf801c360 con 0x7f5d08163570 2024-04-28T13:35:58.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.927+0000 7f5d0f200640 1 -- 172.21.15.196:0/3912872708 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f5d080d09a0 con 0x7f5cec06c980 2024-04-28T13:35:58.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.928+0000 7f5cfe000640 1 -- 172.21.15.196:0/3912872708 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5d080d09a0 con 0x7f5cec06c980 2024-04-28T13:35:58.928 INFO:teuthology.orchestra.run.smithi196.stderr:instructed osd(s) 6 to deep-scrub 2024-04-28T13:35:58.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.931+0000 7f5d0f200640 1 -- 172.21.15.196:0/3912872708 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5cec06c980 msgr2=0x7f5cec06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.931+0000 7f5d0f200640 1 --2- 172.21.15.196:0/3912872708 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5cec06c980 0x7f5cec06ee40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5d00033000 tx=0x7f5d00009000 comp rx=0 tx=0).stop 2024-04-28T13:35:58.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.931+0000 7f5d0f200640 1 -- 172.21.15.196:0/3912872708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d08163570 msgr2=0x7f5d0815c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:58.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.931+0000 7f5d0f200640 1 --2- 172.21.15.196:0/3912872708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d08163570 0x7f5d0815c8d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800d7d0 tx=0x7f5cf800dc90 comp rx=0 tx=0).stop 2024-04-28T13:35:58.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.931+0000 7f5d0f200640 1 -- 172.21.15.196:0/3912872708 shutdown_connections 2024-04-28T13:35:58.931 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.931+0000 7f5d0f200640 1 --2- 172.21.15.196:0/3912872708 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5cec06c980 0x7f5cec06ee40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.931+0000 7f5d0f200640 1 --2- 172.21.15.196:0/3912872708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d08163bf0 0x7f5d080ba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.931+0000 7f5d0f200640 1 --2- 172.21.15.196:0/3912872708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d08163570 0x7f5d0815c8d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.931+0000 7f5d0f200640 1 --2- 172.21.15.196:0/3912872708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d081629c0 0x7f5d080b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:58.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.931+0000 7f5d0f200640 1 -- 172.21.15.196:0/3912872708 >> 172.21.15.196:0/3912872708 conn(0x7f5d0809e570 msgr2=0x7f5d0806d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:58.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.931+0000 7f5d0f200640 1 -- 172.21.15.196:0/3912872708 shutdown_connections 2024-04-28T13:35:58.932 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:58.931+0000 7f5d0f200640 1 -- 172.21.15.196:0/3912872708 wait complete. 2024-04-28T13:35:58.950 INFO:tasks.ceph:Scrubbing osd.7 2024-04-28T13:35:58.951 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-28T13:35:59.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.155+0000 7f5ea3e00640 1 Processor -- start 2024-04-28T13:35:59.155 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.156+0000 7f5ea3e00640 1 -- start start 2024-04-28T13:35:59.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.157+0000 7f5ea3e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea40b50c0 0x7f5ea40d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.157+0000 7f5ea3e00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ea40d3900 con 0x7f5ea40cfb40 2024-04-28T13:35:59.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.157+0000 7f5ea3e00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ea40d3be0 con 0x7f5ea41629c0 2024-04-28T13:35:59.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.157+0000 7f5ea3e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea40b5580 con 0x7f5ea40b50c0 2024-04-28T13:35:59.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.157+0000 7f5ea3400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f5ea40cfb40 0x7f5ea40b5aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:53218/0 (socket says 172.21.15.196:53218) 2024-04-28T13:35:59.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.157+0000 7f5ea3400640 1 -- 172.21.15.196:0/895569518 learned_addr learned my addr 172.21.15.196:0/895569518 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:59.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.158+0000 7f5ea2000640 1 --2- 172.21.15.196:0/895569518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea40b50c0 0x7f5ea40d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.158+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1925392611 0 0) 0x7f5ea40d3900 con 0x7f5ea40cfb40 2024-04-28T13:35:59.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.158+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e88003520 con 0x7f5ea40cfb40 2024-04-28T13:35:59.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.158+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 35624013 0 0) 0x7f5ea40d3be0 con 0x7f5ea41629c0 2024-04-28T13:35:59.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.158+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ea40d3900 con 0x7f5ea41629c0 2024-04-28T13:35:59.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.158+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 <== mon.0 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2678103114 0 0) 0x7f5e88003520 con 0x7f5ea40cfb40 2024-04-28T13:35:59.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.158+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea40b50c0 msgr2=0x7f5ea40d5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:59.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.158+0000 7f5ea1600640 1 --2- 172.21.15.196:0/895569518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea40b50c0 0x7f5ea40d5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.159+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 >> v1:172.21.15.203:6789/0 conn(0x7f5ea41629c0 legacy=0x7f5ea40b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.159+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea40b4140 con 0x7f5ea40cfb40 2024-04-28T13:35:59.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.159+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 <== mon.0 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2746993265 0 0) 0x7f5e9c001670 con 0x7f5ea40cfb40 2024-04-28T13:35:59.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.159+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 >> v1:172.21.15.196:6789/0 conn(0x7f5ea40cfb40 legacy=0x7f5ea40b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.159+0000 7f5ea1600640 1 --2- 172.21.15.196:0/895569518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e88003b70 0x7f5e88003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.159+0000 7f5ea1600640 1 --2- 172.21.15.196:0/895569518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea40b50c0 0x7f5e88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.159+0000 7f5ea1600640 1 --2- 172.21.15.196:0/895569518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e88004d40 0x7f5e880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.159+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea40b5580 con 0x7f5e88003b70 2024-04-28T13:35:59.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.159+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8800ba70 con 0x7f5e88004d40 2024-04-28T13:35:59.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.159+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5e8800bc20 con 0x7f5ea40b50c0 2024-04-28T13:35:59.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.159+0000 7f5ea2a00640 1 --2- 172.21.15.196:0/895569518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e88003b70 0x7f5e88003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.160+0000 7f5ea2a00640 1 -- 172.21.15.196:0/895569518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea40b50c0 msgr2=0x7f5e88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:59.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.160+0000 7f5ea2a00640 1 --2- 172.21.15.196:0/895569518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea40b50c0 0x7f5e88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.160+0000 7f5ea2a00640 1 -- 172.21.15.196:0/895569518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e88004d40 msgr2=0x7f5e880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:59.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.160+0000 7f5ea2a00640 1 --2- 172.21.15.196:0/895569518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e88004d40 0x7f5e880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.160+0000 7f5ea2a00640 1 -- 172.21.15.196:0/895569518 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e88003520 con 0x7f5e88003b70 2024-04-28T13:35:59.163 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.161+0000 7f5ea2a00640 1 --2- 172.21.15.196:0/895569518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e88003b70 0x7f5e88003f70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c0018d0 tx=0x7f5e88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b2b529e45e9b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:59.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.161+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c0014f0 con 0x7f5e88003b70 2024-04-28T13:35:59.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.161+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e8c0052d0 con 0x7f5e88003b70 2024-04-28T13:35:59.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.161+0000 7f5ea1600640 1 -- 172.21.15.196:0/895569518 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c02ac90 con 0x7f5e88003b70 2024-04-28T13:35:59.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.161+0000 7f5ea3e00640 1 -- 172.21.15.196:0/895569518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e88003b70 msgr2=0x7f5e88003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.161+0000 7f5ea3e00640 1 --2- 172.21.15.196:0/895569518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e88003b70 0x7f5e88003f70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c0018d0 tx=0x7f5e88005830 comp rx=0 tx=0).stop 2024-04-28T13:35:59.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.161+0000 7f5ea3400640 1 -- 172.21.15.196:0/895569518 reap_dead start 2024-04-28T13:35:59.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.162+0000 7f5ea3e00640 1 -- 172.21.15.196:0/895569518 shutdown_connections 2024-04-28T13:35:59.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.162+0000 7f5ea3e00640 1 -- 172.21.15.196:0/895569518 >> 172.21.15.196:0/895569518 conn(0x7f5ea409e570 msgr2=0x7f5ea40d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:59.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.162+0000 7f5ea3e00640 1 -- 172.21.15.196:0/895569518 shutdown_connections 2024-04-28T13:35:59.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.162+0000 7f5ea3e00640 1 -- 172.21.15.196:0/895569518 wait complete. 2024-04-28T13:35:59.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.163+0000 7f5ea3e00640 1 Processor -- start 2024-04-28T13:35:59.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.164+0000 7f5ea3e00640 1 -- start start 2024-04-28T13:35:59.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea3e00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e88003b70 0x7f5ea415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea3e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ea40c9770 0x7f5ea406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ea40c9bf0 0x7f5ea406dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea3e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea40d4040 con 0x7f5ea40c9770 2024-04-28T13:35:59.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea3e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea40d3ec0 con 0x7f5ea40c9bf0 2024-04-28T13:35:59.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea3e00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea40b3880 con 0x7f5e88003b70 2024-04-28T13:35:59.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea2000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ea40c9770 0x7f5ea406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea2000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ea40c9770 0x7f5ea406d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39166/0 (socket says 172.21.15.196:39166) 2024-04-28T13:35:59.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea2000640 1 -- 172.21.15.196:0/1423370804 learned_addr learned my addr 172.21.15.196:0/1423370804 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:59.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea2a00640 1 --2- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e88003b70 0x7f5ea415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea3400640 1 --2- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ea40c9bf0 0x7f5ea406dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea2000640 1 -- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e88003b70 msgr2=0x7f5ea415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea2000640 1 --2- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e88003b70 0x7f5ea415c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea2000640 1 -- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ea40c9bf0 msgr2=0x7f5ea406dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea2000640 1 --2- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ea40c9bf0 0x7f5ea406dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.168 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.165+0000 7f5ea2000640 1 -- 172.21.15.196:0/1423370804 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea40ca1c0 con 0x7f5ea40c9770 2024-04-28T13:35:59.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.166+0000 7f5ea2000640 1 --2- 172.21.15.196:0/1423370804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ea40c9770 0x7f5ea406d6b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400ef10 tx=0x7f5e9400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:59.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.166+0000 7f5e9aa00640 1 -- 172.21.15.196:0/1423370804 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94018020 con 0x7f5ea40c9770 2024-04-28T13:35:59.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.166+0000 7f5e9aa00640 1 -- 172.21.15.196:0/1423370804 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e94014400 con 0x7f5ea40c9770 2024-04-28T13:35:59.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.166+0000 7f5ea3e00640 1 -- 172.21.15.196:0/1423370804 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea40ca4a0 con 0x7f5ea40c9770 2024-04-28T13:35:59.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.166+0000 7f5e9aa00640 1 -- 172.21.15.196:0/1423370804 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e940147d0 con 0x7f5ea40c9770 2024-04-28T13:35:59.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.166+0000 7f5ea3e00640 1 -- 172.21.15.196:0/1423370804 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea40caa60 con 0x7f5ea40c9770 2024-04-28T13:35:59.169 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.167+0000 7f5ea3e00640 1 -- 172.21.15.196:0/1423370804 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ea4050730 con 0x7f5ea40c9770 2024-04-28T13:35:59.170 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.168+0000 7f5e9aa00640 1 -- 172.21.15.196:0/1423370804 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f5e94005d10 con 0x7f5ea40c9770 2024-04-28T13:35:59.170 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.169+0000 7f5e9aa00640 1 --2- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e8006c9b0 0x7f5e8006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.171 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.169+0000 7f5e9aa00640 1 -- 172.21.15.196:0/1423370804 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f5e9408c620 con 0x7f5ea40c9770 2024-04-28T13:35:59.171 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.170+0000 7f5e9aa00640 1 --2- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:6802/2023229662,v1:172.21.15.203:6805/2023229662] conn(0x7f5e80074c10 0x7f5e80077050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.171 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.170+0000 7f5e9aa00640 1 -- 172.21.15.196:0/1423370804 --> [v2:172.21.15.203:6802/2023229662,v1:172.21.15.203:6805/2023229662] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e80077720 con 0x7f5e80074c10 2024-04-28T13:35:59.171 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.170+0000 7f5e9aa00640 1 -- 172.21.15.196:0/1423370804 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5e94093050 con 0x7f5ea40c9770 2024-04-28T13:35:59.171 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.170+0000 7f5ea2a00640 1 --2- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e8006c9b0 0x7f5e8006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.171 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.170+0000 7f5ea3400640 1 --2- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:6802/2023229662,v1:172.21.15.203:6805/2023229662] conn(0x7f5e80074c10 0x7f5e80077050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.171+0000 7f5ea2a00640 1 --2- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e8006c9b0 0x7f5e8006ee70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c032000 tx=0x7f5e8c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:59.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.172+0000 7f5ea3400640 1 --2- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:6802/2023229662,v1:172.21.15.203:6805/2023229662] conn(0x7f5e80074c10 0x7f5e80077050 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:59.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.174+0000 7f5e9aa00640 1 -- 172.21.15.196:0/1423370804 <== osd.7 v2:172.21.15.203:6802/2023229662 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e80077720 con 0x7f5e80074c10 2024-04-28T13:35:59.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.188+0000 7f5ea3e00640 1 -- 172.21.15.196:0/1423370804 --> [v2:172.21.15.203:6802/2023229662,v1:172.21.15.203:6805/2023229662] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5ea4160fb0 con 0x7f5e80074c10 2024-04-28T13:35:59.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.189+0000 7f5e9aa00640 1 -- 172.21.15.196:0/1423370804 <== osd.7 v2:172.21.15.203:6802/2023229662 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5ea4160fb0 con 0x7f5e80074c10 2024-04-28T13:35:59.189 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-28T13:35:59.189 INFO:teuthology.orchestra.run.smithi196.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T13:35:59.189 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-28T13:35:59.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.190+0000 7f5ea3e00640 1 -- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:6802/2023229662,v1:172.21.15.203:6805/2023229662] conn(0x7f5e80074c10 msgr2=0x7f5e80077050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.190+0000 7f5ea3e00640 1 --2- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:6802/2023229662,v1:172.21.15.203:6805/2023229662] conn(0x7f5e80074c10 0x7f5e80077050 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.190+0000 7f5ea3e00640 1 -- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e8006c9b0 msgr2=0x7f5e8006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.190+0000 7f5ea3e00640 1 --2- 172.21.15.196:0/1423370804 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f5e8006c9b0 0x7f5e8006ee70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c032000 tx=0x7f5e8c009030 comp rx=0 tx=0).stop 2024-04-28T13:35:59.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.190+0000 7f5ea3e00640 1 -- 172.21.15.196:0/1423370804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ea40c9770 msgr2=0x7f5ea406d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.190+0000 7f5ea3e00640 1 --2- 172.21.15.196:0/1423370804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ea40c9770 0x7f5ea406d6b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400ef10 tx=0x7f5e9400b480 comp rx=0 tx=0).stop 2024-04-28T13:35:59.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.190+0000 7f5ea3400640 1 -- 172.21.15.196:0/1423370804 reap_dead start 2024-04-28T13:35:59.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.190+0000 7f5ea3e00640 1 -- 172.21.15.196:0/1423370804 shutdown_connections 2024-04-28T13:35:59.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.190+0000 7f5ea3e00640 1 -- 172.21.15.196:0/1423370804 >> 172.21.15.196:0/1423370804 conn(0x7f5ea409e570 msgr2=0x7f5ea40d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:59.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.190+0000 7f5ea3e00640 1 -- 172.21.15.196:0/1423370804 shutdown_connections 2024-04-28T13:35:59.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.190+0000 7f5ea3e00640 1 -- 172.21.15.196:0/1423370804 wait complete. 2024-04-28T13:35:59.208 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-28T13:35:59.400 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.400+0000 7feb33200640 1 Processor -- start 2024-04-28T13:35:59.400 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.400+0000 7feb33200640 1 -- start start 2024-04-28T13:35:59.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.402+0000 7feb33200640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7feb2c0b9060 0x7feb2c0b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.402+0000 7feb33200640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb2c0d0220 con 0x7feb2c0cfda0 2024-04-28T13:35:59.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.402+0000 7feb33200640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb2c0d0500 con 0x7feb2c0d1b80 2024-04-28T13:35:59.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.402+0000 7feb33200640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7feb2c0d07e0 con 0x7feb2c0b9060 2024-04-28T13:35:59.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.402+0000 7feb2b400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7feb2c0cfda0 0x7feb2c0b60a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:53230/0 (socket says 172.21.15.196:53230) 2024-04-28T13:35:59.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.402+0000 7feb2b400640 1 -- 172.21.15.196:0/151724187 learned_addr learned my addr 172.21.15.196:0/151724187 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:59.402 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.402+0000 7feb31200640 1 --2- 172.21.15.196:0/151724187 >> v2:172.21.15.196:3300/0 conn(0x7feb2c0b9060 0x7feb2c0b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.403+0000 7feb2aa00640 1 -- 172.21.15.196:0/151724187 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2645805293 0 0) 0x7feb2c0d0220 con 0x7feb2c0cfda0 2024-04-28T13:35:59.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.403+0000 7feb31200640 1 -- 172.21.15.196:0/151724187 >> v1:172.21.15.203:6789/0 conn(0x7feb2c0d1b80 legacy=0x7feb2c0d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:59.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.403+0000 7feb31200640 1 -- 172.21.15.196:0/151724187 >> v1:172.21.15.196:6789/0 conn(0x7feb2c0cfda0 legacy=0x7feb2c0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.403+0000 7feb31200640 1 -- 172.21.15.196:0/151724187 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c0d0ac0 con 0x7feb2c0b9060 2024-04-28T13:35:59.404 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.404+0000 7feb31200640 1 --2- 172.21.15.196:0/151724187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb2c0b9060 0x7feb2c0b67b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feb24009d90 tx=0x7feb2402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b600aee8f5b6923b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:59.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.404+0000 7feb2aa00640 1 -- 172.21.15.196:0/151724187 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24007740 con 0x7feb2c0b9060 2024-04-28T13:35:59.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.404+0000 7feb2aa00640 1 -- 172.21.15.196:0/151724187 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb240078e0 con 0x7feb2c0b9060 2024-04-28T13:35:59.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.404+0000 7feb2aa00640 1 -- 172.21.15.196:0/151724187 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24007cb0 con 0x7feb2c0b9060 2024-04-28T13:35:59.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.405+0000 7feb33200640 1 -- 172.21.15.196:0/151724187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb2c0b9060 msgr2=0x7feb2c0b67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.405+0000 7feb33200640 1 --2- 172.21.15.196:0/151724187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb2c0b9060 0x7feb2c0b67b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feb24009d90 tx=0x7feb2402a6a0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.405+0000 7feb33200640 1 -- 172.21.15.196:0/151724187 shutdown_connections 2024-04-28T13:35:59.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.405+0000 7feb33200640 1 --2- 172.21.15.196:0/151724187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb2c0b9060 0x7feb2c0b67b0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.405 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.405+0000 7feb33200640 1 -- 172.21.15.196:0/151724187 >> 172.21.15.196:0/151724187 conn(0x7feb2c09e540 msgr2=0x7feb2c0d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:59.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.405+0000 7feb33200640 1 -- 172.21.15.196:0/151724187 shutdown_connections 2024-04-28T13:35:59.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.405+0000 7feb33200640 1 -- 172.21.15.196:0/151724187 wait complete. 2024-04-28T13:35:59.406 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.407+0000 7feb33200640 1 Processor -- start 2024-04-28T13:35:59.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.407+0000 7feb33200640 1 -- start start 2024-04-28T13:35:59.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb33200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb2c0b9060 0x7feb2c1f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb33200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb2c0cfda0 0x7feb2c158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb33200640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb2c0d1b80 0x7feb2c0a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb33200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c0d1f60 con 0x7feb2c0cfda0 2024-04-28T13:35:59.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb33200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c0d4170 con 0x7feb2c0b9060 2024-04-28T13:35:59.412 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb33200640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feb2c0d20e0 con 0x7feb2c0d1b80 2024-04-28T13:35:59.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb2b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb2c0cfda0 0x7feb2c158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb2b400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb2c0cfda0 0x7feb2c158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39180/0 (socket says 172.21.15.196:39180) 2024-04-28T13:35:59.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb2b400640 1 -- 172.21.15.196:0/3488458268 learned_addr learned my addr 172.21.15.196:0/3488458268 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:59.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb2be00640 1 --2- 172.21.15.196:0/3488458268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb2c0b9060 0x7feb2c1f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.413 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb31200640 1 --2- 172.21.15.196:0/3488458268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb2c0d1b80 0x7feb2c0a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb2b400640 1 -- 172.21.15.196:0/3488458268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb2c0d1b80 msgr2=0x7feb2c0a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb2b400640 1 --2- 172.21.15.196:0/3488458268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb2c0d1b80 0x7feb2c0a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb2b400640 1 -- 172.21.15.196:0/3488458268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb2c0b9060 msgr2=0x7feb2c1f6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb2b400640 1 --2- 172.21.15.196:0/3488458268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb2c0b9060 0x7feb2c1f6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.408+0000 7feb2b400640 1 -- 172.21.15.196:0/3488458268 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c158c60 con 0x7feb2c0cfda0 2024-04-28T13:35:59.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.409+0000 7feb31200640 1 --2- 172.21.15.196:0/3488458268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb2c0d1b80 0x7feb2c0a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:35:59.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.409+0000 7feb2b400640 1 --2- 172.21.15.196:0/3488458268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb2c0cfda0 0x7feb2c158720 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feb1400c4d0 tx=0x7feb1400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:59.414 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.409+0000 7feb23e00640 1 -- 172.21.15.196:0/3488458268 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14017070 con 0x7feb2c0cfda0 2024-04-28T13:35:59.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.409+0000 7feb23e00640 1 -- 172.21.15.196:0/3488458268 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb14005d00 con 0x7feb2c0cfda0 2024-04-28T13:35:59.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.409+0000 7feb23e00640 1 -- 172.21.15.196:0/3488458268 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb140072b0 con 0x7feb2c0cfda0 2024-04-28T13:35:59.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.409+0000 7feb33200640 1 -- 172.21.15.196:0/3488458268 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2c0b2a50 con 0x7feb2c0cfda0 2024-04-28T13:35:59.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.409+0000 7feb33200640 1 -- 172.21.15.196:0/3488458268 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2c0b32b0 con 0x7feb2c0cfda0 2024-04-28T13:35:59.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.411+0000 7feb33200640 1 -- 172.21.15.196:0/3488458268 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb2c06a290 con 0x7feb2c0cfda0 2024-04-28T13:35:59.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.411+0000 7feb23e00640 1 -- 172.21.15.196:0/3488458268 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7feb1401a070 con 0x7feb2c0cfda0 2024-04-28T13:35:59.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.412+0000 7feb23e00640 1 --2- 172.21.15.196:0/3488458268 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feb0806c980 0x7feb0806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.413+0000 7feb23e00640 1 -- 172.21.15.196:0/3488458268 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7feb1408aff0 con 0x7feb2c0cfda0 2024-04-28T13:35:59.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.413+0000 7feb2be00640 1 --2- 172.21.15.196:0/3488458268 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feb0806c980 0x7feb0806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.414+0000 7feb2be00640 1 --2- 172.21.15.196:0/3488458268 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feb0806c980 0x7feb0806ee40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7feb1c006fd0 tx=0x7feb1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:59.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.417+0000 7feb23e00640 1 -- 172.21.15.196:0/3488458268 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb14059310 con 0x7feb2c0cfda0 2024-04-28T13:35:59.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.617+0000 7feb33200640 1 -- 172.21.15.196:0/3488458268 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7feb2c0d27b0 con 0x7feb0806c980 2024-04-28T13:35:59.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.618+0000 7feb23e00640 1 -- 172.21.15.196:0/3488458268 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7feb2c0d27b0 con 0x7feb0806c980 2024-04-28T13:35:59.618 INFO:teuthology.orchestra.run.smithi196.stderr:instructed osd(s) 7 to deep-scrub 2024-04-28T13:35:59.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.621+0000 7feb33200640 1 -- 172.21.15.196:0/3488458268 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feb0806c980 msgr2=0x7feb0806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.621+0000 7feb33200640 1 --2- 172.21.15.196:0/3488458268 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feb0806c980 0x7feb0806ee40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7feb1c006fd0 tx=0x7feb1c008040 comp rx=0 tx=0).stop 2024-04-28T13:35:59.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.621+0000 7feb33200640 1 -- 172.21.15.196:0/3488458268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb2c0cfda0 msgr2=0x7feb2c158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.621+0000 7feb33200640 1 --2- 172.21.15.196:0/3488458268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb2c0cfda0 0x7feb2c158720 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feb1400c4d0 tx=0x7feb1400c990 comp rx=0 tx=0).stop 2024-04-28T13:35:59.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.621+0000 7feb33200640 1 -- 172.21.15.196:0/3488458268 shutdown_connections 2024-04-28T13:35:59.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.621+0000 7feb33200640 1 --2- 172.21.15.196:0/3488458268 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7feb0806c980 0x7feb0806ee40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.621+0000 7feb33200640 1 --2- 172.21.15.196:0/3488458268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb2c0d1b80 0x7feb2c0a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.621+0000 7feb33200640 1 --2- 172.21.15.196:0/3488458268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb2c0cfda0 0x7feb2c158720 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.621+0000 7feb33200640 1 --2- 172.21.15.196:0/3488458268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb2c0b9060 0x7feb2c1f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.621+0000 7feb33200640 1 -- 172.21.15.196:0/3488458268 >> 172.21.15.196:0/3488458268 conn(0x7feb2c09e540 msgr2=0x7feb2c0d58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:59.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.621+0000 7feb33200640 1 -- 172.21.15.196:0/3488458268 shutdown_connections 2024-04-28T13:35:59.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.621+0000 7feb33200640 1 -- 172.21.15.196:0/3488458268 wait complete. 2024-04-28T13:35:59.639 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T13:35:59.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.831+0000 7f37a6a00640 1 Processor -- start 2024-04-28T13:35:59.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.832+0000 7f37a6a00640 1 -- start start 2024-04-28T13:35:59.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.833+0000 7f37a6a00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f37a0072020 0x7f37a00724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.833+0000 7f37a6a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f37a00b3de0 0x7f37a00b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.833+0000 7f37a6a00640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37a00ceef0 con 0x7f37a014c600 2024-04-28T13:35:59.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.833+0000 7f37a6a00640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f37a014c9e0 con 0x7f37a0072020 2024-04-28T13:35:59.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.833+0000 7f37a6a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f37a00cf1d0 con 0x7f37a00b3de0 2024-04-28T13:35:59.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.833+0000 7f3797400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f37a0072020 0x7f37a00724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.833+0000 7f379fe00640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f37a014c600 0x7f37a00d0f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:53238/0 (socket says 172.21.15.196:53238) 2024-04-28T13:35:59.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.833+0000 7f379fe00640 1 -- 172.21.15.196:0/2060284820 learned_addr learned my addr 172.21.15.196:0/2060284820 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:35:59.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.834+0000 7f37a4a00640 1 --2- 172.21.15.196:0/2060284820 >> v2:172.21.15.203:3300/0 conn(0x7f37a00b3de0 0x7f37a00b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.834+0000 7f3797400640 1 -- 172.21.15.196:0/2060284820 >> v2:172.21.15.203:3300/0 conn(0x7f37a00b3de0 msgr2=0x7f37a00b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:35:59.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.834+0000 7f3797400640 1 --2- 172.21.15.196:0/2060284820 >> v2:172.21.15.203:3300/0 conn(0x7f37a00b3de0 0x7f37a00b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.834+0000 7f3797400640 1 -- 172.21.15.196:0/2060284820 >> v1:172.21.15.196:6789/0 conn(0x7f37a014c600 legacy=0x7f37a00d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.834+0000 7f3797400640 1 -- 172.21.15.196:0/2060284820 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a00cf4b0 con 0x7f37a0072020 2024-04-28T13:35:59.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.834+0000 7f379f400640 1 -- 172.21.15.196:0/2060284820 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2958061645 0 0) 0x7f37a00ceef0 con 0x7f37a014c600 2024-04-28T13:35:59.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.834+0000 7f37a4a00640 1 --2- 172.21.15.196:0/2060284820 >> v2:172.21.15.203:3300/0 conn(0x7f37a00b3de0 0x7f37a00b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:35:59.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.835+0000 7f3797400640 1 --2- 172.21.15.196:0/2060284820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37a0072020 0x7f37a00724b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f37a00d0440 tx=0x7f378802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f589783452436aac server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:59.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.835+0000 7f379f400640 1 -- 172.21.15.196:0/2060284820 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788037070 con 0x7f37a0072020 2024-04-28T13:35:59.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.835+0000 7f379f400640 1 -- 172.21.15.196:0/2060284820 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3788032440 con 0x7f37a0072020 2024-04-28T13:35:59.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.835+0000 7f379f400640 1 -- 172.21.15.196:0/2060284820 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788032810 con 0x7f37a0072020 2024-04-28T13:35:59.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.836+0000 7f37a6a00640 1 -- 172.21.15.196:0/2060284820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37a0072020 msgr2=0x7f37a00724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.836+0000 7f37a6a00640 1 --2- 172.21.15.196:0/2060284820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37a0072020 0x7f37a00724b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f37a00d0440 tx=0x7f378802a1a0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.836+0000 7f37a6a00640 1 -- 172.21.15.196:0/2060284820 shutdown_connections 2024-04-28T13:35:59.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.836+0000 7f37a6a00640 1 --2- 172.21.15.196:0/2060284820 >> v2:172.21.15.203:3300/0 conn(0x7f37a00b3de0 0x7f37a00b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.836+0000 7f37a6a00640 1 --2- 172.21.15.196:0/2060284820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37a0072020 0x7f37a00724b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.836+0000 7f37a6a00640 1 -- 172.21.15.196:0/2060284820 >> 172.21.15.196:0/2060284820 conn(0x7f37a00cd830 msgr2=0x7f37a00cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:35:59.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.836+0000 7f37a6a00640 1 -- 172.21.15.196:0/2060284820 shutdown_connections 2024-04-28T13:35:59.838 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.836+0000 7f37a6a00640 1 -- 172.21.15.196:0/2060284820 wait complete. 2024-04-28T13:35:59.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.838+0000 7f37a6a00640 1 Processor -- start 2024-04-28T13:35:59.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.838+0000 7f37a6a00640 1 -- start start 2024-04-28T13:35:59.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.839+0000 7f37a6a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37a0072020 0x7f37a01e88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.840 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.839+0000 7f37a6a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37a00b3de0 0x7f37a01fcb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.839+0000 7f37a6a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37a014c600 0x7f37a00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.839+0000 7f37a6a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37a00cff90 con 0x7f37a0072020 2024-04-28T13:35:59.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.839+0000 7f37a6a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f37a00cfa70 con 0x7f37a014c600 2024-04-28T13:35:59.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.839+0000 7f37a6a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f37a00d1630 con 0x7f37a00b3de0 2024-04-28T13:35:59.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.839+0000 7f379fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37a0072020 0x7f37a01e88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.839+0000 7f379fe00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37a0072020 0x7f37a01e88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39198/0 (socket says 172.21.15.196:39198) 2024-04-28T13:35:59.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.839+0000 7f379fe00640 1 -- 172.21.15.196:0/1542806789 learned_addr learned my addr 172.21.15.196:0/1542806789 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:35:59.841 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.840+0000 7f3797400640 1 --2- 172.21.15.196:0/1542806789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37a00b3de0 0x7f37a01fcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.840+0000 7f37a4a00640 1 --2- 172.21.15.196:0/1542806789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37a014c600 0x7f37a00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.840+0000 7f379fe00640 1 -- 172.21.15.196:0/1542806789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37a00b3de0 msgr2=0x7f37a01fcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.840+0000 7f379fe00640 1 --2- 172.21.15.196:0/1542806789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37a00b3de0 0x7f37a01fcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.840+0000 7f379fe00640 1 -- 172.21.15.196:0/1542806789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37a014c600 msgr2=0x7f37a00b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:35:59.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.840+0000 7f379fe00640 1 --2- 172.21.15.196:0/1542806789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37a014c600 0x7f37a00b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:35:59.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.840+0000 7f379fe00640 1 -- 172.21.15.196:0/1542806789 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a01fd2d0 con 0x7f37a0072020 2024-04-28T13:35:59.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.840+0000 7f379fe00640 1 --2- 172.21.15.196:0/1542806789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37a0072020 0x7f37a01e88f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f379000c430 tx=0x7f379000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:59.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.840+0000 7f379cc00640 1 -- 172.21.15.196:0/1542806789 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37900074e0 con 0x7f37a0072020 2024-04-28T13:35:59.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.840+0000 7f379cc00640 1 -- 172.21.15.196:0/1542806789 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37900047e0 con 0x7f37a0072020 2024-04-28T13:35:59.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.840+0000 7f379cc00640 1 -- 172.21.15.196:0/1542806789 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790004bd0 con 0x7f37a0072020 2024-04-28T13:35:59.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.840+0000 7f37a6a00640 1 -- 172.21.15.196:0/1542806789 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a01fd5b0 con 0x7f37a0072020 2024-04-28T13:35:59.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.841+0000 7f37a6a00640 1 -- 172.21.15.196:0/1542806789 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a00ae7f0 con 0x7f37a0072020 2024-04-28T13:35:59.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.842+0000 7f37a6a00640 1 -- 172.21.15.196:0/1542806789 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a00ceb50 con 0x7f37a0072020 2024-04-28T13:35:59.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.843+0000 7f379cc00640 1 -- 172.21.15.196:0/1542806789 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f3790011070 con 0x7f37a0072020 2024-04-28T13:35:59.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.844+0000 7f379cc00640 1 --2- 172.21.15.196:0/1542806789 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f377c06c980 0x7f377c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:35:59.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.844+0000 7f379cc00640 1 -- 172.21.15.196:0/1542806789 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f379008b8a0 con 0x7f37a0072020 2024-04-28T13:35:59.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.844+0000 7f3797400640 1 --2- 172.21.15.196:0/1542806789 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f377c06c980 0x7f377c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:35:59.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.846+0000 7f3797400640 1 --2- 172.21.15.196:0/1542806789 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f377c06c980 0x7f377c06ee40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3788034000 tx=0x7f378802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:35:59.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:35:59.848+0000 7f379cc00640 1 -- 172.21.15.196:0/1542806789 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f37a00ceb50 con 0x7f37a0072020 2024-04-28T13:36:00.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:00.043+0000 7f37a6a00640 1 -- 172.21.15.196:0/1542806789 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37a01fd070 con 0x7f377c06c980 2024-04-28T13:36:00.061 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:00.062+0000 7f379cc00640 1 -- 172.21.15.196:0/1542806789 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251290 (secure 0 0 0) 0x7f3788038040 con 0x7f377c06c980 2024-04-28T13:36:00.065 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:36:00.067 INFO:teuthology.orchestra.run.smithi196.stderr:dumped all 2024-04-28T13:36:00.068 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:00.068+0000 7f37a6a00640 1 -- 172.21.15.196:0/1542806789 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f377c06c980 msgr2=0x7f377c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:00.068 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:00.068+0000 7f37a6a00640 1 --2- 172.21.15.196:0/1542806789 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f377c06c980 0x7f377c06ee40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3788034000 tx=0x7f378802e000 comp rx=0 tx=0).stop 2024-04-28T13:36:00.069 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:00.068+0000 7f37a6a00640 1 -- 172.21.15.196:0/1542806789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37a0072020 msgr2=0x7f37a01e88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:00.069 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:00.068+0000 7f37a6a00640 1 --2- 172.21.15.196:0/1542806789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37a0072020 0x7f37a01e88f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f379000c430 tx=0x7f379000c8f0 comp rx=0 tx=0).stop 2024-04-28T13:36:00.069 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:00.068+0000 7f37a6a00640 1 -- 172.21.15.196:0/1542806789 shutdown_connections 2024-04-28T13:36:00.069 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:00.068+0000 7f37a6a00640 1 --2- 172.21.15.196:0/1542806789 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f377c06c980 0x7f377c06ee40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:00.069 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:00.068+0000 7f37a6a00640 1 --2- 172.21.15.196:0/1542806789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37a014c600 0x7f37a00b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:00.069 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:00.068+0000 7f37a6a00640 1 --2- 172.21.15.196:0/1542806789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37a00b3de0 0x7f37a01fcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:00.069 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:00.068+0000 7f37a6a00640 1 --2- 172.21.15.196:0/1542806789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37a0072020 0x7f37a01e88f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:00.069 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:00.068+0000 7f37a6a00640 1 -- 172.21.15.196:0/1542806789 >> 172.21.15.196:0/1542806789 conn(0x7f37a00cd830 msgr2=0x7f37a00d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:36:00.070 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:00.068+0000 7f37a6a00640 1 -- 172.21.15.196:0/1542806789 shutdown_connections 2024-04-28T13:36:00.070 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:00.068+0000 7f37a6a00640 1 -- 172.21.15.196:0/1542806789 wait complete. 2024-04-28T13:36:00.087 INFO:teuthology.orchestra.run.smithi196.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-04-28T13:35:58.286386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594563,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":214,"num_write_kb":2437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225868,"kb_used_data":5544,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748852,"statfs":{"total":773094113280,"available":772862824448,"internally_reserved":0,"allocated":5677056,"data_stored":3015678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":282,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005180"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350675+0000","last_change":"2024-04-28T13:33:33.999818+0000","last_active":"2024-04-28T13:34:04.350675+0000","last_peered":"2024-04-28T13:34:04.350675+0000","last_clean":"2024-04-28T13:34:04.350675+0000","last_became_active":"2024-04-28T13:33:33.999486+0000","last_became_peered":"2024-04-28T13:33:33.999486+0000","last_unstale":"2024-04-28T13:34:04.350675+0000","last_undegraded":"2024-04-28T13:34:04.350675+0000","last_fullsized":"2024-04-28T13:34:04.350675+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:32:47.409486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355721+0000","last_change":"2024-04-28T13:33:34.003919+0000","last_active":"2024-04-28T13:34:04.355721+0000","last_peered":"2024-04-28T13:34:04.355721+0000","last_clean":"2024-04-28T13:34:04.355721+0000","last_became_active":"2024-04-28T13:33:34.003551+0000","last_became_peered":"2024-04-28T13:33:34.003551+0000","last_unstale":"2024-04-28T13:34:04.355721+0000","last_undegraded":"2024-04-28T13:34:04.355721+0000","last_fullsized":"2024-04-28T13:34:04.355721+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347343+0000","last_change":"2024-04-28T13:33:33.996733+0000","last_active":"2024-04-28T13:34:04.347343+0000","last_peered":"2024-04-28T13:34:04.347343+0000","last_clean":"2024-04-28T13:34:04.347343+0000","last_became_active":"2024-04-28T13:33:33.996413+0000","last_became_peered":"2024-04-28T13:33:33.996413+0000","last_unstale":"2024-04-28T13:34:04.347343+0000","last_undegraded":"2024-04-28T13:34:04.347343+0000","last_fullsized":"2024-04-28T13:34:04.347343+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:46.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:35.267183+0000","last_change":"2024-04-28T13:33:34.004787+0000","last_active":"2024-04-28T13:35:35.267183+0000","last_peered":"2024-04-28T13:35:35.267183+0000","last_clean":"2024-04-28T13:35:35.267183+0000","last_became_active":"2024-04-28T13:33:33.999818+0000","last_became_peered":"2024-04-28T13:33:33.999818+0000","last_unstale":"2024-04-28T13:35:35.267183+0000","last_undegraded":"2024-04-28T13:35:35.267183+0000","last_fullsized":"2024-04-28T13:35:35.267183+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354517+0000","last_change":"2024-04-28T13:33:34.004715+0000","last_active":"2024-04-28T13:34:04.354517+0000","last_peered":"2024-04-28T13:34:04.354517+0000","last_clean":"2024-04-28T13:34:04.354517+0000","last_became_active":"2024-04-28T13:33:34.004436+0000","last_became_peered":"2024-04-28T13:33:34.004436+0000","last_unstale":"2024-04-28T13:34:04.354517+0000","last_undegraded":"2024-04-28T13:34:04.354517+0000","last_fullsized":"2024-04-28T13:34:04.354517+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:46.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344494+0000","last_change":"2024-04-28T13:33:34.012959+0000","last_active":"2024-04-28T13:34:04.344494+0000","last_peered":"2024-04-28T13:34:04.344494+0000","last_clean":"2024-04-28T13:34:04.344494+0000","last_became_active":"2024-04-28T13:33:34.010839+0000","last_became_peered":"2024-04-28T13:33:34.010839+0000","last_unstale":"2024-04-28T13:34:04.344494+0000","last_undegraded":"2024-04-28T13:34:04.344494+0000","last_fullsized":"2024-04-28T13:34:04.344494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:48:58.152999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349347+0000","last_change":"2024-04-28T13:33:34.003448+0000","last_active":"2024-04-28T13:34:04.349347+0000","last_peered":"2024-04-28T13:34:04.349347+0000","last_clean":"2024-04-28T13:34:04.349347+0000","last_became_active":"2024-04-28T13:33:34.002948+0000","last_became_peered":"2024-04-28T13:33:34.002948+0000","last_unstale":"2024-04-28T13:34:04.349347+0000","last_undegraded":"2024-04-28T13:34:04.349347+0000","last_fullsized":"2024-04-28T13:34:04.349347+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:09:39.634648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.145422+0000","last_change":"2024-04-28T13:34:05.143133+0000","last_active":"2024-04-28T13:34:05.145422+0000","last_peered":"2024-04-28T13:34:05.145422+0000","last_clean":"2024-04-28T13:34:05.145422+0000","last_became_active":"2024-04-28T13:34:05.142351+0000","last_became_peered":"2024-04-28T13:34:05.142351+0000","last_unstale":"2024-04-28T13:34:05.145422+0000","last_undegraded":"2024-04-28T13:34:05.145422+0000","last_fullsized":"2024-04-28T13:34:05.145422+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:53:12.865627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.881132+0000","last_change":"2024-04-28T13:33:34.013366+0000","last_active":"2024-04-28T13:34:04.881132+0000","last_peered":"2024-04-28T13:34:04.881132+0000","last_clean":"2024-04-28T13:34:04.881132+0000","last_became_active":"2024-04-28T13:33:34.012900+0000","last_became_peered":"2024-04-28T13:33:34.012900+0000","last_unstale":"2024-04-28T13:34:04.881132+0000","last_undegraded":"2024-04-28T13:34:04.881132+0000","last_fullsized":"2024-04-28T13:34:04.881132+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880436+0000","last_change":"2024-04-28T13:33:34.007098+0000","last_active":"2024-04-28T13:34:04.880436+0000","last_peered":"2024-04-28T13:34:04.880436+0000","last_clean":"2024-04-28T13:34:04.880436+0000","last_became_active":"2024-04-28T13:33:34.005157+0000","last_became_peered":"2024-04-28T13:33:34.005157+0000","last_unstale":"2024-04-28T13:34:04.880436+0000","last_undegraded":"2024-04-28T13:34:04.880436+0000","last_fullsized":"2024-04-28T13:34:04.880436+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.879822+0000","last_change":"2024-04-28T13:33:34.011118+0000","last_active":"2024-04-28T13:34:04.879822+0000","last_peered":"2024-04-28T13:34:04.879822+0000","last_clean":"2024-04-28T13:34:04.879822+0000","last_became_active":"2024-04-28T13:33:34.010811+0000","last_became_peered":"2024-04-28T13:33:34.010811+0000","last_unstale":"2024-04-28T13:34:04.879822+0000","last_undegraded":"2024-04-28T13:34:04.879822+0000","last_fullsized":"2024-04-28T13:34:04.879822+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355402+0000","last_change":"2024-04-28T13:33:34.011900+0000","last_active":"2024-04-28T13:34:04.355402+0000","last_peered":"2024-04-28T13:34:04.355402+0000","last_clean":"2024-04-28T13:34:04.355402+0000","last_became_active":"2024-04-28T13:33:34.011580+0000","last_became_peered":"2024-04-28T13:33:34.011580+0000","last_unstale":"2024-04-28T13:34:04.355402+0000","last_undegraded":"2024-04-28T13:34:04.355402+0000","last_fullsized":"2024-04-28T13:34:04.355402+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.532062+0000","last_change":"2024-04-28T13:33:34.000874+0000","last_active":"2024-04-28T13:34:05.532062+0000","last_peered":"2024-04-28T13:34:05.532062+0000","last_clean":"2024-04-28T13:34:05.532062+0000","last_became_active":"2024-04-28T13:33:34.000568+0000","last_became_peered":"2024-04-28T13:33:34.000568+0000","last_unstale":"2024-04-28T13:34:05.532062+0000","last_undegraded":"2024-04-28T13:34:05.532062+0000","last_fullsized":"2024-04-28T13:34:05.532062+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:56:40.722825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.502702+0000","last_change":"2024-04-28T13:34:04.496689+0000","last_active":"2024-04-28T13:34:04.502702+0000","last_peered":"2024-04-28T13:34:04.502702+0000","last_clean":"2024-04-28T13:34:04.502702+0000","last_became_active":"2024-04-28T13:34:04.495748+0000","last_became_peered":"2024-04-28T13:34:04.495748+0000","last_unstale":"2024-04-28T13:34:04.502702+0000","last_undegraded":"2024-04-28T13:34:04.502702+0000","last_fullsized":"2024-04-28T13:34:04.502702+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:11.732507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344332+0000","last_change":"2024-04-28T13:33:33.998387+0000","last_active":"2024-04-28T13:34:04.344332+0000","last_peered":"2024-04-28T13:34:04.344332+0000","last_clean":"2024-04-28T13:34:04.344332+0000","last_became_active":"2024-04-28T13:33:33.998087+0000","last_became_peered":"2024-04-28T13:33:33.998087+0000","last_unstale":"2024-04-28T13:34:04.344332+0000","last_undegraded":"2024-04-28T13:34:04.344332+0000","last_fullsized":"2024-04-28T13:34:04.344332+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:52:24.477401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139689+0000","last_change":"2024-04-28T13:33:33.989154+0000","last_active":"2024-04-28T13:34:05.139689+0000","last_peered":"2024-04-28T13:34:05.139689+0000","last_clean":"2024-04-28T13:34:05.139689+0000","last_became_active":"2024-04-28T13:33:33.988827+0000","last_became_peered":"2024-04-28T13:33:33.988827+0000","last_unstale":"2024-04-28T13:34:05.139689+0000","last_undegraded":"2024-04-28T13:34:05.139689+0000","last_fullsized":"2024-04-28T13:34:05.139689+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:52:46.527475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346912+0000","last_change":"2024-04-28T13:33:34.008467+0000","last_active":"2024-04-28T13:34:04.346912+0000","last_peered":"2024-04-28T13:34:04.346912+0000","last_clean":"2024-04-28T13:34:04.346912+0000","last_became_active":"2024-04-28T13:33:34.008181+0000","last_became_peered":"2024-04-28T13:33:34.008181+0000","last_unstale":"2024-04-28T13:34:04.346912+0000","last_undegraded":"2024-04-28T13:34:04.346912+0000","last_fullsized":"2024-04-28T13:34:04.346912+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347723+0000","last_change":"2024-04-28T13:33:34.007505+0000","last_active":"2024-04-28T13:34:04.347723+0000","last_peered":"2024-04-28T13:34:04.347723+0000","last_clean":"2024-04-28T13:34:04.347723+0000","last_became_active":"2024-04-28T13:33:34.006793+0000","last_became_peered":"2024-04-28T13:33:34.006793+0000","last_unstale":"2024-04-28T13:34:04.347723+0000","last_undegraded":"2024-04-28T13:34:04.347723+0000","last_fullsized":"2024-04-28T13:34:04.347723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:22:31.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.140956+0000","last_change":"2024-04-28T13:33:34.009974+0000","last_active":"2024-04-28T13:34:05.140956+0000","last_peered":"2024-04-28T13:34:05.140956+0000","last_clean":"2024-04-28T13:34:05.140956+0000","last_became_active":"2024-04-28T13:33:34.009657+0000","last_became_peered":"2024-04-28T13:33:34.009657+0000","last_unstale":"2024-04-28T13:34:05.140956+0000","last_undegraded":"2024-04-28T13:34:05.140956+0000","last_fullsized":"2024-04-28T13:34:05.140956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:18:56.809253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531828+0000","last_change":"2024-04-28T13:33:34.004233+0000","last_active":"2024-04-28T13:34:05.531828+0000","last_peered":"2024-04-28T13:34:05.531828+0000","last_clean":"2024-04-28T13:34:05.531828+0000","last_became_active":"2024-04-28T13:33:33.999567+0000","last_became_peered":"2024-04-28T13:33:33.999567+0000","last_unstale":"2024-04-28T13:34:05.531828+0000","last_undegraded":"2024-04-28T13:34:05.531828+0000","last_fullsized":"2024-04-28T13:34:05.531828+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:36.248393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344842+0000","last_change":"2024-04-28T13:33:34.008932+0000","last_active":"2024-04-28T13:34:04.344842+0000","last_peered":"2024-04-28T13:34:04.344842+0000","last_clean":"2024-04-28T13:34:04.344842+0000","last_became_active":"2024-04-28T13:33:34.008652+0000","last_became_peered":"2024-04-28T13:33:34.008652+0000","last_unstale":"2024-04-28T13:34:04.344842+0000","last_undegraded":"2024-04-28T13:34:04.344842+0000","last_fullsized":"2024-04-28T13:34:04.344842+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:43:22.000390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:50.269365+0000","last_change":"2024-04-28T13:33:34.006521+0000","last_active":"2024-04-28T13:35:50.269365+0000","last_peered":"2024-04-28T13:35:50.269365+0000","last_clean":"2024-04-28T13:35:50.269365+0000","last_became_active":"2024-04-28T13:33:34.006227+0000","last_became_peered":"2024-04-28T13:33:34.006227+0000","last_unstale":"2024-04-28T13:35:50.269365+0000","last_undegraded":"2024-04-28T13:35:50.269365+0000","last_fullsized":"2024-04-28T13:35:50.269365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139737+0000","last_change":"2024-04-28T13:33:33.999169+0000","last_active":"2024-04-28T13:34:05.139737+0000","last_peered":"2024-04-28T13:34:05.139737+0000","last_clean":"2024-04-28T13:34:05.139737+0000","last_became_active":"2024-04-28T13:33:33.998179+0000","last_became_peered":"2024-04-28T13:33:33.998179+0000","last_unstale":"2024-04-28T13:34:05.139737+0000","last_undegraded":"2024-04-28T13:34:05.139737+0000","last_fullsized":"2024-04-28T13:34:05.139737+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:12.546153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139678+0000","last_change":"2024-04-28T13:33:33.998034+0000","last_active":"2024-04-28T13:34:05.139678+0000","last_peered":"2024-04-28T13:34:05.139678+0000","last_clean":"2024-04-28T13:34:05.139678+0000","last_became_active":"2024-04-28T13:33:33.997648+0000","last_became_peered":"2024-04-28T13:33:33.997648+0000","last_unstale":"2024-04-28T13:34:05.139678+0000","last_undegraded":"2024-04-28T13:34:05.139678+0000","last_fullsized":"2024-04-28T13:34:05.139678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:30.570296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.141197+0000","last_change":"2024-04-28T13:33:34.007275+0000","last_active":"2024-04-28T13:34:05.141197+0000","last_peered":"2024-04-28T13:34:05.141197+0000","last_clean":"2024-04-28T13:34:05.141197+0000","last_became_active":"2024-04-28T13:33:34.005017+0000","last_became_peered":"2024-04-28T13:33:34.005017+0000","last_unstale":"2024-04-28T13:34:05.141197+0000","last_undegraded":"2024-04-28T13:34:05.141197+0000","last_fullsized":"2024-04-28T13:34:05.141197+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:37:53.261649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531787+0000","last_change":"2024-04-28T13:33:33.995804+0000","last_active":"2024-04-28T13:34:05.531787+0000","last_peered":"2024-04-28T13:34:05.531787+0000","last_clean":"2024-04-28T13:34:05.531787+0000","last_became_active":"2024-04-28T13:33:33.995502+0000","last_became_peered":"2024-04-28T13:33:33.995502+0000","last_unstale":"2024-04-28T13:34:05.531787+0000","last_undegraded":"2024-04-28T13:34:05.531787+0000","last_fullsized":"2024-04-28T13:34:05.531787+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:27:53.804713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347538+0000","last_change":"2024-04-28T13:33:34.006447+0000","last_active":"2024-04-28T13:34:04.347538+0000","last_peered":"2024-04-28T13:34:04.347538+0000","last_clean":"2024-04-28T13:34:04.347538+0000","last_became_active":"2024-04-28T13:33:34.005656+0000","last_became_peered":"2024-04-28T13:33:34.005656+0000","last_unstale":"2024-04-28T13:34:04.347538+0000","last_undegraded":"2024-04-28T13:34:04.347538+0000","last_fullsized":"2024-04-28T13:34:04.347538+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355564+0000","last_change":"2024-04-28T13:33:34.005608+0000","last_active":"2024-04-28T13:34:04.355564+0000","last_peered":"2024-04-28T13:34:04.355564+0000","last_clean":"2024-04-28T13:34:04.355564+0000","last_became_active":"2024-04-28T13:33:34.005316+0000","last_became_peered":"2024-04-28T13:33:34.005316+0000","last_unstale":"2024-04-28T13:34:04.355564+0000","last_undegraded":"2024-04-28T13:34:04.355564+0000","last_fullsized":"2024-04-28T13:34:04.355564+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:04.373486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139766+0000","last_change":"2024-04-28T13:33:34.002648+0000","last_active":"2024-04-28T13:34:05.139766+0000","last_peered":"2024-04-28T13:34:05.139766+0000","last_clean":"2024-04-28T13:34:05.139766+0000","last_became_active":"2024-04-28T13:33:34.002375+0000","last_became_peered":"2024-04-28T13:33:34.002375+0000","last_unstale":"2024-04-28T13:34:05.139766+0000","last_undegraded":"2024-04-28T13:34:05.139766+0000","last_fullsized":"2024-04-28T13:34:05.139766+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:42.082730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346448+0000","last_change":"2024-04-28T13:33:33.999443+0000","last_active":"2024-04-28T13:34:04.346448+0000","last_peered":"2024-04-28T13:34:04.346448+0000","last_clean":"2024-04-28T13:34:04.346448+0000","last_became_active":"2024-04-28T13:33:33.998905+0000","last_became_peered":"2024-04-28T13:33:33.998905+0000","last_unstale":"2024-04-28T13:34:04.346448+0000","last_undegraded":"2024-04-28T13:34:04.346448+0000","last_fullsized":"2024-04-28T13:34:04.346448+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:47.537234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344364+0000","last_change":"2024-04-28T13:33:34.008881+0000","last_active":"2024-04-28T13:34:04.344364+0000","last_peered":"2024-04-28T13:34:04.344364+0000","last_clean":"2024-04-28T13:34:04.344364+0000","last_became_active":"2024-04-28T13:33:34.008297+0000","last_became_peered":"2024-04-28T13:33:34.008297+0000","last_unstale":"2024-04-28T13:34:04.344364+0000","last_undegraded":"2024-04-28T13:34:04.344364+0000","last_fullsized":"2024-04-28T13:34:04.344364+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:41:26.988112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531605+0000","last_change":"2024-04-28T13:33:34.003176+0000","last_active":"2024-04-28T13:34:05.531605+0000","last_peered":"2024-04-28T13:34:05.531605+0000","last_clean":"2024-04-28T13:34:05.531605+0000","last_became_active":"2024-04-28T13:33:34.002915+0000","last_became_peered":"2024-04-28T13:33:34.002915+0000","last_unstale":"2024-04-28T13:34:05.531605+0000","last_undegraded":"2024-04-28T13:34:05.531605+0000","last_fullsized":"2024-04-28T13:34:05.531605+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:52:06.849498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493215+0000","last_change":"2024-04-28T13:33:34.005669+0000","last_active":"2024-04-28T13:34:04.493215+0000","last_peered":"2024-04-28T13:34:04.493215+0000","last_clean":"2024-04-28T13:34:04.493215+0000","last_became_active":"2024-04-28T13:33:34.005365+0000","last_became_peered":"2024-04-28T13:33:34.005365+0000","last_unstale":"2024-04-28T13:34:04.493215+0000","last_undegraded":"2024-04-28T13:34:04.493215+0000","last_fullsized":"2024-04-28T13:34:04.493215+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:18:56.809253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":23,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345243+0000","last_change":"2024-04-28T13:33:32.998037+0000","last_active":"2024-04-28T13:34:04.345243+0000","last_peered":"2024-04-28T13:34:04.345243+0000","last_clean":"2024-04-28T13:34:04.345243+0000","last_became_active":"2024-04-28T13:33:32.997694+0000","last_became_peered":"2024-04-28T13:33:32.997694+0000","last_unstale":"2024-04-28T13:34:04.345243+0000","last_undegraded":"2024-04-28T13:34:04.345243+0000","last_fullsized":"2024-04-28T13:34:04.345243+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:02.432193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.141337+0000","last_change":"2024-04-28T13:33:33.989048+0000","last_active":"2024-04-28T13:34:05.141337+0000","last_peered":"2024-04-28T13:34:05.141337+0000","last_clean":"2024-04-28T13:34:05.141337+0000","last_became_active":"2024-04-28T13:33:33.988758+0000","last_became_peered":"2024-04-28T13:33:33.988758+0000","last_unstale":"2024-04-28T13:34:05.141337+0000","last_undegraded":"2024-04-28T13:34:05.141337+0000","last_fullsized":"2024-04-28T13:34:05.141337+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:14.152642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347981+0000","last_change":"2024-04-28T13:33:32.999239+0000","last_active":"2024-04-28T13:34:04.347981+0000","last_peered":"2024-04-28T13:34:04.347981+0000","last_clean":"2024-04-28T13:34:04.347981+0000","last_became_active":"2024-04-28T13:33:32.998813+0000","last_became_peered":"2024-04-28T13:33:32.998813+0000","last_unstale":"2024-04-28T13:34:04.347981+0000","last_undegraded":"2024-04-28T13:34:04.347981+0000","last_fullsized":"2024-04-28T13:34:04.347981+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:22:34.444155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355706+0000","last_change":"2024-04-28T13:33:34.003176+0000","last_active":"2024-04-28T13:34:04.355706+0000","last_peered":"2024-04-28T13:34:04.355706+0000","last_clean":"2024-04-28T13:34:04.355706+0000","last_became_active":"2024-04-28T13:33:34.002904+0000","last_became_peered":"2024-04-28T13:33:34.002904+0000","last_unstale":"2024-04-28T13:34:04.355706+0000","last_undegraded":"2024-04-28T13:34:04.355706+0000","last_fullsized":"2024-04-28T13:34:04.355706+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:55.386924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493775+0000","last_change":"2024-04-28T13:33:32.997729+0000","last_active":"2024-04-28T13:34:04.493775+0000","last_peered":"2024-04-28T13:34:04.493775+0000","last_clean":"2024-04-28T13:34:04.493775+0000","last_became_active":"2024-04-28T13:33:32.996881+0000","last_became_peered":"2024-04-28T13:33:32.996881+0000","last_unstale":"2024-04-28T13:34:04.493775+0000","last_undegraded":"2024-04-28T13:34:04.493775+0000","last_fullsized":"2024-04-28T13:34:04.493775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493141+0000","last_change":"2024-04-28T13:33:34.004339+0000","last_active":"2024-04-28T13:34:04.493141+0000","last_peered":"2024-04-28T13:34:04.493141+0000","last_clean":"2024-04-28T13:34:04.493141+0000","last_became_active":"2024-04-28T13:33:34.002361+0000","last_became_peered":"2024-04-28T13:33:34.002361+0000","last_unstale":"2024-04-28T13:34:04.493141+0000","last_undegraded":"2024-04-28T13:34:04.493141+0000","last_fullsized":"2024-04-28T13:34:04.493141+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:31.671210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350207+0000","last_change":"2024-04-28T13:33:33.000754+0000","last_active":"2024-04-28T13:34:04.350207+0000","last_peered":"2024-04-28T13:34:04.350207+0000","last_clean":"2024-04-28T13:34:04.350207+0000","last_became_active":"2024-04-28T13:33:33.000387+0000","last_became_peered":"2024-04-28T13:33:33.000387+0000","last_unstale":"2024-04-28T13:34:04.350207+0000","last_undegraded":"2024-04-28T13:34:04.350207+0000","last_fullsized":"2024-04-28T13:34:04.350207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:28.481114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346329+0000","last_change":"2024-04-28T13:33:33.994316+0000","last_active":"2024-04-28T13:34:04.346329+0000","last_peered":"2024-04-28T13:34:04.346329+0000","last_clean":"2024-04-28T13:34:04.346329+0000","last_became_active":"2024-04-28T13:33:33.994002+0000","last_became_peered":"2024-04-28T13:33:33.994002+0000","last_unstale":"2024-04-28T13:34:04.346329+0000","last_undegraded":"2024-04-28T13:34:04.346329+0000","last_fullsized":"2024-04-28T13:34:04.346329+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:03.446736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344611+0000","last_change":"2024-04-28T13:33:32.999540+0000","last_active":"2024-04-28T13:34:04.344611+0000","last_peered":"2024-04-28T13:34:04.344611+0000","last_clean":"2024-04-28T13:34:04.344611+0000","last_became_active":"2024-04-28T13:33:32.999147+0000","last_became_peered":"2024-04-28T13:33:32.999147+0000","last_unstale":"2024-04-28T13:34:04.344611+0000","last_undegraded":"2024-04-28T13:34:04.344611+0000","last_fullsized":"2024-04-28T13:34:04.344611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:45.657949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880688+0000","last_change":"2024-04-28T13:33:34.000424+0000","last_active":"2024-04-28T13:34:04.880688+0000","last_peered":"2024-04-28T13:34:04.880688+0000","last_clean":"2024-04-28T13:34:04.880688+0000","last_became_active":"2024-04-28T13:33:33.999414+0000","last_became_peered":"2024-04-28T13:33:33.999414+0000","last_unstale":"2024-04-28T13:34:04.880688+0000","last_undegraded":"2024-04-28T13:34:04.880688+0000","last_fullsized":"2024-04-28T13:34:04.880688+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346593+0000","last_change":"2024-04-28T13:33:32.988943+0000","last_active":"2024-04-28T13:34:04.346593+0000","last_peered":"2024-04-28T13:34:04.346593+0000","last_clean":"2024-04-28T13:34:04.346593+0000","last_became_active":"2024-04-28T13:33:32.986596+0000","last_became_peered":"2024-04-28T13:33:32.986596+0000","last_unstale":"2024-04-28T13:34:04.346593+0000","last_undegraded":"2024-04-28T13:34:04.346593+0000","last_fullsized":"2024-04-28T13:34:04.346593+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880001+0000","last_change":"2024-04-28T13:33:34.002579+0000","last_active":"2024-04-28T13:34:04.880001+0000","last_peered":"2024-04-28T13:34:04.880001+0000","last_clean":"2024-04-28T13:34:04.880001+0000","last_became_active":"2024-04-28T13:33:34.002254+0000","last_became_peered":"2024-04-28T13:33:34.002254+0000","last_unstale":"2024-04-28T13:34:04.880001+0000","last_undegraded":"2024-04-28T13:34:04.880001+0000","last_fullsized":"2024-04-28T13:34:04.880001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354254+0000","last_change":"2024-04-28T13:33:32.996052+0000","last_active":"2024-04-28T13:34:04.354254+0000","last_peered":"2024-04-28T13:34:04.354254+0000","last_clean":"2024-04-28T13:34:04.354254+0000","last_became_active":"2024-04-28T13:33:32.990258+0000","last_became_peered":"2024-04-28T13:33:32.990258+0000","last_unstale":"2024-04-28T13:34:04.354254+0000","last_undegraded":"2024-04-28T13:34:04.354254+0000","last_fullsized":"2024-04-28T13:34:04.354254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:51.278295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139482+0000","last_change":"2024-04-28T13:33:33.998180+0000","last_active":"2024-04-28T13:34:05.139482+0000","last_peered":"2024-04-28T13:34:05.139482+0000","last_clean":"2024-04-28T13:34:05.139482+0000","last_became_active":"2024-04-28T13:33:33.997730+0000","last_became_peered":"2024-04-28T13:33:33.997730+0000","last_unstale":"2024-04-28T13:34:05.139482+0000","last_undegraded":"2024-04-28T13:34:05.139482+0000","last_fullsized":"2024-04-28T13:34:05.139482+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:22:35.458698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.880250+0000","last_change":"2024-04-28T13:33:32.989759+0000","last_active":"2024-04-28T13:34:04.880250+0000","last_peered":"2024-04-28T13:34:04.880250+0000","last_clean":"2024-04-28T13:34:04.880250+0000","last_became_active":"2024-04-28T13:33:32.988565+0000","last_became_peered":"2024-04-28T13:33:32.988565+0000","last_unstale":"2024-04-28T13:34:04.880250+0000","last_undegraded":"2024-04-28T13:34:04.880250+0000","last_fullsized":"2024-04-28T13:34:04.880250+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347610+0000","last_change":"2024-04-28T13:33:34.007736+0000","last_active":"2024-04-28T13:34:04.347610+0000","last_peered":"2024-04-28T13:34:04.347610+0000","last_clean":"2024-04-28T13:34:04.347610+0000","last_became_active":"2024-04-28T13:33:34.007170+0000","last_became_peered":"2024-04-28T13:33:34.007170+0000","last_unstale":"2024-04-28T13:34:04.347610+0000","last_undegraded":"2024-04-28T13:34:04.347610+0000","last_fullsized":"2024-04-28T13:34:04.347610+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:01:33.789936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347247+0000","last_change":"2024-04-28T13:33:32.992483+0000","last_active":"2024-04-28T13:34:04.347247+0000","last_peered":"2024-04-28T13:34:04.347247+0000","last_clean":"2024-04-28T13:34:04.347247+0000","last_became_active":"2024-04-28T13:33:32.992216+0000","last_became_peered":"2024-04-28T13:33:32.992216+0000","last_unstale":"2024-04-28T13:34:04.347247+0000","last_undegraded":"2024-04-28T13:34:04.347247+0000","last_fullsized":"2024-04-28T13:34:04.347247+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:11.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350567+0000","last_change":"2024-04-28T13:33:34.009420+0000","last_active":"2024-04-28T13:34:04.350567+0000","last_peered":"2024-04-28T13:34:04.350567+0000","last_clean":"2024-04-28T13:34:04.350567+0000","last_became_active":"2024-04-28T13:33:34.009020+0000","last_became_peered":"2024-04-28T13:33:34.009020+0000","last_unstale":"2024-04-28T13:34:04.350567+0000","last_undegraded":"2024-04-28T13:34:04.350567+0000","last_fullsized":"2024-04-28T13:34:04.350567+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:48:58.152999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.881032+0000","last_change":"2024-04-28T13:33:32.997498+0000","last_active":"2024-04-28T13:34:04.881032+0000","last_peered":"2024-04-28T13:34:04.881032+0000","last_clean":"2024-04-28T13:34:04.881032+0000","last_became_active":"2024-04-28T13:33:32.997237+0000","last_became_peered":"2024-04-28T13:33:32.997237+0000","last_unstale":"2024-04-28T13:34:04.881032+0000","last_undegraded":"2024-04-28T13:34:04.881032+0000","last_fullsized":"2024-04-28T13:34:04.881032+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344944+0000","last_change":"2024-04-28T13:33:34.002865+0000","last_active":"2024-04-28T13:34:04.344944+0000","last_peered":"2024-04-28T13:34:04.344944+0000","last_clean":"2024-04-28T13:34:04.344944+0000","last_became_active":"2024-04-28T13:33:34.002421+0000","last_became_peered":"2024-04-28T13:33:34.002421+0000","last_unstale":"2024-04-28T13:34:04.344944+0000","last_undegraded":"2024-04-28T13:34:04.344944+0000","last_fullsized":"2024-04-28T13:34:04.344944+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:51:46.332983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350650+0000","last_change":"2024-04-28T13:33:33.004559+0000","last_active":"2024-04-28T13:34:04.350650+0000","last_peered":"2024-04-28T13:34:04.350650+0000","last_clean":"2024-04-28T13:34:04.350650+0000","last_became_active":"2024-04-28T13:33:33.004153+0000","last_became_peered":"2024-04-28T13:33:33.004153+0000","last_unstale":"2024-04-28T13:34:04.350650+0000","last_undegraded":"2024-04-28T13:34:04.350650+0000","last_fullsized":"2024-04-28T13:34:04.350650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:02.432193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350366+0000","last_change":"2024-04-28T13:33:34.006854+0000","last_active":"2024-04-28T13:34:04.350366+0000","last_peered":"2024-04-28T13:34:04.350366+0000","last_clean":"2024-04-28T13:34:04.350366+0000","last_became_active":"2024-04-28T13:33:34.006561+0000","last_became_peered":"2024-04-28T13:33:34.006561+0000","last_unstale":"2024-04-28T13:34:04.350366+0000","last_undegraded":"2024-04-28T13:34:04.350366+0000","last_fullsized":"2024-04-28T13:34:04.350366+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:54.852573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'12","reported_seq":31,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:50.069921+0000","last_change":"2024-04-28T13:33:32.991367+0000","last_active":"2024-04-28T13:35:50.069921+0000","last_peered":"2024-04-28T13:35:50.069921+0000","last_clean":"2024-04-28T13:35:50.069921+0000","last_became_active":"2024-04-28T13:33:32.991060+0000","last_became_peered":"2024-04-28T13:33:32.991060+0000","last_unstale":"2024-04-28T13:35:50.069921+0000","last_undegraded":"2024-04-28T13:35:50.069921+0000","last_fullsized":"2024-04-28T13:35:50.069921+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:00:16.144423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3397,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.139961+0000","last_change":"2024-04-28T13:33:33.996658+0000","last_active":"2024-04-28T13:34:05.139961+0000","last_peered":"2024-04-28T13:34:05.139961+0000","last_clean":"2024-04-28T13:34:05.139961+0000","last_became_active":"2024-04-28T13:33:33.996260+0000","last_became_peered":"2024-04-28T13:33:33.996260+0000","last_unstale":"2024-04-28T13:34:05.139961+0000","last_undegraded":"2024-04-28T13:34:05.139961+0000","last_fullsized":"2024-04-28T13:34:05.139961+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:43:52.292838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345103+0000","last_change":"2024-04-28T13:33:32.990968+0000","last_active":"2024-04-28T13:34:04.345103+0000","last_peered":"2024-04-28T13:34:04.345103+0000","last_clean":"2024-04-28T13:34:04.345103+0000","last_became_active":"2024-04-28T13:33:32.990718+0000","last_became_peered":"2024-04-28T13:33:32.990718+0000","last_unstale":"2024-04-28T13:34:04.345103+0000","last_undegraded":"2024-04-28T13:34:04.345103+0000","last_fullsized":"2024-04-28T13:34:04.345103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:03.358943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:55.257532+0000","last_change":"2024-04-28T13:35:55.257532+0000","last_active":"2024-04-28T13:35:55.257532+0000","last_peered":"2024-04-28T13:35:55.257532+0000","last_clean":"2024-04-28T13:35:55.257532+0000","last_became_active":"2024-04-28T13:33:34.003599+0000","last_became_peered":"2024-04-28T13:33:34.003599+0000","last_unstale":"2024-04-28T13:35:55.257532+0000","last_undegraded":"2024-04-28T13:35:55.257532+0000","last_fullsized":"2024-04-28T13:35:55.257532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:35:55.257326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:35:55.257326+0000","last_clean_scrub_stamp":"2024-04-28T13:35:55.257326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:30:58.763462+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349662+0000","last_change":"2024-04-28T13:33:33.000521+0000","last_active":"2024-04-28T13:34:04.349662+0000","last_peered":"2024-04-28T13:34:04.349662+0000","last_clean":"2024-04-28T13:34:04.349662+0000","last_became_active":"2024-04-28T13:33:32.998867+0000","last_became_peered":"2024-04-28T13:33:32.998867+0000","last_unstale":"2024-04-28T13:34:04.349662+0000","last_undegraded":"2024-04-28T13:34:04.349662+0000","last_fullsized":"2024-04-28T13:34:04.349662+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:43.562896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344244+0000","last_change":"2024-04-28T13:33:34.006368+0000","last_active":"2024-04-28T13:34:04.344244+0000","last_peered":"2024-04-28T13:34:04.344244+0000","last_clean":"2024-04-28T13:34:04.344244+0000","last_became_active":"2024-04-28T13:33:34.002579+0000","last_became_peered":"2024-04-28T13:33:34.002579+0000","last_unstale":"2024-04-28T13:34:04.344244+0000","last_undegraded":"2024-04-28T13:34:04.344244+0000","last_fullsized":"2024-04-28T13:34:04.344244+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:27:51.330692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345465+0000","last_change":"2024-04-28T13:33:32.991648+0000","last_active":"2024-04-28T13:34:04.345465+0000","last_peered":"2024-04-28T13:34:04.345465+0000","last_clean":"2024-04-28T13:34:04.345465+0000","last_became_active":"2024-04-28T13:33:32.991369+0000","last_became_peered":"2024-04-28T13:33:32.991369+0000","last_unstale":"2024-04-28T13:34:04.345465+0000","last_undegraded":"2024-04-28T13:34:04.345465+0000","last_fullsized":"2024-04-28T13:34:04.345465+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:40:43.562896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349732+0000","last_change":"2024-04-28T13:33:34.010407+0000","last_active":"2024-04-28T13:34:04.349732+0000","last_peered":"2024-04-28T13:34:04.349732+0000","last_clean":"2024-04-28T13:34:04.349732+0000","last_became_active":"2024-04-28T13:33:34.010077+0000","last_became_peered":"2024-04-28T13:33:34.010077+0000","last_unstale":"2024-04-28T13:34:04.349732+0000","last_undegraded":"2024-04-28T13:34:04.349732+0000","last_fullsized":"2024-04-28T13:34:04.349732+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:20:07.691466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:54.777046+0000","last_change":"2024-04-28T13:33:32.989587+0000","last_active":"2024-04-28T13:35:54.777046+0000","last_peered":"2024-04-28T13:35:54.777046+0000","last_clean":"2024-04-28T13:35:54.777046+0000","last_became_active":"2024-04-28T13:33:32.988579+0000","last_became_peered":"2024-04-28T13:33:32.988579+0000","last_unstale":"2024-04-28T13:35:54.777046+0000","last_undegraded":"2024-04-28T13:35:54.777046+0000","last_fullsized":"2024-04-28T13:35:54.777046+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.532050+0000","last_change":"2024-04-28T13:33:34.012322+0000","last_active":"2024-04-28T13:34:05.532050+0000","last_peered":"2024-04-28T13:34:05.532050+0000","last_clean":"2024-04-28T13:34:05.532050+0000","last_became_active":"2024-04-28T13:33:34.012112+0000","last_became_peered":"2024-04-28T13:33:34.012112+0000","last_unstale":"2024-04-28T13:34:05.532050+0000","last_undegraded":"2024-04-28T13:34:05.532050+0000","last_fullsized":"2024-04-28T13:34:05.532050+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:10.015862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531304+0000","last_change":"2024-04-28T13:33:32.990958+0000","last_active":"2024-04-28T13:34:05.531304+0000","last_peered":"2024-04-28T13:34:05.531304+0000","last_clean":"2024-04-28T13:34:05.531304+0000","last_became_active":"2024-04-28T13:33:32.990659+0000","last_became_peered":"2024-04-28T13:33:32.990659+0000","last_unstale":"2024-04-28T13:34:05.531304+0000","last_undegraded":"2024-04-28T13:34:05.531304+0000","last_fullsized":"2024-04-28T13:34:05.531304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:24:34.028457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:50.269430+0000","last_change":"2024-04-28T13:33:34.005832+0000","last_active":"2024-04-28T13:35:50.269430+0000","last_peered":"2024-04-28T13:35:50.269430+0000","last_clean":"2024-04-28T13:35:50.269430+0000","last_became_active":"2024-04-28T13:33:34.005364+0000","last_became_peered":"2024-04-28T13:33:34.005364+0000","last_unstale":"2024-04-28T13:35:50.269430+0000","last_undegraded":"2024-04-28T13:35:50.269430+0000","last_fullsized":"2024-04-28T13:35:50.269430+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.348084+0000","last_change":"2024-04-28T13:33:32.995338+0000","last_active":"2024-04-28T13:34:04.348084+0000","last_peered":"2024-04-28T13:34:04.348084+0000","last_clean":"2024-04-28T13:34:04.348084+0000","last_became_active":"2024-04-28T13:33:32.994798+0000","last_became_peered":"2024-04-28T13:33:32.994798+0000","last_unstale":"2024-04-28T13:34:04.348084+0000","last_undegraded":"2024-04-28T13:34:04.348084+0000","last_fullsized":"2024-04-28T13:34:04.348084+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:29.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.140801+0000","last_change":"2024-04-28T13:33:34.002242+0000","last_active":"2024-04-28T13:34:05.140801+0000","last_peered":"2024-04-28T13:34:05.140801+0000","last_clean":"2024-04-28T13:34:05.140801+0000","last_became_active":"2024-04-28T13:33:34.001893+0000","last_became_peered":"2024-04-28T13:33:34.001893+0000","last_unstale":"2024-04-28T13:34:05.140801+0000","last_undegraded":"2024-04-28T13:34:05.140801+0000","last_fullsized":"2024-04-28T13:34:05.140801+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:35.969568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.355718+0000","last_change":"2024-04-28T13:33:32.991441+0000","last_active":"2024-04-28T13:34:04.355718+0000","last_peered":"2024-04-28T13:34:04.355718+0000","last_clean":"2024-04-28T13:34:04.355718+0000","last_became_active":"2024-04-28T13:33:32.990745+0000","last_became_peered":"2024-04-28T13:33:32.990745+0000","last_unstale":"2024-04-28T13:34:04.355718+0000","last_undegraded":"2024-04-28T13:34:04.355718+0000","last_fullsized":"2024-04-28T13:34:04.355718+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:30.656667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.140616+0000","last_change":"2024-04-28T13:33:34.003094+0000","last_active":"2024-04-28T13:34:05.140616+0000","last_peered":"2024-04-28T13:34:05.140616+0000","last_clean":"2024-04-28T13:34:05.140616+0000","last_became_active":"2024-04-28T13:33:34.002228+0000","last_became_peered":"2024-04-28T13:33:34.002228+0000","last_unstale":"2024-04-28T13:34:05.140616+0000","last_undegraded":"2024-04-28T13:34:05.140616+0000","last_fullsized":"2024-04-28T13:34:05.140616+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:55:31.671210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344933+0000","last_change":"2024-04-28T13:33:33.003720+0000","last_active":"2024-04-28T13:34:04.344933+0000","last_peered":"2024-04-28T13:34:04.344933+0000","last_clean":"2024-04-28T13:34:04.344933+0000","last_became_active":"2024-04-28T13:33:33.003389+0000","last_became_peered":"2024-04-28T13:33:33.003389+0000","last_unstale":"2024-04-28T13:34:04.344933+0000","last_undegraded":"2024-04-28T13:34:04.344933+0000","last_fullsized":"2024-04-28T13:34:04.344933+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:18.814381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345100+0000","last_change":"2024-04-28T13:33:34.006579+0000","last_active":"2024-04-28T13:34:04.345100+0000","last_peered":"2024-04-28T13:34:04.345100+0000","last_clean":"2024-04-28T13:34:04.345100+0000","last_became_active":"2024-04-28T13:33:34.006282+0000","last_became_peered":"2024-04-28T13:33:34.006282+0000","last_unstale":"2024-04-28T13:34:04.345100+0000","last_undegraded":"2024-04-28T13:34:04.345100+0000","last_fullsized":"2024-04-28T13:34:04.345100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:18:15.243319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345377+0000","last_change":"2024-04-28T13:33:32.999347+0000","last_active":"2024-04-28T13:34:04.345377+0000","last_peered":"2024-04-28T13:34:04.345377+0000","last_clean":"2024-04-28T13:34:04.345377+0000","last_became_active":"2024-04-28T13:33:32.998923+0000","last_became_peered":"2024-04-28T13:33:32.998923+0000","last_unstale":"2024-04-28T13:34:04.345377+0000","last_undegraded":"2024-04-28T13:34:04.345377+0000","last_fullsized":"2024-04-28T13:34:04.345377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:17:46.522691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.492925+0000","last_change":"2024-04-28T13:33:34.011875+0000","last_active":"2024-04-28T13:34:04.492925+0000","last_peered":"2024-04-28T13:34:04.492925+0000","last_clean":"2024-04-28T13:34:04.492925+0000","last_became_active":"2024-04-28T13:33:34.011271+0000","last_became_peered":"2024-04-28T13:33:34.011271+0000","last_unstale":"2024-04-28T13:34:04.492925+0000","last_undegraded":"2024-04-28T13:34:04.492925+0000","last_fullsized":"2024-04-28T13:34:04.492925+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:35.969568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354592+0000","last_change":"2024-04-28T13:33:33.002216+0000","last_active":"2024-04-28T13:34:04.354592+0000","last_peered":"2024-04-28T13:34:04.354592+0000","last_clean":"2024-04-28T13:34:04.354592+0000","last_became_active":"2024-04-28T13:33:33.000335+0000","last_became_peered":"2024-04-28T13:33:33.000335+0000","last_unstale":"2024-04-28T13:34:04.354592+0000","last_undegraded":"2024-04-28T13:34:04.354592+0000","last_fullsized":"2024-04-28T13:34:04.354592+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:55:34.955026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.531906+0000","last_change":"2024-04-28T13:33:34.000104+0000","last_active":"2024-04-28T13:34:05.531906+0000","last_peered":"2024-04-28T13:34:05.531906+0000","last_clean":"2024-04-28T13:34:05.531906+0000","last_became_active":"2024-04-28T13:33:33.999839+0000","last_became_peered":"2024-04-28T13:33:33.999839+0000","last_unstale":"2024-04-28T13:34:05.531906+0000","last_undegraded":"2024-04-28T13:34:05.531906+0000","last_fullsized":"2024-04-28T13:34:05.531906+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:03:14.657009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:45.268829+0000","last_change":"2024-04-28T13:33:32.992149+0000","last_active":"2024-04-28T13:35:45.268829+0000","last_peered":"2024-04-28T13:35:45.268829+0000","last_clean":"2024-04-28T13:35:45.268829+0000","last_became_active":"2024-04-28T13:33:32.991800+0000","last_became_peered":"2024-04-28T13:33:32.991800+0000","last_unstale":"2024-04-28T13:35:45.268829+0000","last_undegraded":"2024-04-28T13:35:45.268829+0000","last_fullsized":"2024-04-28T13:35:45.268829+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:45.268790+0000","last_change":"2024-04-28T13:33:34.011133+0000","last_active":"2024-04-28T13:35:45.268790+0000","last_peered":"2024-04-28T13:35:45.268790+0000","last_clean":"2024-04-28T13:35:45.268790+0000","last_became_active":"2024-04-28T13:33:34.010896+0000","last_became_peered":"2024-04-28T13:33:34.010896+0000","last_unstale":"2024-04-28T13:35:45.268790+0000","last_undegraded":"2024-04-28T13:35:45.268790+0000","last_fullsized":"2024-04-28T13:35:45.268790+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.348967+0000","last_change":"2024-04-28T13:33:33.000859+0000","last_active":"2024-04-28T13:34:04.348967+0000","last_peered":"2024-04-28T13:34:04.348967+0000","last_clean":"2024-04-28T13:34:04.348967+0000","last_became_active":"2024-04-28T13:33:33.000095+0000","last_became_peered":"2024-04-28T13:33:33.000095+0000","last_unstale":"2024-04-28T13:34:04.348967+0000","last_undegraded":"2024-04-28T13:34:04.348967+0000","last_fullsized":"2024-04-28T13:34:04.348967+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:18.221214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:05.141479+0000","last_change":"2024-04-28T13:33:34.008405+0000","last_active":"2024-04-28T13:34:05.141479+0000","last_peered":"2024-04-28T13:34:05.141479+0000","last_clean":"2024-04-28T13:34:05.141479+0000","last_became_active":"2024-04-28T13:33:34.007425+0000","last_became_peered":"2024-04-28T13:33:34.007425+0000","last_unstale":"2024-04-28T13:34:05.141479+0000","last_undegraded":"2024-04-28T13:34:05.141479+0000","last_fullsized":"2024-04-28T13:34:05.141479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:29.495657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"47'3","reported_seq":22,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:45.273788+0000","last_change":"2024-04-28T13:33:32.998594+0000","last_active":"2024-04-28T13:35:45.273788+0000","last_peered":"2024-04-28T13:35:45.273788+0000","last_clean":"2024-04-28T13:35:45.273788+0000","last_became_active":"2024-04-28T13:33:32.998328+0000","last_became_peered":"2024-04-28T13:33:32.998328+0000","last_unstale":"2024-04-28T13:35:45.273788+0000","last_undegraded":"2024-04-28T13:35:45.273788+0000","last_fullsized":"2024-04-28T13:35:45.273788+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:18.221214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:40.268049+0000","last_change":"2024-04-28T13:33:33.998134+0000","last_active":"2024-04-28T13:35:40.268049+0000","last_peered":"2024-04-28T13:35:40.268049+0000","last_clean":"2024-04-28T13:35:40.268049+0000","last_became_active":"2024-04-28T13:33:33.997834+0000","last_became_peered":"2024-04-28T13:33:33.997834+0000","last_unstale":"2024-04-28T13:35:40.268049+0000","last_undegraded":"2024-04-28T13:35:40.268049+0000","last_fullsized":"2024-04-28T13:35:40.268049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350856+0000","last_change":"2024-04-28T13:33:33.005737+0000","last_active":"2024-04-28T13:34:04.350856+0000","last_peered":"2024-04-28T13:34:04.350856+0000","last_clean":"2024-04-28T13:34:04.350856+0000","last_became_active":"2024-04-28T13:33:33.005179+0000","last_became_peered":"2024-04-28T13:33:33.005179+0000","last_unstale":"2024-04-28T13:34:04.350856+0000","last_undegraded":"2024-04-28T13:34:04.350856+0000","last_fullsized":"2024-04-28T13:34:04.350856+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:31:45.657949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350592+0000","last_change":"2024-04-28T13:33:34.011725+0000","last_active":"2024-04-28T13:34:04.350592+0000","last_peered":"2024-04-28T13:34:04.350592+0000","last_clean":"2024-04-28T13:34:04.350592+0000","last_became_active":"2024-04-28T13:33:34.011422+0000","last_became_peered":"2024-04-28T13:33:34.011422+0000","last_unstale":"2024-04-28T13:34:04.350592+0000","last_undegraded":"2024-04-28T13:34:04.350592+0000","last_fullsized":"2024-04-28T13:34:04.350592+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:28.548233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.492969+0000","last_change":"2024-04-28T13:33:32.997338+0000","last_active":"2024-04-28T13:34:04.492969+0000","last_peered":"2024-04-28T13:34:04.492969+0000","last_clean":"2024-04-28T13:34:04.492969+0000","last_became_active":"2024-04-28T13:33:32.996934+0000","last_became_peered":"2024-04-28T13:33:32.996934+0000","last_unstale":"2024-04-28T13:34:04.492969+0000","last_undegraded":"2024-04-28T13:34:04.492969+0000","last_fullsized":"2024-04-28T13:34:04.492969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:08:29.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.345593+0000","last_change":"2024-04-28T13:33:34.020082+0000","last_active":"2024-04-28T13:34:04.345593+0000","last_peered":"2024-04-28T13:34:04.345593+0000","last_clean":"2024-04-28T13:34:04.345593+0000","last_became_active":"2024-04-28T13:33:34.018922+0000","last_became_peered":"2024-04-28T13:33:34.018922+0000","last_unstale":"2024-04-28T13:34:04.345593+0000","last_undegraded":"2024-04-28T13:34:04.345593+0000","last_fullsized":"2024-04-28T13:34:04.345593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:53:12.865627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.350434+0000","last_change":"2024-04-28T13:33:33.000626+0000","last_active":"2024-04-28T13:34:04.350434+0000","last_peered":"2024-04-28T13:34:04.350434+0000","last_clean":"2024-04-28T13:34:04.350434+0000","last_became_active":"2024-04-28T13:33:33.000049+0000","last_became_peered":"2024-04-28T13:33:33.000049+0000","last_unstale":"2024-04-28T13:34:04.350434+0000","last_undegraded":"2024-04-28T13:34:04.350434+0000","last_fullsized":"2024-04-28T13:34:04.350434+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:18.814381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.344459+0000","last_change":"2024-04-28T13:33:34.004230+0000","last_active":"2024-04-28T13:34:04.344459+0000","last_peered":"2024-04-28T13:34:04.344459+0000","last_clean":"2024-04-28T13:34:04.344459+0000","last_became_active":"2024-04-28T13:33:34.003940+0000","last_became_peered":"2024-04-28T13:33:34.003940+0000","last_unstale":"2024-04-28T13:34:04.344459+0000","last_undegraded":"2024-04-28T13:34:04.344459+0000","last_fullsized":"2024-04-28T13:34:04.344459+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:11.732507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.346733+0000","last_change":"2024-04-28T13:33:32.991869+0000","last_active":"2024-04-28T13:34:04.346733+0000","last_peered":"2024-04-28T13:34:04.346733+0000","last_clean":"2024-04-28T13:34:04.346733+0000","last_became_active":"2024-04-28T13:33:32.991435+0000","last_became_peered":"2024-04-28T13:33:32.991435+0000","last_unstale":"2024-04-28T13:34:04.346733+0000","last_undegraded":"2024-04-28T13:34:04.346733+0000","last_fullsized":"2024-04-28T13:34:04.346733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:41.068187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.354423+0000","last_change":"2024-04-28T13:33:34.004178+0000","last_active":"2024-04-28T13:34:04.354423+0000","last_peered":"2024-04-28T13:34:04.354423+0000","last_clean":"2024-04-28T13:34:04.354423+0000","last_became_active":"2024-04-28T13:33:34.002092+0000","last_became_peered":"2024-04-28T13:33:34.002092+0000","last_unstale":"2024-04-28T13:34:04.354423+0000","last_undegraded":"2024-04-28T13:34:04.354423+0000","last_fullsized":"2024-04-28T13:34:04.354423+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:03.446736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.493055+0000","last_change":"2024-04-28T13:33:33.003796+0000","last_active":"2024-04-28T13:34:04.493055+0000","last_peered":"2024-04-28T13:34:04.493055+0000","last_clean":"2024-04-28T13:34:04.493055+0000","last_became_active":"2024-04-28T13:33:33.003374+0000","last_became_peered":"2024-04-28T13:33:33.003374+0000","last_unstale":"2024-04-28T13:34:04.493055+0000","last_undegraded":"2024-04-28T13:34:04.493055+0000","last_fullsized":"2024-04-28T13:34:04.493055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:53:11.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.347052+0000","last_change":"2024-04-28T13:33:34.001551+0000","last_active":"2024-04-28T13:34:04.347052+0000","last_peered":"2024-04-28T13:34:04.347052+0000","last_clean":"2024-04-28T13:34:04.347052+0000","last_became_active":"2024-04-28T13:33:33.999963+0000","last_became_peered":"2024-04-28T13:33:33.999963+0000","last_unstale":"2024-04-28T13:34:04.347052+0000","last_undegraded":"2024-04-28T13:34:04.347052+0000","last_fullsized":"2024-04-28T13:34:04.347052+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:27:19.828924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:35.267129+0000","last_change":"2024-04-28T13:33:32.991033+0000","last_active":"2024-04-28T13:35:35.267129+0000","last_peered":"2024-04-28T13:35:35.267129+0000","last_clean":"2024-04-28T13:35:35.267129+0000","last_became_active":"2024-04-28T13:33:32.990726+0000","last_became_peered":"2024-04-28T13:33:32.990726+0000","last_unstale":"2024-04-28T13:35:35.267129+0000","last_undegraded":"2024-04-28T13:35:35.267129+0000","last_fullsized":"2024-04-28T13:35:35.267129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:45.268870+0000","last_change":"2024-04-28T13:33:34.005421+0000","last_active":"2024-04-28T13:35:45.268870+0000","last_peered":"2024-04-28T13:35:45.268870+0000","last_clean":"2024-04-28T13:35:45.268870+0000","last_became_active":"2024-04-28T13:33:34.004776+0000","last_became_peered":"2024-04-28T13:33:34.004776+0000","last_unstale":"2024-04-28T13:35:45.268870+0000","last_undegraded":"2024-04-28T13:35:45.268870+0000","last_fullsized":"2024-04-28T13:35:45.268870+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:32.947967+0000","last_clean_scrub_stamp":"2024-04-28T13:33:32.947967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.356034+0000","last_change":"2024-04-28T13:33:32.991462+0000","last_active":"2024-04-28T13:34:04.356034+0000","last_peered":"2024-04-28T13:34:04.356034+0000","last_clean":"2024-04-28T13:34:04.356034+0000","last_became_active":"2024-04-28T13:33:32.991149+0000","last_became_peered":"2024-04-28T13:33:32.991149+0000","last_unstale":"2024-04-28T13:34:04.356034+0000","last_undegraded":"2024-04-28T13:34:04.356034+0000","last_fullsized":"2024-04-28T13:34:04.356034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:33:31.933424+0000","last_clean_scrub_stamp":"2024-04-28T13:33:31.933424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:23:41.068187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":220,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:34:04.349073+0000","last_change":"2024-04-28T13:31:11.831538+0000","last_active":"2024-04-28T13:34:04.349073+0000","last_peered":"2024-04-28T13:34:04.349073+0000","last_clean":"2024-04-28T13:34:04.349073+0000","last_became_active":"2024-04-28T13:31:11.828959+0000","last_became_peered":"2024-04-28T13:31:11.828959+0000","last_unstale":"2024-04-28T13:34:04.349073+0000","last_undegraded":"2024-04-28T13:34:04.349073+0000","last_fullsized":"2024-04-28T13:34:04.349073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:31:10.444388+0000","last_clean_scrub_stamp":"2024-04-28T13:31:10.444388+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:25:27.078911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":4195,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":47,"ondisk_log_size":47,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343692,"statfs":{"total":96636764160,"available":96607940608,"internally_reserved":0,"allocated":1150976,"data_stored":818777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":561152,"data_stored":228991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":228389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":565248,"data_stored":231716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":565248,"data_stored":228957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":569344,"data_stored":231738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":228389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1150976,"data_stored":818721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T13:36:00.093 INFO:tasks.ceph:pgid 7.3f last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.093 INFO:tasks.ceph:pgid 7.3e last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.093 INFO:tasks.ceph:pgid 7.3d last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.093 INFO:tasks.ceph:pgid 7.3c last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.093 INFO:tasks.ceph:pgid 7.3b last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.094 INFO:tasks.ceph:pgid 7.3a last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.094 INFO:tasks.ceph:pgid 7.39 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.094 INFO:tasks.ceph:pgid 7.38 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.094 INFO:tasks.ceph:pgid 7.37 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.094 INFO:tasks.ceph:pgid 7.36 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.094 INFO:tasks.ceph:pgid 7.35 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.095 INFO:tasks.ceph:pgid 7.34 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.095 INFO:tasks.ceph:pgid 7.33 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.095 INFO:tasks.ceph:pgid 7.32 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.095 INFO:tasks.ceph:pgid 7.31 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.095 INFO:tasks.ceph:pgid 7.30 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.095 INFO:tasks.ceph:pgid 7.2f last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.095 INFO:tasks.ceph:pgid 7.2e last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.096 INFO:tasks.ceph:pgid 7.2d last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.096 INFO:tasks.ceph:pgid 7.2c last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.096 INFO:tasks.ceph:pgid 7.2b last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.096 INFO:tasks.ceph:pgid 7.2a last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.096 INFO:tasks.ceph:pgid 7.29 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.096 INFO:tasks.ceph:pgid 7.28 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.097 INFO:tasks.ceph:pgid 7.27 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.097 INFO:tasks.ceph:pgid 7.26 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.097 INFO:tasks.ceph:pgid 7.25 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.097 INFO:tasks.ceph:pgid 7.24 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.097 INFO:tasks.ceph:pgid 7.23 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.097 INFO:tasks.ceph:pgid 7.22 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.097 INFO:tasks.ceph:pgid 7.21 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.098 INFO:tasks.ceph:pgid 7.20 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.098 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.098 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.098 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.098 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.098 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.099 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.099 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.099 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.099 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.099 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.099 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.099 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.100 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.100 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.100 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.100 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.100 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.100 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.101 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.101 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.101 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.101 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.101 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.101 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.102 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.102 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.102 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.102 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.102 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.102 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.102 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.103 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.103 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.103 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.103 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.103 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.103 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.103 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.104 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.104 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.104 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.104 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.104 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.104 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.104 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.105 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.105 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.105 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.105 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.105 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.105 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.106 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.106 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.106 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.106 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.106 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.106 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.106 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.107 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.107 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.107 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.107 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-04-28T13:33:32.947967+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=32, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.107 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-04-28T13:33:31.933424+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=33, tm_sec=31, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.107 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-28T13:31:10.444388+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=31, tm_sec=10, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=13, tm_min=35, tm_sec=53, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T13:36:00.107 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-28T13:36:10.760 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:36:10.768 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:36:20.109 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T13:36:20.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.320+0000 7faa89000640 1 Processor -- start 2024-04-28T13:36:20.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.320+0000 7faa89000640 1 -- start start 2024-04-28T13:36:20.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.322+0000 7faa89000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa841629c0 0x7faa840d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:20.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.322+0000 7faa89000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7faa84163bf0 0x7faa84163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:20.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.322+0000 7faa89000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faa84163570 0x7faa840b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:20.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.322+0000 7faa89000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7faa84162da0 con 0x7faa84163570 2024-04-28T13:36:20.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.322+0000 7faa89000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7faa840b6190 con 0x7faa84163bf0 2024-04-28T13:36:20.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.322+0000 7faa89000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7faa840b6470 con 0x7faa841629c0 2024-04-28T13:36:20.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.322+0000 7faa83e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faa84163570 0x7faa840b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:20.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.322+0000 7faa83e00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faa84163570 0x7faa840b46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43978/0 (socket says 172.21.15.196:43978) 2024-04-28T13:36:20.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.322+0000 7faa83e00640 1 -- 172.21.15.196:0/3976727854 learned_addr learned my addr 172.21.15.196:0/3976727854 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:36:20.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.323+0000 7faa83400640 1 --2- 172.21.15.196:0/3976727854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa841629c0 0x7faa840d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:20.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.323+0000 7faa83e00640 1 -- 172.21.15.196:0/3976727854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa841629c0 msgr2=0x7faa840d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:36:20.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.323+0000 7faa7aa00640 1 --2- 172.21.15.196:0/3976727854 >> v2:172.21.15.203:3300/0 conn(0x7faa84163bf0 0x7faa84163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:20.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.323+0000 7faa83e00640 1 --2- 172.21.15.196:0/3976727854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa841629c0 0x7faa840d3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.323+0000 7faa83e00640 1 -- 172.21.15.196:0/3976727854 >> v2:172.21.15.203:3300/0 conn(0x7faa84163bf0 msgr2=0x7faa84163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:36:20.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.323+0000 7faa83e00640 1 --2- 172.21.15.196:0/3976727854 >> v2:172.21.15.203:3300/0 conn(0x7faa84163bf0 0x7faa84163030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.323+0000 7faa83e00640 1 -- 172.21.15.196:0/3976727854 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa840b6750 con 0x7faa84163570 2024-04-28T13:36:20.326 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.324+0000 7faa83e00640 1 --2- 172.21.15.196:0/3976727854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa84163570 0x7faa840b46b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7faa8408f950 tx=0x7faa7c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7dbca2c9e04df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:36:20.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.324+0000 7faa82a00640 1 -- 172.21.15.196:0/3976727854 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c036070 con 0x7faa84163570 2024-04-28T13:36:20.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.324+0000 7faa82a00640 1 -- 172.21.15.196:0/3976727854 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa7c031600 con 0x7faa84163570 2024-04-28T13:36:20.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.324+0000 7faa82a00640 1 -- 172.21.15.196:0/3976727854 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c0319d0 con 0x7faa84163570 2024-04-28T13:36:20.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.324+0000 7faa89000640 1 -- 172.21.15.196:0/3976727854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa84163570 msgr2=0x7faa840b46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:20.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.324+0000 7faa89000640 1 --2- 172.21.15.196:0/3976727854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa84163570 0x7faa840b46b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7faa8408f950 tx=0x7faa7c02a6a0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.325+0000 7faa89000640 1 -- 172.21.15.196:0/3976727854 shutdown_connections 2024-04-28T13:36:20.327 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.325+0000 7faa89000640 1 --2- 172.21.15.196:0/3976727854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa84163570 0x7faa840b46b0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.325+0000 7faa89000640 1 --2- 172.21.15.196:0/3976727854 >> v2:172.21.15.203:3300/0 conn(0x7faa84163bf0 0x7faa84163030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.325+0000 7faa89000640 1 --2- 172.21.15.196:0/3976727854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa841629c0 0x7faa840d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.325+0000 7faa89000640 1 -- 172.21.15.196:0/3976727854 >> 172.21.15.196:0/3976727854 conn(0x7faa8409e570 msgr2=0x7faa840d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:36:20.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.325+0000 7faa89000640 1 -- 172.21.15.196:0/3976727854 shutdown_connections 2024-04-28T13:36:20.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.325+0000 7faa89000640 1 -- 172.21.15.196:0/3976727854 wait complete. 2024-04-28T13:36:20.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.326+0000 7faa89000640 1 Processor -- start 2024-04-28T13:36:20.330 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.327+0000 7faa89000640 1 -- start start 2024-04-28T13:36:20.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.327+0000 7faa89000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa841629c0 0x7faa841f9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:20.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa89000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa84163570 0x7faa840b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:20.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa89000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa84163bf0 0x7faa8415c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:20.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa89000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa840d3c50 con 0x7faa841629c0 2024-04-28T13:36:20.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa89000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faa840b1e60 con 0x7faa84163570 2024-04-28T13:36:20.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa89000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7faa840d3dd0 con 0x7faa84163bf0 2024-04-28T13:36:20.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa83400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa841629c0 0x7faa841f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:20.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa83400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa841629c0 0x7faa841f9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43990/0 (socket says 172.21.15.196:43990) 2024-04-28T13:36:20.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa83400640 1 -- 172.21.15.196:0/770496697 learned_addr learned my addr 172.21.15.196:0/770496697 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:36:20.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa83e00640 1 --2- 172.21.15.196:0/770496697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa84163bf0 0x7faa8415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:20.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa7aa00640 1 --2- 172.21.15.196:0/770496697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa84163570 0x7faa840b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:20.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa83400640 1 -- 172.21.15.196:0/770496697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa84163bf0 msgr2=0x7faa8415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:20.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa83400640 1 --2- 172.21.15.196:0/770496697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa84163bf0 0x7faa8415c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa83400640 1 -- 172.21.15.196:0/770496697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa84163570 msgr2=0x7faa840b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:20.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa83400640 1 --2- 172.21.15.196:0/770496697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa84163570 0x7faa840b9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa83400640 1 -- 172.21.15.196:0/770496697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa840ba070 con 0x7faa841629c0 2024-04-28T13:36:20.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.328+0000 7faa7aa00640 1 --2- 172.21.15.196:0/770496697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa84163570 0x7faa840b9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T13:36:20.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.329+0000 7faa83400640 1 --2- 172.21.15.196:0/770496697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa841629c0 0x7faa841f9810 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7faa7400c5a0 tx=0x7faa7400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:36:20.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.329+0000 7faa7be00640 1 -- 172.21.15.196:0/770496697 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74015070 con 0x7faa841629c0 2024-04-28T13:36:20.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.329+0000 7faa7be00640 1 -- 172.21.15.196:0/770496697 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa74004220 con 0x7faa841629c0 2024-04-28T13:36:20.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.329+0000 7faa7be00640 1 -- 172.21.15.196:0/770496697 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74003610 con 0x7faa841629c0 2024-04-28T13:36:20.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.329+0000 7faa89000640 1 -- 172.21.15.196:0/770496697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa840ba310 con 0x7faa841629c0 2024-04-28T13:36:20.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.329+0000 7faa89000640 1 -- 172.21.15.196:0/770496697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa840ba8d0 con 0x7faa841629c0 2024-04-28T13:36:20.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.331+0000 7faa89000640 1 -- 172.21.15.196:0/770496697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa8406a290 con 0x7faa841629c0 2024-04-28T13:36:20.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.331+0000 7faa7be00640 1 -- 172.21.15.196:0/770496697 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7faa740037b0 con 0x7faa841629c0 2024-04-28T13:36:20.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.332+0000 7faa7be00640 1 --2- 172.21.15.196:0/770496697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faa6006c980 0x7faa6006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:20.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.332+0000 7faa7be00640 1 -- 172.21.15.196:0/770496697 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7faa7408bb20 con 0x7faa841629c0 2024-04-28T13:36:20.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.333+0000 7faa7aa00640 1 --2- 172.21.15.196:0/770496697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faa6006c980 0x7faa6006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:20.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.334+0000 7faa7aa00640 1 --2- 172.21.15.196:0/770496697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faa6006c980 0x7faa6006ee40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faa841601f0 tx=0x7faa6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:36:20.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.337+0000 7faa7be00640 1 -- 172.21.15.196:0/770496697 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa74059da0 con 0x7faa841629c0 2024-04-28T13:36:20.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.534+0000 7faa89000640 1 -- 172.21.15.196:0/770496697 --> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa840cbcb0 con 0x7faa6006c980 2024-04-28T13:36:20.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.552+0000 7faa7be00640 1 -- 172.21.15.196:0/770496697 <== mgr.4103 v2:172.21.15.203:6832/3172262857 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251903 (secure 0 0 0) 0x7faa840cbcb0 con 0x7faa6006c980 2024-04-28T13:36:20.556 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:36:20.558 INFO:teuthology.orchestra.run.smithi196.stderr:dumped all 2024-04-28T13:36:20.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.557+0000 7faa89000640 1 -- 172.21.15.196:0/770496697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faa6006c980 msgr2=0x7faa6006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:20.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.557+0000 7faa89000640 1 --2- 172.21.15.196:0/770496697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faa6006c980 0x7faa6006ee40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faa841601f0 tx=0x7faa6c008040 comp rx=0 tx=0).stop 2024-04-28T13:36:20.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.557+0000 7faa89000640 1 -- 172.21.15.196:0/770496697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa841629c0 msgr2=0x7faa841f9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:20.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.557+0000 7faa89000640 1 --2- 172.21.15.196:0/770496697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa841629c0 0x7faa841f9810 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7faa7400c5a0 tx=0x7faa7400ca60 comp rx=0 tx=0).stop 2024-04-28T13:36:20.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.557+0000 7faa89000640 1 -- 172.21.15.196:0/770496697 shutdown_connections 2024-04-28T13:36:20.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.557+0000 7faa89000640 1 --2- 172.21.15.196:0/770496697 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7faa6006c980 0x7faa6006ee40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.557+0000 7faa89000640 1 --2- 172.21.15.196:0/770496697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa84163bf0 0x7faa8415c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.557+0000 7faa89000640 1 --2- 172.21.15.196:0/770496697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa84163570 0x7faa840b9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.557+0000 7faa89000640 1 --2- 172.21.15.196:0/770496697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa841629c0 0x7faa841f9810 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.557+0000 7faa89000640 1 -- 172.21.15.196:0/770496697 >> 172.21.15.196:0/770496697 conn(0x7faa8409e570 msgr2=0x7faa84160e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:36:20.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.557+0000 7faa89000640 1 -- 172.21.15.196:0/770496697 shutdown_connections 2024-04-28T13:36:20.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.557+0000 7faa89000640 1 -- 172.21.15.196:0/770496697 wait complete. 2024-04-28T13:36:20.579 INFO:teuthology.orchestra.run.smithi196.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-04-28T13:36:20.295519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594563,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":214,"num_write_kb":2437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":185,"ondisk_log_size":185,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225832,"kb_used_data":5544,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748888,"statfs":{"total":773094113280,"available":772862861312,"internally_reserved":0,"allocated":5677056,"data_stored":3015678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004700"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:07.230845+0000","last_change":"2024-04-28T13:36:07.230899+0000","last_active":"2024-04-28T13:36:07.230845+0000","last_peered":"2024-04-28T13:36:07.230845+0000","last_clean":"2024-04-28T13:36:07.230845+0000","last_became_active":"2024-04-28T13:33:33.999486+0000","last_became_peered":"2024-04-28T13:33:33.999486+0000","last_unstale":"2024-04-28T13:36:07.230845+0000","last_undegraded":"2024-04-28T13:36:07.230845+0000","last_fullsized":"2024-04-28T13:36:07.230845+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:07.230644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:07.230644+0000","last_clean_scrub_stamp":"2024-04-28T13:36:07.230644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:18:26.298550+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:03.956068+0000","last_change":"2024-04-28T13:36:03.956121+0000","last_active":"2024-04-28T13:36:03.956068+0000","last_peered":"2024-04-28T13:36:03.956068+0000","last_clean":"2024-04-28T13:36:03.956068+0000","last_became_active":"2024-04-28T13:33:34.003551+0000","last_became_peered":"2024-04-28T13:33:34.003551+0000","last_unstale":"2024-04-28T13:36:03.956068+0000","last_undegraded":"2024-04-28T13:36:03.956068+0000","last_fullsized":"2024-04-28T13:36:03.956068+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:03.955849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:03.955849+0000","last_clean_scrub_stamp":"2024-04-28T13:36:03.955849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:38:15.690300+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:03.677649+0000","last_change":"2024-04-28T13:36:03.677703+0000","last_active":"2024-04-28T13:36:03.677649+0000","last_peered":"2024-04-28T13:36:03.677649+0000","last_clean":"2024-04-28T13:36:03.677649+0000","last_became_active":"2024-04-28T13:33:33.996413+0000","last_became_peered":"2024-04-28T13:33:33.996413+0000","last_unstale":"2024-04-28T13:36:03.677649+0000","last_undegraded":"2024-04-28T13:36:03.677649+0000","last_fullsized":"2024-04-28T13:36:03.677649+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:03.677447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:03.677447+0000","last_clean_scrub_stamp":"2024-04-28T13:36:03.677447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:25:43.810651+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:04.294710+0000","last_change":"2024-04-28T13:36:04.294768+0000","last_active":"2024-04-28T13:36:04.294710+0000","last_peered":"2024-04-28T13:36:04.294710+0000","last_clean":"2024-04-28T13:36:04.294710+0000","last_became_active":"2024-04-28T13:33:33.999818+0000","last_became_peered":"2024-04-28T13:33:33.999818+0000","last_unstale":"2024-04-28T13:36:04.294710+0000","last_undegraded":"2024-04-28T13:36:04.294710+0000","last_fullsized":"2024-04-28T13:36:04.294710+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:04.294501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:04.294501+0000","last_clean_scrub_stamp":"2024-04-28T13:36:04.294501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:18:23.362407+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:05.932569+0000","last_change":"2024-04-28T13:36:05.932626+0000","last_active":"2024-04-28T13:36:05.932569+0000","last_peered":"2024-04-28T13:36:05.932569+0000","last_clean":"2024-04-28T13:36:05.932569+0000","last_became_active":"2024-04-28T13:33:34.004436+0000","last_became_peered":"2024-04-28T13:33:34.004436+0000","last_unstale":"2024-04-28T13:36:05.932569+0000","last_undegraded":"2024-04-28T13:36:05.932569+0000","last_fullsized":"2024-04-28T13:36:05.932569+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:05.932360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:05.932360+0000","last_clean_scrub_stamp":"2024-04-28T13:36:05.932360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:06:21.333081+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:11.454771+0000","last_change":"2024-04-28T13:36:11.454868+0000","last_active":"2024-04-28T13:36:11.454771+0000","last_peered":"2024-04-28T13:36:11.454771+0000","last_clean":"2024-04-28T13:36:11.454771+0000","last_became_active":"2024-04-28T13:33:34.010839+0000","last_became_peered":"2024-04-28T13:33:34.010839+0000","last_unstale":"2024-04-28T13:36:11.454771+0000","last_undegraded":"2024-04-28T13:36:11.454771+0000","last_fullsized":"2024-04-28T13:36:11.454771+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:11.454410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:11.454410+0000","last_clean_scrub_stamp":"2024-04-28T13:36:11.454410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:38:31.990734+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:08.192631+0000","last_change":"2024-04-28T13:36:08.192687+0000","last_active":"2024-04-28T13:36:08.192631+0000","last_peered":"2024-04-28T13:36:08.192631+0000","last_clean":"2024-04-28T13:36:08.192631+0000","last_became_active":"2024-04-28T13:33:34.002948+0000","last_became_peered":"2024-04-28T13:33:34.002948+0000","last_unstale":"2024-04-28T13:36:08.192631+0000","last_undegraded":"2024-04-28T13:36:08.192631+0000","last_fullsized":"2024-04-28T13:36:08.192631+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:08.192386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:08.192386+0000","last_clean_scrub_stamp":"2024-04-28T13:36:08.192386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:19:44.217162+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:10.688736+0000","last_change":"2024-04-28T13:36:10.688794+0000","last_active":"2024-04-28T13:36:10.688736+0000","last_peered":"2024-04-28T13:36:10.688736+0000","last_clean":"2024-04-28T13:36:10.688736+0000","last_became_active":"2024-04-28T13:34:05.142351+0000","last_became_peered":"2024-04-28T13:34:05.142351+0000","last_unstale":"2024-04-28T13:36:10.688736+0000","last_undegraded":"2024-04-28T13:36:10.688736+0000","last_fullsized":"2024-04-28T13:36:10.688736+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:10.688534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:10.688534+0000","last_clean_scrub_stamp":"2024-04-28T13:36:10.688534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:45:02.637812+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:11.311665+0000","last_change":"2024-04-28T13:36:11.311721+0000","last_active":"2024-04-28T13:36:11.311665+0000","last_peered":"2024-04-28T13:36:11.311665+0000","last_clean":"2024-04-28T13:36:11.311665+0000","last_became_active":"2024-04-28T13:33:34.012900+0000","last_became_peered":"2024-04-28T13:33:34.012900+0000","last_unstale":"2024-04-28T13:36:11.311665+0000","last_undegraded":"2024-04-28T13:36:11.311665+0000","last_fullsized":"2024-04-28T13:36:11.311665+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:11.311431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:11.311431+0000","last_clean_scrub_stamp":"2024-04-28T13:36:11.311431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:10:01.714757+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:08.266648+0000","last_change":"2024-04-28T13:36:08.266701+0000","last_active":"2024-04-28T13:36:08.266648+0000","last_peered":"2024-04-28T13:36:08.266648+0000","last_clean":"2024-04-28T13:36:08.266648+0000","last_became_active":"2024-04-28T13:33:34.005157+0000","last_became_peered":"2024-04-28T13:33:34.005157+0000","last_unstale":"2024-04-28T13:36:08.266648+0000","last_undegraded":"2024-04-28T13:36:08.266648+0000","last_fullsized":"2024-04-28T13:36:08.266648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:08.266401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:08.266401+0000","last_clean_scrub_stamp":"2024-04-28T13:36:08.266401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:45:00.215678+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:09.306606+0000","last_change":"2024-04-28T13:36:09.306693+0000","last_active":"2024-04-28T13:36:09.306606+0000","last_peered":"2024-04-28T13:36:09.306606+0000","last_clean":"2024-04-28T13:36:09.306606+0000","last_became_active":"2024-04-28T13:33:34.010811+0000","last_became_peered":"2024-04-28T13:33:34.010811+0000","last_unstale":"2024-04-28T13:36:09.306606+0000","last_undegraded":"2024-04-28T13:36:09.306606+0000","last_fullsized":"2024-04-28T13:36:09.306606+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:09.306397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:09.306397+0000","last_clean_scrub_stamp":"2024-04-28T13:36:09.306397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:15:12.790443+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:07.926589+0000","last_change":"2024-04-28T13:36:07.926642+0000","last_active":"2024-04-28T13:36:07.926589+0000","last_peered":"2024-04-28T13:36:07.926589+0000","last_clean":"2024-04-28T13:36:07.926589+0000","last_became_active":"2024-04-28T13:33:34.011580+0000","last_became_peered":"2024-04-28T13:33:34.011580+0000","last_unstale":"2024-04-28T13:36:07.926589+0000","last_undegraded":"2024-04-28T13:36:07.926589+0000","last_fullsized":"2024-04-28T13:36:07.926589+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:07.926374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:07.926374+0000","last_clean_scrub_stamp":"2024-04-28T13:36:07.926374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:22:25.878481+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:00.836109+0000","last_change":"2024-04-28T13:36:00.836169+0000","last_active":"2024-04-28T13:36:00.836109+0000","last_peered":"2024-04-28T13:36:00.836109+0000","last_clean":"2024-04-28T13:36:00.836109+0000","last_became_active":"2024-04-28T13:33:34.000568+0000","last_became_peered":"2024-04-28T13:33:34.000568+0000","last_unstale":"2024-04-28T13:36:00.836109+0000","last_undegraded":"2024-04-28T13:36:00.836109+0000","last_fullsized":"2024-04-28T13:36:00.836109+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:00.835907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:00.835907+0000","last_clean_scrub_stamp":"2024-04-28T13:36:00.835907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:04:31.607942+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:08.972900+0000","last_change":"2024-04-28T13:36:08.972952+0000","last_active":"2024-04-28T13:36:08.972900+0000","last_peered":"2024-04-28T13:36:08.972900+0000","last_clean":"2024-04-28T13:36:08.972900+0000","last_became_active":"2024-04-28T13:34:04.495748+0000","last_became_peered":"2024-04-28T13:34:04.495748+0000","last_unstale":"2024-04-28T13:36:08.972900+0000","last_undegraded":"2024-04-28T13:36:08.972900+0000","last_fullsized":"2024-04-28T13:36:08.972900+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:08.972704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:08.972704+0000","last_clean_scrub_stamp":"2024-04-28T13:36:08.972704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:02:58.949326+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:04.457588+0000","last_change":"2024-04-28T13:36:04.457680+0000","last_active":"2024-04-28T13:36:04.457588+0000","last_peered":"2024-04-28T13:36:04.457588+0000","last_clean":"2024-04-28T13:36:04.457588+0000","last_became_active":"2024-04-28T13:33:33.998087+0000","last_became_peered":"2024-04-28T13:33:33.998087+0000","last_unstale":"2024-04-28T13:36:04.457588+0000","last_undegraded":"2024-04-28T13:36:04.457588+0000","last_fullsized":"2024-04-28T13:36:04.457588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:04.457385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:04.457385+0000","last_clean_scrub_stamp":"2024-04-28T13:36:04.457385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:59:30.265253+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:00.464371+0000","last_change":"2024-04-28T13:36:00.464429+0000","last_active":"2024-04-28T13:36:00.464371+0000","last_peered":"2024-04-28T13:36:00.464371+0000","last_clean":"2024-04-28T13:36:00.464371+0000","last_became_active":"2024-04-28T13:33:33.988827+0000","last_became_peered":"2024-04-28T13:33:33.988827+0000","last_unstale":"2024-04-28T13:36:00.464371+0000","last_undegraded":"2024-04-28T13:36:00.464371+0000","last_fullsized":"2024-04-28T13:36:00.464371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:00.464171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:00.464171+0000","last_clean_scrub_stamp":"2024-04-28T13:36:00.464171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:52:46.512305+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:09.649695+0000","last_change":"2024-04-28T13:36:09.649744+0000","last_active":"2024-04-28T13:36:09.649695+0000","last_peered":"2024-04-28T13:36:09.649695+0000","last_clean":"2024-04-28T13:36:09.649695+0000","last_became_active":"2024-04-28T13:33:34.008181+0000","last_became_peered":"2024-04-28T13:33:34.008181+0000","last_unstale":"2024-04-28T13:36:09.649695+0000","last_undegraded":"2024-04-28T13:36:09.649695+0000","last_fullsized":"2024-04-28T13:36:09.649695+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:09.649488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:09.649488+0000","last_clean_scrub_stamp":"2024-04-28T13:36:09.649488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:49:58.092650+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:07.709826+0000","last_change":"2024-04-28T13:36:07.709904+0000","last_active":"2024-04-28T13:36:07.709826+0000","last_peered":"2024-04-28T13:36:07.709826+0000","last_clean":"2024-04-28T13:36:07.709826+0000","last_became_active":"2024-04-28T13:33:34.006793+0000","last_became_peered":"2024-04-28T13:33:34.006793+0000","last_unstale":"2024-04-28T13:36:07.709826+0000","last_undegraded":"2024-04-28T13:36:07.709826+0000","last_fullsized":"2024-04-28T13:36:07.709826+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:07.709631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:07.709631+0000","last_clean_scrub_stamp":"2024-04-28T13:36:07.709631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:26:50.331899+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:10.472835+0000","last_change":"2024-04-28T13:36:10.472889+0000","last_active":"2024-04-28T13:36:10.472835+0000","last_peered":"2024-04-28T13:36:10.472835+0000","last_clean":"2024-04-28T13:36:10.472835+0000","last_became_active":"2024-04-28T13:33:34.009657+0000","last_became_peered":"2024-04-28T13:33:34.009657+0000","last_unstale":"2024-04-28T13:36:10.472835+0000","last_undegraded":"2024-04-28T13:36:10.472835+0000","last_fullsized":"2024-04-28T13:36:10.472835+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:10.472579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:10.472579+0000","last_clean_scrub_stamp":"2024-04-28T13:36:10.472579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:06:25.873299+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:02.838771+0000","last_change":"2024-04-28T13:36:02.838861+0000","last_active":"2024-04-28T13:36:02.838771+0000","last_peered":"2024-04-28T13:36:02.838771+0000","last_clean":"2024-04-28T13:36:02.838771+0000","last_became_active":"2024-04-28T13:33:33.999567+0000","last_became_peered":"2024-04-28T13:33:33.999567+0000","last_unstale":"2024-04-28T13:36:02.838771+0000","last_undegraded":"2024-04-28T13:36:02.838771+0000","last_fullsized":"2024-04-28T13:36:02.838771+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:02.838518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:02.838518+0000","last_clean_scrub_stamp":"2024-04-28T13:36:02.838518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:22.568071+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:10.414052+0000","last_change":"2024-04-28T13:36:10.414105+0000","last_active":"2024-04-28T13:36:10.414052+0000","last_peered":"2024-04-28T13:36:10.414052+0000","last_clean":"2024-04-28T13:36:10.414052+0000","last_became_active":"2024-04-28T13:33:34.008652+0000","last_became_peered":"2024-04-28T13:33:34.008652+0000","last_unstale":"2024-04-28T13:36:10.414052+0000","last_undegraded":"2024-04-28T13:36:10.414052+0000","last_fullsized":"2024-04-28T13:36:10.414052+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:10.413867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:10.413867+0000","last_clean_scrub_stamp":"2024-04-28T13:36:10.413867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:48:50.816988+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:07.313556+0000","last_change":"2024-04-28T13:36:07.313641+0000","last_active":"2024-04-28T13:36:07.313556+0000","last_peered":"2024-04-28T13:36:07.313556+0000","last_clean":"2024-04-28T13:36:07.313556+0000","last_became_active":"2024-04-28T13:33:34.006227+0000","last_became_peered":"2024-04-28T13:33:34.006227+0000","last_unstale":"2024-04-28T13:36:07.313556+0000","last_undegraded":"2024-04-28T13:36:07.313556+0000","last_fullsized":"2024-04-28T13:36:07.313556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:07.313364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:07.313364+0000","last_clean_scrub_stamp":"2024-04-28T13:36:07.313364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:49:55.756526+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:04.427281+0000","last_change":"2024-04-28T13:36:04.427336+0000","last_active":"2024-04-28T13:36:04.427281+0000","last_peered":"2024-04-28T13:36:04.427281+0000","last_clean":"2024-04-28T13:36:04.427281+0000","last_became_active":"2024-04-28T13:33:33.998179+0000","last_became_peered":"2024-04-28T13:33:33.998179+0000","last_unstale":"2024-04-28T13:36:04.427281+0000","last_undegraded":"2024-04-28T13:36:04.427281+0000","last_fullsized":"2024-04-28T13:36:04.427281+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:04.427085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:04.427085+0000","last_clean_scrub_stamp":"2024-04-28T13:36:04.427085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:59:10.008659+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:02.379495+0000","last_change":"2024-04-28T13:36:02.379553+0000","last_active":"2024-04-28T13:36:02.379495+0000","last_peered":"2024-04-28T13:36:02.379495+0000","last_clean":"2024-04-28T13:36:02.379495+0000","last_became_active":"2024-04-28T13:33:33.997648+0000","last_became_peered":"2024-04-28T13:33:33.997648+0000","last_unstale":"2024-04-28T13:36:02.379495+0000","last_undegraded":"2024-04-28T13:36:02.379495+0000","last_fullsized":"2024-04-28T13:36:02.379495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:02.379300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:02.379300+0000","last_clean_scrub_stamp":"2024-04-28T13:36:02.379300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:14:44.481109+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:08.467295+0000","last_change":"2024-04-28T13:36:08.467360+0000","last_active":"2024-04-28T13:36:08.467295+0000","last_peered":"2024-04-28T13:36:08.467295+0000","last_clean":"2024-04-28T13:36:08.467295+0000","last_became_active":"2024-04-28T13:33:34.005017+0000","last_became_peered":"2024-04-28T13:33:34.005017+0000","last_unstale":"2024-04-28T13:36:08.467295+0000","last_undegraded":"2024-04-28T13:36:08.467295+0000","last_fullsized":"2024-04-28T13:36:08.467295+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:08.467103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:08.467103+0000","last_clean_scrub_stamp":"2024-04-28T13:36:08.467103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:56:30.970923+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:58.861156+0000","last_change":"2024-04-28T13:35:58.861206+0000","last_active":"2024-04-28T13:35:58.861156+0000","last_peered":"2024-04-28T13:35:58.861156+0000","last_clean":"2024-04-28T13:35:58.861156+0000","last_became_active":"2024-04-28T13:33:33.995502+0000","last_became_peered":"2024-04-28T13:33:33.995502+0000","last_unstale":"2024-04-28T13:35:58.861156+0000","last_undegraded":"2024-04-28T13:35:58.861156+0000","last_fullsized":"2024-04-28T13:35:58.861156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:35:58.860957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:35:58.860957+0000","last_clean_scrub_stamp":"2024-04-28T13:35:58.860957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:44:09.792540+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:06.678365+0000","last_change":"2024-04-28T13:36:06.678417+0000","last_active":"2024-04-28T13:36:06.678365+0000","last_peered":"2024-04-28T13:36:06.678365+0000","last_clean":"2024-04-28T13:36:06.678365+0000","last_became_active":"2024-04-28T13:33:34.005656+0000","last_became_peered":"2024-04-28T13:33:34.005656+0000","last_unstale":"2024-04-28T13:36:06.678365+0000","last_undegraded":"2024-04-28T13:36:06.678365+0000","last_fullsized":"2024-04-28T13:36:06.678365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:06.678178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:06.678178+0000","last_clean_scrub_stamp":"2024-04-28T13:36:06.678178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:02:56.654799+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:06.955027+0000","last_change":"2024-04-28T13:36:06.955127+0000","last_active":"2024-04-28T13:36:06.955027+0000","last_peered":"2024-04-28T13:36:06.955027+0000","last_clean":"2024-04-28T13:36:06.955027+0000","last_became_active":"2024-04-28T13:33:34.005316+0000","last_became_peered":"2024-04-28T13:33:34.005316+0000","last_unstale":"2024-04-28T13:36:06.955027+0000","last_undegraded":"2024-04-28T13:36:06.955027+0000","last_fullsized":"2024-04-28T13:36:06.955027+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:06.954778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:06.954778+0000","last_clean_scrub_stamp":"2024-04-28T13:36:06.954778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:22:44.912929+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:06.494419+0000","last_change":"2024-04-28T13:36:06.494477+0000","last_active":"2024-04-28T13:36:06.494419+0000","last_peered":"2024-04-28T13:36:06.494419+0000","last_clean":"2024-04-28T13:36:06.494419+0000","last_became_active":"2024-04-28T13:33:34.002375+0000","last_became_peered":"2024-04-28T13:33:34.002375+0000","last_unstale":"2024-04-28T13:36:06.494419+0000","last_undegraded":"2024-04-28T13:36:06.494419+0000","last_fullsized":"2024-04-28T13:36:06.494419+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:06.494207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:06.494207+0000","last_clean_scrub_stamp":"2024-04-28T13:36:06.494207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:11:41.250530+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:04.632549+0000","last_change":"2024-04-28T13:36:04.632606+0000","last_active":"2024-04-28T13:36:04.632549+0000","last_peered":"2024-04-28T13:36:04.632549+0000","last_clean":"2024-04-28T13:36:04.632549+0000","last_became_active":"2024-04-28T13:33:33.998905+0000","last_became_peered":"2024-04-28T13:33:33.998905+0000","last_unstale":"2024-04-28T13:36:04.632549+0000","last_undegraded":"2024-04-28T13:36:04.632549+0000","last_fullsized":"2024-04-28T13:36:04.632549+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:04.632324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:04.632324+0000","last_clean_scrub_stamp":"2024-04-28T13:36:04.632324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:07:17.068877+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:09.443974+0000","last_change":"2024-04-28T13:36:09.444081+0000","last_active":"2024-04-28T13:36:09.443974+0000","last_peered":"2024-04-28T13:36:09.443974+0000","last_clean":"2024-04-28T13:36:09.443974+0000","last_became_active":"2024-04-28T13:33:34.008297+0000","last_became_peered":"2024-04-28T13:33:34.008297+0000","last_unstale":"2024-04-28T13:36:09.443974+0000","last_undegraded":"2024-04-28T13:36:09.443974+0000","last_fullsized":"2024-04-28T13:36:09.443974+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:09.443713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:09.443713+0000","last_clean_scrub_stamp":"2024-04-28T13:36:09.443713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:50:18.782680+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:01.809230+0000","last_change":"2024-04-28T13:36:01.809286+0000","last_active":"2024-04-28T13:36:01.809230+0000","last_peered":"2024-04-28T13:36:01.809230+0000","last_clean":"2024-04-28T13:36:01.809230+0000","last_became_active":"2024-04-28T13:33:34.002915+0000","last_became_peered":"2024-04-28T13:33:34.002915+0000","last_unstale":"2024-04-28T13:36:01.809230+0000","last_undegraded":"2024-04-28T13:36:01.809230+0000","last_fullsized":"2024-04-28T13:36:01.809230+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:01.809034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:01.809034+0000","last_clean_scrub_stamp":"2024-04-28T13:36:01.809034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:34:41.866235+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:00.296553+0000","last_change":"2024-04-28T13:36:00.296632+0000","last_active":"2024-04-28T13:36:00.296553+0000","last_peered":"2024-04-28T13:36:00.296553+0000","last_clean":"2024-04-28T13:36:00.296553+0000","last_became_active":"2024-04-28T13:33:34.005365+0000","last_became_peered":"2024-04-28T13:33:34.005365+0000","last_unstale":"2024-04-28T13:36:00.296553+0000","last_undegraded":"2024-04-28T13:36:00.296553+0000","last_fullsized":"2024-04-28T13:36:00.296553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:00.296359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:00.296359+0000","last_clean_scrub_stamp":"2024-04-28T13:36:00.296359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:00:40.384659+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'6","reported_seq":34,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:59.358517+0000","last_change":"2024-04-28T13:35:59.358580+0000","last_active":"2024-04-28T13:35:59.358517+0000","last_peered":"2024-04-28T13:35:59.358517+0000","last_clean":"2024-04-28T13:35:59.358517+0000","last_became_active":"2024-04-28T13:33:32.997694+0000","last_became_peered":"2024-04-28T13:33:32.997694+0000","last_unstale":"2024-04-28T13:35:59.358517+0000","last_undegraded":"2024-04-28T13:35:59.358517+0000","last_fullsized":"2024-04-28T13:35:59.358517+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'6","last_scrub_stamp":"2024-04-28T13:35:59.358327+0000","last_deep_scrub":"47'6","last_deep_scrub_stamp":"2024-04-28T13:35:59.358327+0000","last_clean_scrub_stamp":"2024-04-28T13:35:59.358327+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:22:37.316477+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:59.507403+0000","last_change":"2024-04-28T13:35:59.507500+0000","last_active":"2024-04-28T13:35:59.507403+0000","last_peered":"2024-04-28T13:35:59.507403+0000","last_clean":"2024-04-28T13:35:59.507403+0000","last_became_active":"2024-04-28T13:33:33.988758+0000","last_became_peered":"2024-04-28T13:33:33.988758+0000","last_unstale":"2024-04-28T13:35:59.507403+0000","last_undegraded":"2024-04-28T13:35:59.507403+0000","last_fullsized":"2024-04-28T13:35:59.507403+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:35:59.507058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:35:59.507058+0000","last_clean_scrub_stamp":"2024-04-28T13:35:59.507058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T01:32:00.622878+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":29,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:01.679902+0000","last_change":"2024-04-28T13:36:01.679967+0000","last_active":"2024-04-28T13:36:01.679902+0000","last_peered":"2024-04-28T13:36:01.679902+0000","last_clean":"2024-04-28T13:36:01.679902+0000","last_became_active":"2024-04-28T13:33:32.998813+0000","last_became_peered":"2024-04-28T13:33:32.998813+0000","last_unstale":"2024-04-28T13:36:01.679902+0000","last_undegraded":"2024-04-28T13:36:01.679902+0000","last_fullsized":"2024-04-28T13:36:01.679902+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'1","last_scrub_stamp":"2024-04-28T13:36:01.679708+0000","last_deep_scrub":"44'1","last_deep_scrub_stamp":"2024-04-28T13:36:01.679708+0000","last_clean_scrub_stamp":"2024-04-28T13:36:01.679708+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:38:13.414160+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:02.929640+0000","last_change":"2024-04-28T13:36:02.929640+0000","last_active":"2024-04-28T13:36:02.929640+0000","last_peered":"2024-04-28T13:36:02.929640+0000","last_clean":"2024-04-28T13:36:02.929640+0000","last_became_active":"2024-04-28T13:33:34.002904+0000","last_became_peered":"2024-04-28T13:33:34.002904+0000","last_unstale":"2024-04-28T13:36:02.929640+0000","last_undegraded":"2024-04-28T13:36:02.929640+0000","last_fullsized":"2024-04-28T13:36:02.929640+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:02.929385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:02.929385+0000","last_clean_scrub_stamp":"2024-04-28T13:36:02.929385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:16:06.766275+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"47'3","reported_seq":31,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:57.384720+0000","last_change":"2024-04-28T13:35:57.384720+0000","last_active":"2024-04-28T13:35:57.384720+0000","last_peered":"2024-04-28T13:35:57.384720+0000","last_clean":"2024-04-28T13:35:57.384720+0000","last_became_active":"2024-04-28T13:33:32.996881+0000","last_became_peered":"2024-04-28T13:33:32.996881+0000","last_unstale":"2024-04-28T13:35:57.384720+0000","last_undegraded":"2024-04-28T13:35:57.384720+0000","last_fullsized":"2024-04-28T13:35:57.384720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'3","last_scrub_stamp":"2024-04-28T13:35:57.384436+0000","last_deep_scrub":"47'3","last_deep_scrub_stamp":"2024-04-28T13:35:57.384436+0000","last_clean_scrub_stamp":"2024-04-28T13:35:57.384436+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:17.113989+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:59.342026+0000","last_change":"2024-04-28T13:35:59.342026+0000","last_active":"2024-04-28T13:35:59.342026+0000","last_peered":"2024-04-28T13:35:59.342026+0000","last_clean":"2024-04-28T13:35:59.342026+0000","last_became_active":"2024-04-28T13:33:34.002361+0000","last_became_peered":"2024-04-28T13:33:34.002361+0000","last_unstale":"2024-04-28T13:35:59.342026+0000","last_undegraded":"2024-04-28T13:35:59.342026+0000","last_fullsized":"2024-04-28T13:35:59.342026+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:35:59.341879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:35:59.341879+0000","last_clean_scrub_stamp":"2024-04-28T13:35:59.341879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:16:42.753030+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'3","reported_seq":31,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:03.303429+0000","last_change":"2024-04-28T13:36:03.303429+0000","last_active":"2024-04-28T13:36:03.303429+0000","last_peered":"2024-04-28T13:36:03.303429+0000","last_clean":"2024-04-28T13:36:03.303429+0000","last_became_active":"2024-04-28T13:33:33.000387+0000","last_became_peered":"2024-04-28T13:33:33.000387+0000","last_unstale":"2024-04-28T13:36:03.303429+0000","last_undegraded":"2024-04-28T13:36:03.303429+0000","last_fullsized":"2024-04-28T13:36:03.303429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'3","last_scrub_stamp":"2024-04-28T13:36:03.303163+0000","last_deep_scrub":"47'3","last_deep_scrub_stamp":"2024-04-28T13:36:03.303163+0000","last_clean_scrub_stamp":"2024-04-28T13:36:03.303163+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:43:45.333364+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:02.727263+0000","last_change":"2024-04-28T13:36:02.727263+0000","last_active":"2024-04-28T13:36:02.727263+0000","last_peered":"2024-04-28T13:36:02.727263+0000","last_clean":"2024-04-28T13:36:02.727263+0000","last_became_active":"2024-04-28T13:33:33.994002+0000","last_became_peered":"2024-04-28T13:33:33.994002+0000","last_unstale":"2024-04-28T13:36:02.727263+0000","last_undegraded":"2024-04-28T13:36:02.727263+0000","last_fullsized":"2024-04-28T13:36:02.727263+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:02.727023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:02.727023+0000","last_clean_scrub_stamp":"2024-04-28T13:36:02.727023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:43:44.757223+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":29,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:02.392808+0000","last_change":"2024-04-28T13:36:02.392808+0000","last_active":"2024-04-28T13:36:02.392808+0000","last_peered":"2024-04-28T13:36:02.392808+0000","last_clean":"2024-04-28T13:36:02.392808+0000","last_became_active":"2024-04-28T13:33:32.999147+0000","last_became_peered":"2024-04-28T13:33:32.999147+0000","last_unstale":"2024-04-28T13:36:02.392808+0000","last_undegraded":"2024-04-28T13:36:02.392808+0000","last_fullsized":"2024-04-28T13:36:02.392808+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'1","last_scrub_stamp":"2024-04-28T13:36:02.392516+0000","last_deep_scrub":"44'1","last_deep_scrub_stamp":"2024-04-28T13:36:02.392516+0000","last_clean_scrub_stamp":"2024-04-28T13:36:02.392516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:19:38.417292+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:02.287581+0000","last_change":"2024-04-28T13:36:02.287581+0000","last_active":"2024-04-28T13:36:02.287581+0000","last_peered":"2024-04-28T13:36:02.287581+0000","last_clean":"2024-04-28T13:36:02.287581+0000","last_became_active":"2024-04-28T13:33:33.999414+0000","last_became_peered":"2024-04-28T13:33:33.999414+0000","last_unstale":"2024-04-28T13:36:02.287581+0000","last_undegraded":"2024-04-28T13:36:02.287581+0000","last_fullsized":"2024-04-28T13:36:02.287581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:02.287335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:02.287335+0000","last_clean_scrub_stamp":"2024-04-28T13:36:02.287335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:04:57.745257+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":29,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:57.662887+0000","last_change":"2024-04-28T13:35:57.662887+0000","last_active":"2024-04-28T13:35:57.662887+0000","last_peered":"2024-04-28T13:35:57.662887+0000","last_clean":"2024-04-28T13:35:57.662887+0000","last_became_active":"2024-04-28T13:33:32.986596+0000","last_became_peered":"2024-04-28T13:33:32.986596+0000","last_unstale":"2024-04-28T13:35:57.662887+0000","last_undegraded":"2024-04-28T13:35:57.662887+0000","last_fullsized":"2024-04-28T13:35:57.662887+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'1","last_scrub_stamp":"2024-04-28T13:35:57.662631+0000","last_deep_scrub":"44'1","last_deep_scrub_stamp":"2024-04-28T13:35:57.662631+0000","last_clean_scrub_stamp":"2024-04-28T13:35:57.662631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:31:01.168767+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:03.247574+0000","last_change":"2024-04-28T13:36:03.247662+0000","last_active":"2024-04-28T13:36:03.247574+0000","last_peered":"2024-04-28T13:36:03.247574+0000","last_clean":"2024-04-28T13:36:03.247574+0000","last_became_active":"2024-04-28T13:33:34.002254+0000","last_became_peered":"2024-04-28T13:33:34.002254+0000","last_unstale":"2024-04-28T13:36:03.247574+0000","last_undegraded":"2024-04-28T13:36:03.247574+0000","last_fullsized":"2024-04-28T13:36:03.247574+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:03.247368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:03.247368+0000","last_clean_scrub_stamp":"2024-04-28T13:36:03.247368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:00.935071+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:00.886495+0000","last_change":"2024-04-28T13:36:00.886552+0000","last_active":"2024-04-28T13:36:00.886495+0000","last_peered":"2024-04-28T13:36:00.886495+0000","last_clean":"2024-04-28T13:36:00.886495+0000","last_became_active":"2024-04-28T13:33:32.990258+0000","last_became_peered":"2024-04-28T13:33:32.990258+0000","last_unstale":"2024-04-28T13:36:00.886495+0000","last_undegraded":"2024-04-28T13:36:00.886495+0000","last_fullsized":"2024-04-28T13:36:00.886495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:00.886287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:00.886287+0000","last_clean_scrub_stamp":"2024-04-28T13:36:00.886287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:51:46.110412+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:03.400383+0000","last_change":"2024-04-28T13:36:03.400485+0000","last_active":"2024-04-28T13:36:03.400383+0000","last_peered":"2024-04-28T13:36:03.400383+0000","last_clean":"2024-04-28T13:36:03.400383+0000","last_became_active":"2024-04-28T13:33:33.997730+0000","last_became_peered":"2024-04-28T13:33:33.997730+0000","last_unstale":"2024-04-28T13:36:03.400383+0000","last_undegraded":"2024-04-28T13:36:03.400383+0000","last_fullsized":"2024-04-28T13:36:03.400383+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:03.400087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:03.400087+0000","last_clean_scrub_stamp":"2024-04-28T13:36:03.400087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:46:15.781321+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:57.263847+0000","last_change":"2024-04-28T13:35:57.263847+0000","last_active":"2024-04-28T13:35:57.263847+0000","last_peered":"2024-04-28T13:35:57.263847+0000","last_clean":"2024-04-28T13:35:57.263847+0000","last_became_active":"2024-04-28T13:33:32.988565+0000","last_became_peered":"2024-04-28T13:33:32.988565+0000","last_unstale":"2024-04-28T13:35:57.263847+0000","last_undegraded":"2024-04-28T13:35:57.263847+0000","last_fullsized":"2024-04-28T13:35:57.263847+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:35:57.263676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:35:57.263676+0000","last_clean_scrub_stamp":"2024-04-28T13:35:57.263676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:45:10.493580+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:08.675464+0000","last_change":"2024-04-28T13:36:08.675516+0000","last_active":"2024-04-28T13:36:08.675464+0000","last_peered":"2024-04-28T13:36:08.675464+0000","last_clean":"2024-04-28T13:36:08.675464+0000","last_became_active":"2024-04-28T13:33:34.007170+0000","last_became_peered":"2024-04-28T13:33:34.007170+0000","last_unstale":"2024-04-28T13:36:08.675464+0000","last_undegraded":"2024-04-28T13:36:08.675464+0000","last_fullsized":"2024-04-28T13:36:08.675464+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:08.675255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:08.675255+0000","last_clean_scrub_stamp":"2024-04-28T13:36:08.675255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:11:15.016607+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:59.680586+0000","last_change":"2024-04-28T13:35:59.680586+0000","last_active":"2024-04-28T13:35:59.680586+0000","last_peered":"2024-04-28T13:35:59.680586+0000","last_clean":"2024-04-28T13:35:59.680586+0000","last_became_active":"2024-04-28T13:33:32.992216+0000","last_became_peered":"2024-04-28T13:33:32.992216+0000","last_unstale":"2024-04-28T13:35:59.680586+0000","last_undegraded":"2024-04-28T13:35:59.680586+0000","last_fullsized":"2024-04-28T13:35:59.680586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:35:59.680342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:35:59.680342+0000","last_clean_scrub_stamp":"2024-04-28T13:35:59.680342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:11:34.436665+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:10.261381+0000","last_change":"2024-04-28T13:36:10.261438+0000","last_active":"2024-04-28T13:36:10.261381+0000","last_peered":"2024-04-28T13:36:10.261381+0000","last_clean":"2024-04-28T13:36:10.261381+0000","last_became_active":"2024-04-28T13:33:34.009020+0000","last_became_peered":"2024-04-28T13:33:34.009020+0000","last_unstale":"2024-04-28T13:36:10.261381+0000","last_undegraded":"2024-04-28T13:36:10.261381+0000","last_fullsized":"2024-04-28T13:36:10.261381+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:10.261176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:10.261176+0000","last_clean_scrub_stamp":"2024-04-28T13:36:10.261176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:49:58.704338+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:00.310479+0000","last_change":"2024-04-28T13:36:00.310530+0000","last_active":"2024-04-28T13:36:00.310479+0000","last_peered":"2024-04-28T13:36:00.310479+0000","last_clean":"2024-04-28T13:36:00.310479+0000","last_became_active":"2024-04-28T13:33:32.997237+0000","last_became_peered":"2024-04-28T13:33:32.997237+0000","last_unstale":"2024-04-28T13:36:00.310479+0000","last_undegraded":"2024-04-28T13:36:00.310479+0000","last_fullsized":"2024-04-28T13:36:00.310479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:00.310285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:00.310285+0000","last_clean_scrub_stamp":"2024-04-28T13:36:00.310285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:25:40.443488+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:05.421967+0000","last_change":"2024-04-28T13:36:05.422019+0000","last_active":"2024-04-28T13:36:05.421967+0000","last_peered":"2024-04-28T13:36:05.421967+0000","last_clean":"2024-04-28T13:36:05.421967+0000","last_became_active":"2024-04-28T13:33:34.002421+0000","last_became_peered":"2024-04-28T13:33:34.002421+0000","last_unstale":"2024-04-28T13:36:05.421967+0000","last_undegraded":"2024-04-28T13:36:05.421967+0000","last_fullsized":"2024-04-28T13:36:05.421967+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:05.421765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:05.421765+0000","last_clean_scrub_stamp":"2024-04-28T13:36:05.421765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:09:43.610868+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'5","reported_seq":33,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:05.315478+0000","last_change":"2024-04-28T13:36:05.315548+0000","last_active":"2024-04-28T13:36:05.315478+0000","last_peered":"2024-04-28T13:36:05.315478+0000","last_clean":"2024-04-28T13:36:05.315478+0000","last_became_active":"2024-04-28T13:33:33.004153+0000","last_became_peered":"2024-04-28T13:33:33.004153+0000","last_unstale":"2024-04-28T13:36:05.315478+0000","last_undegraded":"2024-04-28T13:36:05.315478+0000","last_fullsized":"2024-04-28T13:36:05.315478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'5","last_scrub_stamp":"2024-04-28T13:36:05.315270+0000","last_deep_scrub":"47'5","last_deep_scrub_stamp":"2024-04-28T13:36:05.315270+0000","last_clean_scrub_stamp":"2024-04-28T13:36:05.315270+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:22:43.273420+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:09.212200+0000","last_change":"2024-04-28T13:36:09.212253+0000","last_active":"2024-04-28T13:36:09.212200+0000","last_peered":"2024-04-28T13:36:09.212200+0000","last_clean":"2024-04-28T13:36:09.212200+0000","last_became_active":"2024-04-28T13:33:34.006561+0000","last_became_peered":"2024-04-28T13:33:34.006561+0000","last_unstale":"2024-04-28T13:36:09.212200+0000","last_undegraded":"2024-04-28T13:36:09.212200+0000","last_fullsized":"2024-04-28T13:36:09.212200+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:09.211907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:09.211907+0000","last_clean_scrub_stamp":"2024-04-28T13:36:09.211907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:25.324983+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'14","reported_seq":42,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:57.846730+0000","last_change":"2024-04-28T13:35:57.846800+0000","last_active":"2024-04-28T13:35:57.846730+0000","last_peered":"2024-04-28T13:35:57.846730+0000","last_clean":"2024-04-28T13:35:57.846730+0000","last_became_active":"2024-04-28T13:33:32.991060+0000","last_became_peered":"2024-04-28T13:33:32.991060+0000","last_unstale":"2024-04-28T13:35:57.846730+0000","last_undegraded":"2024-04-28T13:35:57.846730+0000","last_fullsized":"2024-04-28T13:35:57.846730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'14","last_scrub_stamp":"2024-04-28T13:35:57.846546+0000","last_deep_scrub":"47'14","last_deep_scrub_stamp":"2024-04-28T13:35:57.846546+0000","last_clean_scrub_stamp":"2024-04-28T13:35:57.846546+0000","objects_scrubbed":4,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:18:02.788950+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3397,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:01.422045+0000","last_change":"2024-04-28T13:36:01.422105+0000","last_active":"2024-04-28T13:36:01.422045+0000","last_peered":"2024-04-28T13:36:01.422045+0000","last_clean":"2024-04-28T13:36:01.422045+0000","last_became_active":"2024-04-28T13:33:33.996260+0000","last_became_peered":"2024-04-28T13:33:33.996260+0000","last_unstale":"2024-04-28T13:36:01.422045+0000","last_undegraded":"2024-04-28T13:36:01.422045+0000","last_fullsized":"2024-04-28T13:36:01.422045+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:01.421811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:01.421811+0000","last_clean_scrub_stamp":"2024-04-28T13:36:01.421811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:20:25.140895+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'4","reported_seq":32,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:57.304954+0000","last_change":"2024-04-28T13:35:57.304981+0000","last_active":"2024-04-28T13:35:57.304954+0000","last_peered":"2024-04-28T13:35:57.304954+0000","last_clean":"2024-04-28T13:35:57.304954+0000","last_became_active":"2024-04-28T13:33:32.990718+0000","last_became_peered":"2024-04-28T13:33:32.990718+0000","last_unstale":"2024-04-28T13:35:57.304954+0000","last_undegraded":"2024-04-28T13:35:57.304954+0000","last_fullsized":"2024-04-28T13:35:57.304954+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'4","last_scrub_stamp":"2024-04-28T13:35:57.304887+0000","last_deep_scrub":"47'4","last_deep_scrub_stamp":"2024-04-28T13:35:57.304887+0000","last_clean_scrub_stamp":"2024-04-28T13:35:57.304887+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:43:39.335087+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:55.257532+0000","last_change":"2024-04-28T13:35:55.257532+0000","last_active":"2024-04-28T13:35:55.257532+0000","last_peered":"2024-04-28T13:35:55.257532+0000","last_clean":"2024-04-28T13:35:55.257532+0000","last_became_active":"2024-04-28T13:33:34.003599+0000","last_became_peered":"2024-04-28T13:33:34.003599+0000","last_unstale":"2024-04-28T13:35:55.257532+0000","last_undegraded":"2024-04-28T13:35:55.257532+0000","last_fullsized":"2024-04-28T13:35:55.257532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:35:55.257326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:35:55.257326+0000","last_clean_scrub_stamp":"2024-04-28T13:35:55.257326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:30:58.763462+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:01.319153+0000","last_change":"2024-04-28T13:36:01.319230+0000","last_active":"2024-04-28T13:36:01.319153+0000","last_peered":"2024-04-28T13:36:01.319153+0000","last_clean":"2024-04-28T13:36:01.319153+0000","last_became_active":"2024-04-28T13:33:32.998867+0000","last_became_peered":"2024-04-28T13:33:32.998867+0000","last_unstale":"2024-04-28T13:36:01.319153+0000","last_undegraded":"2024-04-28T13:36:01.319153+0000","last_fullsized":"2024-04-28T13:36:01.319153+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:01.318886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:01.318886+0000","last_clean_scrub_stamp":"2024-04-28T13:36:01.318886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:45:14.548789+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:07.391566+0000","last_change":"2024-04-28T13:36:07.391660+0000","last_active":"2024-04-28T13:36:07.391566+0000","last_peered":"2024-04-28T13:36:07.391566+0000","last_clean":"2024-04-28T13:36:07.391566+0000","last_became_active":"2024-04-28T13:33:34.002579+0000","last_became_peered":"2024-04-28T13:33:34.002579+0000","last_unstale":"2024-04-28T13:36:07.391566+0000","last_undegraded":"2024-04-28T13:36:07.391566+0000","last_fullsized":"2024-04-28T13:36:07.391566+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:07.391376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:07.391376+0000","last_clean_scrub_stamp":"2024-04-28T13:36:07.391376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:00:44.564798+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:58.339660+0000","last_change":"2024-04-28T13:35:58.339691+0000","last_active":"2024-04-28T13:35:58.339660+0000","last_peered":"2024-04-28T13:35:58.339660+0000","last_clean":"2024-04-28T13:35:58.339660+0000","last_became_active":"2024-04-28T13:33:32.991369+0000","last_became_peered":"2024-04-28T13:33:32.991369+0000","last_unstale":"2024-04-28T13:35:58.339660+0000","last_undegraded":"2024-04-28T13:35:58.339660+0000","last_fullsized":"2024-04-28T13:35:58.339660+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:35:58.339532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:35:58.339532+0000","last_clean_scrub_stamp":"2024-04-28T13:35:58.339532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:06:13.740252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:11.269277+0000","last_change":"2024-04-28T13:36:11.269334+0000","last_active":"2024-04-28T13:36:11.269277+0000","last_peered":"2024-04-28T13:36:11.269277+0000","last_clean":"2024-04-28T13:36:11.269277+0000","last_became_active":"2024-04-28T13:33:34.010077+0000","last_became_peered":"2024-04-28T13:33:34.010077+0000","last_unstale":"2024-04-28T13:36:11.269277+0000","last_undegraded":"2024-04-28T13:36:11.269277+0000","last_fullsized":"2024-04-28T13:36:11.269277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:11.269071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:11.269071+0000","last_clean_scrub_stamp":"2024-04-28T13:36:11.269071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:09:49.458174+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'12","reported_seq":40,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:56.229019+0000","last_change":"2024-04-28T13:35:56.229060+0000","last_active":"2024-04-28T13:35:56.229019+0000","last_peered":"2024-04-28T13:35:56.229019+0000","last_clean":"2024-04-28T13:35:56.229019+0000","last_became_active":"2024-04-28T13:33:32.988579+0000","last_became_peered":"2024-04-28T13:33:32.988579+0000","last_unstale":"2024-04-28T13:35:56.229019+0000","last_undegraded":"2024-04-28T13:35:56.229019+0000","last_fullsized":"2024-04-28T13:35:56.229019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'12","last_scrub_stamp":"2024-04-28T13:35:56.228916+0000","last_deep_scrub":"47'12","last_deep_scrub_stamp":"2024-04-28T13:35:56.228916+0000","last_clean_scrub_stamp":"2024-04-28T13:35:56.228916+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:33:55.123402+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:03.807821+0000","last_change":"2024-04-28T13:36:03.807873+0000","last_active":"2024-04-28T13:36:03.807821+0000","last_peered":"2024-04-28T13:36:03.807821+0000","last_clean":"2024-04-28T13:36:03.807821+0000","last_became_active":"2024-04-28T13:33:34.012112+0000","last_became_peered":"2024-04-28T13:33:34.012112+0000","last_unstale":"2024-04-28T13:36:03.807821+0000","last_undegraded":"2024-04-28T13:36:03.807821+0000","last_fullsized":"2024-04-28T13:36:03.807821+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:03.807596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:03.807596+0000","last_clean_scrub_stamp":"2024-04-28T13:36:03.807596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:45:48.742161+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'2","reported_seq":30,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:56.811866+0000","last_change":"2024-04-28T13:35:56.811938+0000","last_active":"2024-04-28T13:35:56.811866+0000","last_peered":"2024-04-28T13:35:56.811866+0000","last_clean":"2024-04-28T13:35:56.811866+0000","last_became_active":"2024-04-28T13:33:32.990659+0000","last_became_peered":"2024-04-28T13:33:32.990659+0000","last_unstale":"2024-04-28T13:35:56.811866+0000","last_undegraded":"2024-04-28T13:35:56.811866+0000","last_fullsized":"2024-04-28T13:35:56.811866+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'2","last_scrub_stamp":"2024-04-28T13:35:56.811676+0000","last_deep_scrub":"47'2","last_deep_scrub_stamp":"2024-04-28T13:35:56.811676+0000","last_clean_scrub_stamp":"2024-04-28T13:35:56.811676+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:06:06.273055+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:06.333141+0000","last_change":"2024-04-28T13:36:06.333247+0000","last_active":"2024-04-28T13:36:06.333141+0000","last_peered":"2024-04-28T13:36:06.333141+0000","last_clean":"2024-04-28T13:36:06.333141+0000","last_became_active":"2024-04-28T13:33:34.005364+0000","last_became_peered":"2024-04-28T13:33:34.005364+0000","last_unstale":"2024-04-28T13:36:06.333141+0000","last_undegraded":"2024-04-28T13:36:06.333141+0000","last_fullsized":"2024-04-28T13:36:06.333141+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:06.332906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:06.332906+0000","last_clean_scrub_stamp":"2024-04-28T13:36:06.332906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:22.445981+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'3","reported_seq":31,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:00.660732+0000","last_change":"2024-04-28T13:36:00.660799+0000","last_active":"2024-04-28T13:36:00.660732+0000","last_peered":"2024-04-28T13:36:00.660732+0000","last_clean":"2024-04-28T13:36:00.660732+0000","last_became_active":"2024-04-28T13:33:32.994798+0000","last_became_peered":"2024-04-28T13:33:32.994798+0000","last_unstale":"2024-04-28T13:36:00.660732+0000","last_undegraded":"2024-04-28T13:36:00.660732+0000","last_fullsized":"2024-04-28T13:36:00.660732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'3","last_scrub_stamp":"2024-04-28T13:36:00.660533+0000","last_deep_scrub":"47'3","last_deep_scrub_stamp":"2024-04-28T13:36:00.660533+0000","last_clean_scrub_stamp":"2024-04-28T13:36:00.660533+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:16:04.497423+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:05.460531+0000","last_change":"2024-04-28T13:36:05.460593+0000","last_active":"2024-04-28T13:36:05.460531+0000","last_peered":"2024-04-28T13:36:05.460531+0000","last_clean":"2024-04-28T13:36:05.460531+0000","last_became_active":"2024-04-28T13:33:34.001893+0000","last_became_peered":"2024-04-28T13:33:34.001893+0000","last_unstale":"2024-04-28T13:36:05.460531+0000","last_undegraded":"2024-04-28T13:36:05.460531+0000","last_fullsized":"2024-04-28T13:36:05.460531+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:05.460328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:05.460328+0000","last_clean_scrub_stamp":"2024-04-28T13:36:05.460328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:51:50.684453+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'2","reported_seq":30,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:59.877864+0000","last_change":"2024-04-28T13:35:59.877936+0000","last_active":"2024-04-28T13:35:59.877864+0000","last_peered":"2024-04-28T13:35:59.877864+0000","last_clean":"2024-04-28T13:35:59.877864+0000","last_became_active":"2024-04-28T13:33:32.990745+0000","last_became_peered":"2024-04-28T13:33:32.990745+0000","last_unstale":"2024-04-28T13:35:59.877864+0000","last_undegraded":"2024-04-28T13:35:59.877864+0000","last_fullsized":"2024-04-28T13:35:59.877864+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'2","last_scrub_stamp":"2024-04-28T13:35:59.877648+0000","last_deep_scrub":"47'2","last_deep_scrub_stamp":"2024-04-28T13:35:59.877648+0000","last_clean_scrub_stamp":"2024-04-28T13:35:59.877648+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:59:05.459221+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:07.492903+0000","last_change":"2024-04-28T13:36:07.492959+0000","last_active":"2024-04-28T13:36:07.492903+0000","last_peered":"2024-04-28T13:36:07.492903+0000","last_clean":"2024-04-28T13:36:07.492903+0000","last_became_active":"2024-04-28T13:33:34.002228+0000","last_became_peered":"2024-04-28T13:33:34.002228+0000","last_unstale":"2024-04-28T13:36:07.492903+0000","last_undegraded":"2024-04-28T13:36:07.492903+0000","last_fullsized":"2024-04-28T13:36:07.492903+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:07.492705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:07.492705+0000","last_clean_scrub_stamp":"2024-04-28T13:36:07.492705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:16:11.329595+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:03.418455+0000","last_change":"2024-04-28T13:36:03.418506+0000","last_active":"2024-04-28T13:36:03.418455+0000","last_peered":"2024-04-28T13:36:03.418455+0000","last_clean":"2024-04-28T13:36:03.418455+0000","last_became_active":"2024-04-28T13:33:33.003389+0000","last_became_peered":"2024-04-28T13:33:33.003389+0000","last_unstale":"2024-04-28T13:36:03.418455+0000","last_undegraded":"2024-04-28T13:36:03.418455+0000","last_fullsized":"2024-04-28T13:36:03.418455+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:03.418257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:03.418257+0000","last_clean_scrub_stamp":"2024-04-28T13:36:03.418257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:11:09.759609+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:08.393970+0000","last_change":"2024-04-28T13:36:08.394028+0000","last_active":"2024-04-28T13:36:08.393970+0000","last_peered":"2024-04-28T13:36:08.393970+0000","last_clean":"2024-04-28T13:36:08.393970+0000","last_became_active":"2024-04-28T13:33:34.006282+0000","last_became_peered":"2024-04-28T13:33:34.006282+0000","last_unstale":"2024-04-28T13:36:08.393970+0000","last_undegraded":"2024-04-28T13:36:08.393970+0000","last_fullsized":"2024-04-28T13:36:08.393970+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:08.393780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:08.393780+0000","last_clean_scrub_stamp":"2024-04-28T13:36:08.393780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:50:46.634128+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'3","reported_seq":31,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:01.363667+0000","last_change":"2024-04-28T13:36:01.363779+0000","last_active":"2024-04-28T13:36:01.363667+0000","last_peered":"2024-04-28T13:36:01.363667+0000","last_clean":"2024-04-28T13:36:01.363667+0000","last_became_active":"2024-04-28T13:33:32.998923+0000","last_became_peered":"2024-04-28T13:33:32.998923+0000","last_unstale":"2024-04-28T13:36:01.363667+0000","last_undegraded":"2024-04-28T13:36:01.363667+0000","last_fullsized":"2024-04-28T13:36:01.363667+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'3","last_scrub_stamp":"2024-04-28T13:36:01.363425+0000","last_deep_scrub":"47'3","last_deep_scrub_stamp":"2024-04-28T13:36:01.363425+0000","last_clean_scrub_stamp":"2024-04-28T13:36:01.363425+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:18:20.431331+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:01.266785+0000","last_change":"2024-04-28T13:36:01.266839+0000","last_active":"2024-04-28T13:36:01.266785+0000","last_peered":"2024-04-28T13:36:01.266785+0000","last_clean":"2024-04-28T13:36:01.266785+0000","last_became_active":"2024-04-28T13:33:34.011271+0000","last_became_peered":"2024-04-28T13:33:34.011271+0000","last_unstale":"2024-04-28T13:36:01.266785+0000","last_undegraded":"2024-04-28T13:36:01.266785+0000","last_fullsized":"2024-04-28T13:36:01.266785+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:01.266588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:01.266588+0000","last_clean_scrub_stamp":"2024-04-28T13:36:01.266588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:14:43.368398+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:01.911761+0000","last_change":"2024-04-28T13:36:01.911815+0000","last_active":"2024-04-28T13:36:01.911761+0000","last_peered":"2024-04-28T13:36:01.911761+0000","last_clean":"2024-04-28T13:36:01.911761+0000","last_became_active":"2024-04-28T13:33:33.000335+0000","last_became_peered":"2024-04-28T13:33:33.000335+0000","last_unstale":"2024-04-28T13:36:01.911761+0000","last_undegraded":"2024-04-28T13:36:01.911761+0000","last_fullsized":"2024-04-28T13:36:01.911761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:01.911562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:01.911562+0000","last_clean_scrub_stamp":"2024-04-28T13:36:01.911562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:11:36.667884+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:59.840650+0000","last_change":"2024-04-28T13:35:59.840690+0000","last_active":"2024-04-28T13:35:59.840650+0000","last_peered":"2024-04-28T13:35:59.840650+0000","last_clean":"2024-04-28T13:35:59.840650+0000","last_became_active":"2024-04-28T13:33:33.999839+0000","last_became_peered":"2024-04-28T13:33:33.999839+0000","last_unstale":"2024-04-28T13:35:59.840650+0000","last_undegraded":"2024-04-28T13:35:59.840650+0000","last_fullsized":"2024-04-28T13:35:59.840650+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:35:59.840545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:35:59.840545+0000","last_clean_scrub_stamp":"2024-04-28T13:35:59.840545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:05:17.469948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:59.281191+0000","last_change":"2024-04-28T13:35:59.281230+0000","last_active":"2024-04-28T13:35:59.281191+0000","last_peered":"2024-04-28T13:35:59.281191+0000","last_clean":"2024-04-28T13:35:59.281191+0000","last_became_active":"2024-04-28T13:33:32.991800+0000","last_became_peered":"2024-04-28T13:33:32.991800+0000","last_unstale":"2024-04-28T13:35:59.281191+0000","last_undegraded":"2024-04-28T13:35:59.281191+0000","last_fullsized":"2024-04-28T13:35:59.281191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:35:59.281104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:35:59.281104+0000","last_clean_scrub_stamp":"2024-04-28T13:35:59.281104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:43:41.311305+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:10.326780+0000","last_change":"2024-04-28T13:36:10.326910+0000","last_active":"2024-04-28T13:36:10.326780+0000","last_peered":"2024-04-28T13:36:10.326780+0000","last_clean":"2024-04-28T13:36:10.326780+0000","last_became_active":"2024-04-28T13:33:34.010896+0000","last_became_peered":"2024-04-28T13:33:34.010896+0000","last_unstale":"2024-04-28T13:36:10.326780+0000","last_undegraded":"2024-04-28T13:36:10.326780+0000","last_fullsized":"2024-04-28T13:36:10.326780+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:10.326533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:10.326533+0000","last_clean_scrub_stamp":"2024-04-28T13:36:10.326533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:48:24.388944+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:04.318922+0000","last_change":"2024-04-28T13:36:04.318975+0000","last_active":"2024-04-28T13:36:04.318922+0000","last_peered":"2024-04-28T13:36:04.318922+0000","last_clean":"2024-04-28T13:36:04.318922+0000","last_became_active":"2024-04-28T13:33:33.000095+0000","last_became_peered":"2024-04-28T13:33:33.000095+0000","last_unstale":"2024-04-28T13:36:04.318922+0000","last_undegraded":"2024-04-28T13:36:04.318922+0000","last_fullsized":"2024-04-28T13:36:04.318922+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:04.318730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:04.318730+0000","last_clean_scrub_stamp":"2024-04-28T13:36:04.318730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:25:44.451934+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:09.513791+0000","last_change":"2024-04-28T13:36:09.513843+0000","last_active":"2024-04-28T13:36:09.513791+0000","last_peered":"2024-04-28T13:36:09.513791+0000","last_clean":"2024-04-28T13:36:09.513791+0000","last_became_active":"2024-04-28T13:33:34.007425+0000","last_became_peered":"2024-04-28T13:33:34.007425+0000","last_unstale":"2024-04-28T13:36:09.513791+0000","last_undegraded":"2024-04-28T13:36:09.513791+0000","last_fullsized":"2024-04-28T13:36:09.513791+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:09.513601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:09.513601+0000","last_clean_scrub_stamp":"2024-04-28T13:36:09.513601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:38:51.178119+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"47'3","reported_seq":31,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:00.334363+0000","last_change":"2024-04-28T13:36:00.334423+0000","last_active":"2024-04-28T13:36:00.334363+0000","last_peered":"2024-04-28T13:36:00.334363+0000","last_clean":"2024-04-28T13:36:00.334363+0000","last_became_active":"2024-04-28T13:33:32.998328+0000","last_became_peered":"2024-04-28T13:33:32.998328+0000","last_unstale":"2024-04-28T13:36:00.334363+0000","last_undegraded":"2024-04-28T13:36:00.334363+0000","last_fullsized":"2024-04-28T13:36:00.334363+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'3","last_scrub_stamp":"2024-04-28T13:36:00.334174+0000","last_deep_scrub":"47'3","last_deep_scrub_stamp":"2024-04-28T13:36:00.334174+0000","last_clean_scrub_stamp":"2024-04-28T13:36:00.334174+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:16:58.021877+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:01.316876+0000","last_change":"2024-04-28T13:36:01.316936+0000","last_active":"2024-04-28T13:36:01.316876+0000","last_peered":"2024-04-28T13:36:01.316876+0000","last_clean":"2024-04-28T13:36:01.316876+0000","last_became_active":"2024-04-28T13:33:33.997834+0000","last_became_peered":"2024-04-28T13:33:33.997834+0000","last_unstale":"2024-04-28T13:36:01.316876+0000","last_undegraded":"2024-04-28T13:36:01.316876+0000","last_fullsized":"2024-04-28T13:36:01.316876+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:01.316674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:01.316674+0000","last_clean_scrub_stamp":"2024-04-28T13:36:01.316674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:07:13.753226+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:06.268071+0000","last_change":"2024-04-28T13:36:06.268126+0000","last_active":"2024-04-28T13:36:06.268071+0000","last_peered":"2024-04-28T13:36:06.268071+0000","last_clean":"2024-04-28T13:36:06.268071+0000","last_became_active":"2024-04-28T13:33:33.005179+0000","last_became_peered":"2024-04-28T13:33:33.005179+0000","last_unstale":"2024-04-28T13:36:06.268071+0000","last_undegraded":"2024-04-28T13:36:06.268071+0000","last_fullsized":"2024-04-28T13:36:06.268071+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:06.267831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:06.267831+0000","last_clean_scrub_stamp":"2024-04-28T13:36:06.267831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:22:24.219938+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:12.294797+0000","last_change":"2024-04-28T13:36:12.294854+0000","last_active":"2024-04-28T13:36:12.294797+0000","last_peered":"2024-04-28T13:36:12.294797+0000","last_clean":"2024-04-28T13:36:12.294797+0000","last_became_active":"2024-04-28T13:33:34.011422+0000","last_became_peered":"2024-04-28T13:33:34.011422+0000","last_unstale":"2024-04-28T13:36:12.294797+0000","last_undegraded":"2024-04-28T13:36:12.294797+0000","last_fullsized":"2024-04-28T13:36:12.294797+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:12.294593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:12.294593+0000","last_clean_scrub_stamp":"2024-04-28T13:36:12.294593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:50:27.462517+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:56.404713+0000","last_change":"2024-04-28T13:35:56.404736+0000","last_active":"2024-04-28T13:35:56.404713+0000","last_peered":"2024-04-28T13:35:56.404713+0000","last_clean":"2024-04-28T13:35:56.404713+0000","last_became_active":"2024-04-28T13:33:32.996934+0000","last_became_peered":"2024-04-28T13:33:32.996934+0000","last_unstale":"2024-04-28T13:35:56.404713+0000","last_undegraded":"2024-04-28T13:35:56.404713+0000","last_fullsized":"2024-04-28T13:35:56.404713+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:35:56.404577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:35:56.404577+0000","last_clean_scrub_stamp":"2024-04-28T13:35:56.404577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:34:36.461778+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:12.485110+0000","last_change":"2024-04-28T13:36:12.485163+0000","last_active":"2024-04-28T13:36:12.485110+0000","last_peered":"2024-04-28T13:36:12.485110+0000","last_clean":"2024-04-28T13:36:12.485110+0000","last_became_active":"2024-04-28T13:33:34.018922+0000","last_became_peered":"2024-04-28T13:33:34.018922+0000","last_unstale":"2024-04-28T13:36:12.485110+0000","last_undegraded":"2024-04-28T13:36:12.485110+0000","last_fullsized":"2024-04-28T13:36:12.485110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:12.484909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:12.484909+0000","last_clean_scrub_stamp":"2024-04-28T13:36:12.484909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:46:08.723716+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:02.298491+0000","last_change":"2024-04-28T13:36:02.298547+0000","last_active":"2024-04-28T13:36:02.298491+0000","last_peered":"2024-04-28T13:36:02.298491+0000","last_clean":"2024-04-28T13:36:02.298491+0000","last_became_active":"2024-04-28T13:33:33.000049+0000","last_became_peered":"2024-04-28T13:33:33.000049+0000","last_unstale":"2024-04-28T13:36:02.298491+0000","last_undegraded":"2024-04-28T13:36:02.298491+0000","last_fullsized":"2024-04-28T13:36:02.298491+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:02.298287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:02.298287+0000","last_clean_scrub_stamp":"2024-04-28T13:36:02.298287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:38:14.032738+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:06.427043+0000","last_change":"2024-04-28T13:36:06.427097+0000","last_active":"2024-04-28T13:36:06.427043+0000","last_peered":"2024-04-28T13:36:06.427043+0000","last_clean":"2024-04-28T13:36:06.427043+0000","last_became_active":"2024-04-28T13:33:34.003940+0000","last_became_peered":"2024-04-28T13:33:34.003940+0000","last_unstale":"2024-04-28T13:36:06.427043+0000","last_undegraded":"2024-04-28T13:36:06.427043+0000","last_fullsized":"2024-04-28T13:36:06.427043+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:06.426854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:06.426854+0000","last_clean_scrub_stamp":"2024-04-28T13:36:06.426854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:50:21.594778+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:58.674154+0000","last_change":"2024-04-28T13:35:58.674208+0000","last_active":"2024-04-28T13:35:58.674154+0000","last_peered":"2024-04-28T13:35:58.674154+0000","last_clean":"2024-04-28T13:35:58.674154+0000","last_became_active":"2024-04-28T13:33:32.991435+0000","last_became_peered":"2024-04-28T13:33:32.991435+0000","last_unstale":"2024-04-28T13:35:58.674154+0000","last_undegraded":"2024-04-28T13:35:58.674154+0000","last_fullsized":"2024-04-28T13:35:58.674154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:35:58.673911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:35:58.673911+0000","last_clean_scrub_stamp":"2024-04-28T13:35:58.673911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:41:23.968855+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:04.961575+0000","last_change":"2024-04-28T13:36:04.961681+0000","last_active":"2024-04-28T13:36:04.961575+0000","last_peered":"2024-04-28T13:36:04.961575+0000","last_clean":"2024-04-28T13:36:04.961575+0000","last_became_active":"2024-04-28T13:33:34.002092+0000","last_became_peered":"2024-04-28T13:33:34.002092+0000","last_unstale":"2024-04-28T13:36:04.961575+0000","last_undegraded":"2024-04-28T13:36:04.961575+0000","last_fullsized":"2024-04-28T13:36:04.961575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:04.961337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:04.961337+0000","last_clean_scrub_stamp":"2024-04-28T13:36:04.961337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:38:46.625854+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:58.369919+0000","last_change":"2024-04-28T13:35:58.369973+0000","last_active":"2024-04-28T13:35:58.369919+0000","last_peered":"2024-04-28T13:35:58.369919+0000","last_clean":"2024-04-28T13:35:58.369919+0000","last_became_active":"2024-04-28T13:33:33.003374+0000","last_became_peered":"2024-04-28T13:33:33.003374+0000","last_unstale":"2024-04-28T13:35:58.369919+0000","last_undegraded":"2024-04-28T13:35:58.369919+0000","last_fullsized":"2024-04-28T13:35:58.369919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:35:58.369731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:35:58.369731+0000","last_clean_scrub_stamp":"2024-04-28T13:35:58.369731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:45:43.304296+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:05.672739+0000","last_change":"2024-04-28T13:36:05.672792+0000","last_active":"2024-04-28T13:36:05.672739+0000","last_peered":"2024-04-28T13:36:05.672739+0000","last_clean":"2024-04-28T13:36:05.672739+0000","last_became_active":"2024-04-28T13:33:33.999963+0000","last_became_peered":"2024-04-28T13:33:33.999963+0000","last_unstale":"2024-04-28T13:36:05.672739+0000","last_undegraded":"2024-04-28T13:36:05.672739+0000","last_fullsized":"2024-04-28T13:36:05.672739+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:05.672532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:05.672532+0000","last_clean_scrub_stamp":"2024-04-28T13:36:05.672532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:22:23.624639+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:58.270689+0000","last_change":"2024-04-28T13:35:58.270717+0000","last_active":"2024-04-28T13:35:58.270689+0000","last_peered":"2024-04-28T13:35:58.270689+0000","last_clean":"2024-04-28T13:35:58.270689+0000","last_became_active":"2024-04-28T13:33:32.990726+0000","last_became_peered":"2024-04-28T13:33:32.990726+0000","last_unstale":"2024-04-28T13:35:58.270689+0000","last_undegraded":"2024-04-28T13:35:58.270689+0000","last_fullsized":"2024-04-28T13:35:58.270689+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:35:58.270564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:35:58.270564+0000","last_clean_scrub_stamp":"2024-04-28T13:35:58.270564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:35:29.369202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:05.324534+0000","last_change":"2024-04-28T13:36:05.324591+0000","last_active":"2024-04-28T13:36:05.324534+0000","last_peered":"2024-04-28T13:36:05.324534+0000","last_clean":"2024-04-28T13:36:05.324534+0000","last_became_active":"2024-04-28T13:33:34.004776+0000","last_became_peered":"2024-04-28T13:33:34.004776+0000","last_unstale":"2024-04-28T13:36:05.324534+0000","last_undegraded":"2024-04-28T13:36:05.324534+0000","last_fullsized":"2024-04-28T13:36:05.324534+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T13:36:05.324317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T13:36:05.324317+0000","last_clean_scrub_stamp":"2024-04-28T13:36:05.324317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:59:07.803355+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'3","reported_seq":31,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:35:58.866650+0000","last_change":"2024-04-28T13:35:58.866780+0000","last_active":"2024-04-28T13:35:58.866650+0000","last_peered":"2024-04-28T13:35:58.866650+0000","last_clean":"2024-04-28T13:35:58.866650+0000","last_became_active":"2024-04-28T13:33:32.991149+0000","last_became_peered":"2024-04-28T13:33:32.991149+0000","last_unstale":"2024-04-28T13:35:58.866650+0000","last_undegraded":"2024-04-28T13:35:58.866650+0000","last_fullsized":"2024-04-28T13:35:58.866650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'3","last_scrub_stamp":"2024-04-28T13:35:58.866396+0000","last_deep_scrub":"47'3","last_deep_scrub_stamp":"2024-04-28T13:35:58.866396+0000","last_clean_scrub_stamp":"2024-04-28T13:35:58.866396+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:18:52.339711+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"47'119","reported_seq":231,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-28T13:36:00.277576+0000","last_change":"2024-04-28T13:36:00.277653+0000","last_active":"2024-04-28T13:36:00.277576+0000","last_peered":"2024-04-28T13:36:00.277576+0000","last_clean":"2024-04-28T13:36:00.277576+0000","last_became_active":"2024-04-28T13:31:11.828959+0000","last_became_peered":"2024-04-28T13:31:11.828959+0000","last_unstale":"2024-04-28T13:36:00.277576+0000","last_undegraded":"2024-04-28T13:36:00.277576+0000","last_fullsized":"2024-04-28T13:36:00.277576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'119","last_scrub_stamp":"2024-04-28T13:36:00.277396+0000","last_deep_scrub":"47'119","last_deep_scrub_stamp":"2024-04-28T13:36:00.277396+0000","last_clean_scrub_stamp":"2024-04-28T13:36:00.277396+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:33:59.171882+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":4195,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1150976,"data_stored":818777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":561152,"data_stored":228991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":557056,"data_stored":228389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":565248,"data_stored":231716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":565248,"data_stored":228957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":569344,"data_stored":231738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":557056,"data_stored":228389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1150976,"data_stored":818721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T13:36:20.584 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-28T13:36:20.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.818+0000 7efe4e400640 1 Processor -- start 2024-04-28T13:36:20.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.818+0000 7efe4e400640 1 -- start start 2024-04-28T13:36:20.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.820+0000 7efe4e400640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efe480d0300 0x7efe480b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:20.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.820+0000 7efe4e400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480b3e50 0x7efe480720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:20.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.820+0000 7efe4e400640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe480b46a0 con 0x7efe4806f340 2024-04-28T13:36:20.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.820+0000 7efe4e400640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7efe480b5db0 con 0x7efe480d0300 2024-04-28T13:36:20.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.820+0000 7efe4e400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efe480b6090 con 0x7efe480b3e50 2024-04-28T13:36:20.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.820+0000 7efe3ea00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efe480d0300 0x7efe480b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:20.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.821+0000 7efe3ea00640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efe480d0300 0x7efe480b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:44004/0 (socket says 172.21.15.196:44004) 2024-04-28T13:36:20.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.821+0000 7efe3ea00640 1 -- 172.21.15.196:0/4287339039 learned_addr learned my addr 172.21.15.196:0/4287339039 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:36:20.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.821+0000 7efe47e00640 1 --2- 172.21.15.196:0/4287339039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480b3e50 0x7efe480720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:20.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.821+0000 7efe3ea00640 1 -- 172.21.15.196:0/4287339039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480b3e50 msgr2=0x7efe480720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:36:20.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.821+0000 7efe3ea00640 1 --2- 172.21.15.196:0/4287339039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480b3e50 0x7efe480720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.821+0000 7efe3ea00640 1 -- 172.21.15.196:0/4287339039 >> v1:172.21.15.203:6789/0 conn(0x7efe4806f340 legacy=0x7efe4806f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:20.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.821+0000 7efe3ea00640 1 -- 172.21.15.196:0/4287339039 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe480b6370 con 0x7efe480d0300 2024-04-28T13:36:20.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.822+0000 7efe3ea00640 1 --2- 172.21.15.196:0/4287339039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe480d0300 0x7efe480b6b70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7efe30009870 tx=0x7efe3002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8905622cdccdc01a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:36:20.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.823+0000 7efe46a00640 1 -- 172.21.15.196:0/4287339039 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30037070 con 0x7efe480d0300 2024-04-28T13:36:20.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.823+0000 7efe46a00640 1 -- 172.21.15.196:0/4287339039 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe30032440 con 0x7efe480d0300 2024-04-28T13:36:20.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.823+0000 7efe46a00640 1 -- 172.21.15.196:0/4287339039 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30032810 con 0x7efe480d0300 2024-04-28T13:36:20.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.823+0000 7efe4e400640 1 -- 172.21.15.196:0/4287339039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe480d0300 msgr2=0x7efe480b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:20.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.823+0000 7efe4e400640 1 --2- 172.21.15.196:0/4287339039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe480d0300 0x7efe480b6b70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7efe30009870 tx=0x7efe3002a130 comp rx=0 tx=0).stop 2024-04-28T13:36:20.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.823+0000 7efe4e400640 1 -- 172.21.15.196:0/4287339039 shutdown_connections 2024-04-28T13:36:20.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.823+0000 7efe4e400640 1 --2- 172.21.15.196:0/4287339039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480b3e50 0x7efe480720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.823+0000 7efe4e400640 1 --2- 172.21.15.196:0/4287339039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe480d0300 0x7efe480b6b70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.823+0000 7efe4e400640 1 -- 172.21.15.196:0/4287339039 >> 172.21.15.196:0/4287339039 conn(0x7efe480cd8a0 msgr2=0x7efe480cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:36:20.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.824+0000 7efe4e400640 1 -- 172.21.15.196:0/4287339039 shutdown_connections 2024-04-28T13:36:20.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.824+0000 7efe4e400640 1 -- 172.21.15.196:0/4287339039 wait complete. 2024-04-28T13:36:20.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.825+0000 7efe4e400640 1 Processor -- start 2024-04-28T13:36:20.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.825+0000 7efe4e400640 1 -- start start 2024-04-28T13:36:20.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.826+0000 7efe4e400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe4806f340 0x7efe4813c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:20.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.826+0000 7efe4e400640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480b3e50 0x7efe4815c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:20.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.826+0000 7efe4e400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe480d0300 0x7efe4813ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:20.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.826+0000 7efe4e400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe480b4460 con 0x7efe4806f340 2024-04-28T13:36:20.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.826+0000 7efe4e400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efe480b42e0 con 0x7efe480d0300 2024-04-28T13:36:20.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.826+0000 7efe4e400640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efe48142a30 con 0x7efe480b3e50 2024-04-28T13:36:20.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.827+0000 7efe47400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe4806f340 0x7efe4813c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:20.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.827+0000 7efe47400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe4806f340 0x7efe4813c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:44016/0 (socket says 172.21.15.196:44016) 2024-04-28T13:36:20.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.827+0000 7efe47400640 1 -- 172.21.15.196:0/3985032292 learned_addr learned my addr 172.21.15.196:0/3985032292 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:36:20.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.827+0000 7efe3ea00640 1 --2- 172.21.15.196:0/3985032292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480b3e50 0x7efe4815c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:20.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.827+0000 7efe47400640 1 -- 172.21.15.196:0/3985032292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480b3e50 msgr2=0x7efe4815c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:20.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.827+0000 7efe47e00640 1 --2- 172.21.15.196:0/3985032292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe480d0300 0x7efe4813ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:20.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.827+0000 7efe47400640 1 --2- 172.21.15.196:0/3985032292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480b3e50 0x7efe4815c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.827+0000 7efe47400640 1 -- 172.21.15.196:0/3985032292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe480d0300 msgr2=0x7efe4813ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:20.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.827+0000 7efe47400640 1 --2- 172.21.15.196:0/3985032292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe480d0300 0x7efe4813ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:20.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.827+0000 7efe47400640 1 -- 172.21.15.196:0/3985032292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4815d160 con 0x7efe4806f340 2024-04-28T13:36:20.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.827+0000 7efe47400640 1 --2- 172.21.15.196:0/3985032292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe4806f340 0x7efe4813c7b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7efe3800c400 tx=0x7efe3800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:36:20.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.828+0000 7efe3fe00640 1 -- 172.21.15.196:0/3985032292 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe380052d0 con 0x7efe4806f340 2024-04-28T13:36:20.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.828+0000 7efe3fe00640 1 -- 172.21.15.196:0/3985032292 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe38005490 con 0x7efe4806f340 2024-04-28T13:36:20.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.828+0000 7efe4e400640 1 -- 172.21.15.196:0/3985032292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4815d440 con 0x7efe4806f340 2024-04-28T13:36:20.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.828+0000 7efe3fe00640 1 -- 172.21.15.196:0/3985032292 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38007260 con 0x7efe4806f340 2024-04-28T13:36:20.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.828+0000 7efe4e400640 1 -- 172.21.15.196:0/3985032292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4815da00 con 0x7efe4806f340 2024-04-28T13:36:20.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.829+0000 7efe4e400640 1 -- 172.21.15.196:0/3985032292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe4815dc40 con 0x7efe4806f340 2024-04-28T13:36:20.831 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.830+0000 7efe3fe00640 1 -- 172.21.15.196:0/3985032292 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7efe38007400 con 0x7efe4806f340 2024-04-28T13:36:20.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.831+0000 7efe3fe00640 1 --2- 172.21.15.196:0/3985032292 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efe2406c980 0x7efe2406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:20.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.831+0000 7efe3fe00640 1 -- 172.21.15.196:0/3985032292 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7efe3808b270 con 0x7efe4806f340 2024-04-28T13:36:20.832 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.832+0000 7efe3ea00640 1 --2- 172.21.15.196:0/3985032292 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efe2406c980 0x7efe2406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:20.834 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.833+0000 7efe3ea00640 1 --2- 172.21.15.196:0/3985032292 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efe2406c980 0x7efe2406ee40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efe30034000 tx=0x7efe3002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:36:20.837 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:20.836+0000 7efe3fe00640 1 -- 172.21.15.196:0/3985032292 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efe38059570 con 0x7efe4806f340 2024-04-28T13:36:21.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.036+0000 7efe4e400640 1 -- 172.21.15.196:0/3985032292 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7efe480cd2d0 con 0x7efe4806f340 2024-04-28T13:36:21.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.042+0000 7efe3fe00640 1 -- 172.21.15.196:0/3985032292 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v19)=0 v19) ==== 123+0+0 (secure 0 0 0) 0x7efe38058e70 con 0x7efe4806f340 2024-04-28T13:36:21.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.044+0000 7efe3fe00640 1 -- 172.21.15.196:0/3985032292 <== mon.0 v2:172.21.15.196:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efe3805d220 con 0x7efe4806f340 2024-04-28T13:36:21.046 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.045+0000 7efe4e400640 1 -- 172.21.15.196:0/3985032292 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efe2406c980 msgr2=0x7efe2406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.045+0000 7efe4e400640 1 --2- 172.21.15.196:0/3985032292 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efe2406c980 0x7efe2406ee40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efe30034000 tx=0x7efe3002e000 comp rx=0 tx=0).stop 2024-04-28T13:36:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.045+0000 7efe4e400640 1 -- 172.21.15.196:0/3985032292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe4806f340 msgr2=0x7efe4813c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.045+0000 7efe4e400640 1 --2- 172.21.15.196:0/3985032292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe4806f340 0x7efe4813c7b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7efe3800c400 tx=0x7efe3800c8c0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.046+0000 7efe4e400640 1 -- 172.21.15.196:0/3985032292 shutdown_connections 2024-04-28T13:36:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.046+0000 7efe4e400640 1 --2- 172.21.15.196:0/3985032292 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7efe2406c980 0x7efe2406ee40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.046+0000 7efe4e400640 1 --2- 172.21.15.196:0/3985032292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe480d0300 0x7efe4813ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.046+0000 7efe4e400640 1 --2- 172.21.15.196:0/3985032292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480b3e50 0x7efe4815c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.046+0000 7efe4e400640 1 --2- 172.21.15.196:0/3985032292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe4806f340 0x7efe4813c7b0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.046+0000 7efe4e400640 1 -- 172.21.15.196:0/3985032292 >> 172.21.15.196:0/3985032292 conn(0x7efe480cd8a0 msgr2=0x7efe480b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:36:21.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.046+0000 7efe4e400640 1 -- 172.21.15.196:0/3985032292 shutdown_connections 2024-04-28T13:36:21.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.046+0000 7efe4e400640 1 -- 172.21.15.196:0/3985032292 wait complete. 2024-04-28T13:36:21.064 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-28T13:36:21.064 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:36:21.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.265+0000 7f43dd000640 1 Processor -- start 2024-04-28T13:36:21.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.265+0000 7f43dd000640 1 -- start start 2024-04-28T13:36:21.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.266+0000 7f43dd000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f43d8163bf0 0x7f43d8163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:21.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.266+0000 7f43dd000640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f43d8163570 0x7f43d80b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:21.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.266+0000 7f43dd000640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d8162da0 con 0x7f43d81629c0 2024-04-28T13:36:21.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.266+0000 7f43dd000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f43d80d3e20 con 0x7f43d8163bf0 2024-04-28T13:36:21.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.266+0000 7f43dd000640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f43d80b6190 con 0x7f43d8163570 2024-04-28T13:36:21.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.267+0000 7f43d7400640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f43d81629c0 0x7f43d80d3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:36394/0 (socket says 172.21.15.196:36394) 2024-04-28T13:36:21.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.267+0000 7f43d7400640 1 -- 172.21.15.196:0/1983659965 learned_addr learned my addr 172.21.15.196:0/1983659965 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-28T13:36:21.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.267+0000 7f43d6a00640 1 --2- 172.21.15.196:0/1983659965 >> v2:172.21.15.196:3300/0 conn(0x7f43d8163bf0 0x7f43d8163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:21.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.267+0000 7f43d6000640 1 -- 172.21.15.196:0/1983659965 <== mon.0 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3203632940 0 0) 0x7f43d8162da0 con 0x7f43d81629c0 2024-04-28T13:36:21.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.267+0000 7f43d6000640 1 -- 172.21.15.196:0/1983659965 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43bc003520 con 0x7f43d81629c0 2024-04-28T13:36:21.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.267+0000 7f43d7e00640 1 --2- 172.21.15.196:0/1983659965 >> v2:172.21.15.203:3300/0 conn(0x7f43d8163570 0x7f43d80b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:21.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.267+0000 7f43d6a00640 1 -- 172.21.15.196:0/1983659965 >> v2:172.21.15.203:3300/0 conn(0x7f43d8163570 msgr2=0x7f43d80b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T13:36:21.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.267+0000 7f43d6a00640 1 --2- 172.21.15.196:0/1983659965 >> v2:172.21.15.203:3300/0 conn(0x7f43d8163570 0x7f43d80b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.267+0000 7f43d6a00640 1 -- 172.21.15.196:0/1983659965 >> v1:172.21.15.196:6789/0 conn(0x7f43d81629c0 legacy=0x7f43d80d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:21.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.267+0000 7f43d6a00640 1 -- 172.21.15.196:0/1983659965 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d80b6470 con 0x7f43d8163bf0 2024-04-28T13:36:21.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.268+0000 7f43d6a00640 1 --2- 172.21.15.196:0/1983659965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d8163bf0 0x7f43d8163030 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f43d808eec0 tx=0x7f43c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a159f2b3ab34b515 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:36:21.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.269+0000 7f43d6000640 1 -- 172.21.15.196:0/1983659965 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4037070 con 0x7f43d8163bf0 2024-04-28T13:36:21.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.269+0000 7f43d6000640 1 -- 172.21.15.196:0/1983659965 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f43c4032440 con 0x7f43d8163bf0 2024-04-28T13:36:21.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.269+0000 7f43d6000640 1 -- 172.21.15.196:0/1983659965 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c40327f0 con 0x7f43d8163bf0 2024-04-28T13:36:21.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.269+0000 7f43dd000640 1 -- 172.21.15.196:0/1983659965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d8163bf0 msgr2=0x7f43d8163030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:21.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.269+0000 7f43dd000640 1 --2- 172.21.15.196:0/1983659965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d8163bf0 0x7f43d8163030 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f43d808eec0 tx=0x7f43c402a1a0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.269+0000 7f43dd000640 1 -- 172.21.15.196:0/1983659965 shutdown_connections 2024-04-28T13:36:21.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.269+0000 7f43dd000640 1 --2- 172.21.15.196:0/1983659965 >> v2:172.21.15.203:3300/0 conn(0x7f43d8163570 0x7f43d80b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.269+0000 7f43dd000640 1 --2- 172.21.15.196:0/1983659965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d8163bf0 0x7f43d8163030 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.269+0000 7f43dd000640 1 -- 172.21.15.196:0/1983659965 >> 172.21.15.196:0/1983659965 conn(0x7f43d809e570 msgr2=0x7f43d80d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:36:21.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.269+0000 7f43dd000640 1 -- 172.21.15.196:0/1983659965 shutdown_connections 2024-04-28T13:36:21.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.269+0000 7f43dd000640 1 -- 172.21.15.196:0/1983659965 wait complete. 2024-04-28T13:36:21.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.271+0000 7f43dd000640 1 Processor -- start 2024-04-28T13:36:21.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.271+0000 7f43dd000640 1 -- start start 2024-04-28T13:36:21.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.272+0000 7f43dd000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d81629c0 0x7f43d815b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:21.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.272+0000 7f43dd000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43d8163570 0x7f43d80cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:21.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.272+0000 7f43dd000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43d8163bf0 0x7f43d80b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:21.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.272+0000 7f43dd000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43d80b1e60 con 0x7f43d81629c0 2024-04-28T13:36:21.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.272+0000 7f43dd000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f43d80b6a30 con 0x7f43d8163bf0 2024-04-28T13:36:21.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.272+0000 7f43dd000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f43d80b28c0 con 0x7f43d8163570 2024-04-28T13:36:21.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.272+0000 7f43d7400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d81629c0 0x7f43d815b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:21.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.272+0000 7f43d7400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d81629c0 0x7f43d815b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:44034/0 (socket says 172.21.15.196:44034) 2024-04-28T13:36:21.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.272+0000 7f43d7400640 1 -- 172.21.15.196:0/1385413647 learned_addr learned my addr 172.21.15.196:0/1385413647 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-28T13:36:21.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.273+0000 7f43d7e00640 1 --2- 172.21.15.196:0/1385413647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43d8163bf0 0x7f43d80b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:21.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.273+0000 7f43d7400640 1 -- 172.21.15.196:0/1385413647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43d8163570 msgr2=0x7f43d80cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:21.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.273+0000 7f43d6a00640 1 --2- 172.21.15.196:0/1385413647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43d8163570 0x7f43d80cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:21.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.273+0000 7f43d7400640 1 --2- 172.21.15.196:0/1385413647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43d8163570 0x7f43d80cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.273+0000 7f43d7400640 1 -- 172.21.15.196:0/1385413647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43d8163bf0 msgr2=0x7f43d80b9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:21.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.273+0000 7f43d7400640 1 --2- 172.21.15.196:0/1385413647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43d8163bf0 0x7f43d80b9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.273+0000 7f43d7400640 1 -- 172.21.15.196:0/1385413647 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d80b7e50 con 0x7f43d81629c0 2024-04-28T13:36:21.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.273+0000 7f43d7400640 1 --2- 172.21.15.196:0/1385413647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d81629c0 0x7f43d815b830 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c430 tx=0x7f43c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:36:21.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.273+0000 7f43cf400640 1 -- 172.21.15.196:0/1385413647 <== mon.0 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8017070 con 0x7f43d81629c0 2024-04-28T13:36:21.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.273+0000 7f43cf400640 1 -- 172.21.15.196:0/1385413647 <== mon.0 v2:172.21.15.196:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f43c8005ce0 con 0x7f43d81629c0 2024-04-28T13:36:21.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.273+0000 7f43cf400640 1 -- 172.21.15.196:0/1385413647 <== mon.0 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8007310 con 0x7f43d81629c0 2024-04-28T13:36:21.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.273+0000 7f43dd000640 1 -- 172.21.15.196:0/1385413647 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d80d2780 con 0x7f43d81629c0 2024-04-28T13:36:21.277 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.274+0000 7f43dd000640 1 -- 172.21.15.196:0/1385413647 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d80b8110 con 0x7f43d81629c0 2024-04-28T13:36:21.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.275+0000 7f43dd000640 1 -- 172.21.15.196:0/1385413647 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d806a290 con 0x7f43d81629c0 2024-04-28T13:36:21.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.276+0000 7f43cf400640 1 -- 172.21.15.196:0/1385413647 <== mon.0 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f43c801a070 con 0x7f43d81629c0 2024-04-28T13:36:21.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.277+0000 7f43cf400640 1 --2- 172.21.15.196:0/1385413647 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f43b806c980 0x7f43b806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:36:21.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.277+0000 7f43cf400640 1 -- 172.21.15.196:0/1385413647 <== mon.0 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f43c808b280 con 0x7f43d81629c0 2024-04-28T13:36:21.278 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.277+0000 7f43d6a00640 1 --2- 172.21.15.196:0/1385413647 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f43b806c980 0x7f43b806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:36:21.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.279+0000 7f43d6a00640 1 --2- 172.21.15.196:0/1385413647 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f43b806c980 0x7f43b806ee40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f43c4034000 tx=0x7f43c402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:36:21.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.282+0000 7f43cf400640 1 -- 172.21.15.196:0/1385413647 <== mon.0 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f43c80594f0 con 0x7f43d81629c0 2024-04-28T13:36:21.506 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.506+0000 7f43dd000640 1 -- 172.21.15.196:0/1385413647 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43d8162da0 con 0x7f43d81629c0 2024-04-28T13:36:21.507 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.507+0000 7f43cf400640 1 -- 172.21.15.196:0/1385413647 <== mon.0 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5193 (secure 0 0 0) 0x7f43c8058df0 con 0x7f43d81629c0 2024-04-28T13:36:21.508 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-28T13:36:21.508 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2957566398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2957566398},{"type":"v1","addr":"172.21.15.203:6836","nonce":2957566398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5766,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1101263363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1101263363},{"type":"v1","addr":"172.21.15.196:6834","nonce":1101263363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1157781293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1157781293},{"type":"v1","addr":"172.21.15.196:6835","nonce":1157781293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T13:33:33.440220+0000","modified":"2024-04-28T13:33:36.491827+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5781},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5781":{"gid":5781,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/1525791340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1525791340},{"type":"v1","addr":"172.21.15.203:6837","nonce":1525791340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5781,"qdb_cluster":[5781]},"id":3}]} 2024-04-28T13:36:21.508 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 31 2024-04-28T13:36:21.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.509+0000 7f43dd000640 1 -- 172.21.15.196:0/1385413647 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f43b806c980 msgr2=0x7f43b806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:21.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.509+0000 7f43dd000640 1 --2- 172.21.15.196:0/1385413647 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f43b806c980 0x7f43b806ee40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f43c4034000 tx=0x7f43c402e000 comp rx=0 tx=0).stop 2024-04-28T13:36:21.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.509+0000 7f43dd000640 1 -- 172.21.15.196:0/1385413647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d81629c0 msgr2=0x7f43d815b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:36:21.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.509+0000 7f43dd000640 1 --2- 172.21.15.196:0/1385413647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d81629c0 0x7f43d815b830 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c430 tx=0x7f43c800c8f0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.510+0000 7f43dd000640 1 -- 172.21.15.196:0/1385413647 shutdown_connections 2024-04-28T13:36:21.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.510+0000 7f43dd000640 1 --2- 172.21.15.196:0/1385413647 >> [v2:172.21.15.203:6832/3172262857,v1:172.21.15.203:6833/3172262857] conn(0x7f43b806c980 0x7f43b806ee40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.510+0000 7f43dd000640 1 --2- 172.21.15.196:0/1385413647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43d8163bf0 0x7f43d80b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.510+0000 7f43dd000640 1 --2- 172.21.15.196:0/1385413647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43d8163570 0x7f43d80cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.510+0000 7f43dd000640 1 --2- 172.21.15.196:0/1385413647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d81629c0 0x7f43d815b830 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:36:21.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.510+0000 7f43dd000640 1 -- 172.21.15.196:0/1385413647 >> 172.21.15.196:0/1385413647 conn(0x7f43d809e570 msgr2=0x7f43d80d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:36:21.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.510+0000 7f43dd000640 1 -- 172.21.15.196:0/1385413647 shutdown_connections 2024-04-28T13:36:21.512 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-28T13:36:21.510+0000 7f43dd000640 1 -- 172.21.15.196:0/1385413647 wait complete. 2024-04-28T13:36:21.527 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-28T13:36:21.528 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-28T13:36:21.528 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-28T13:36:21.528 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:21.553 INFO:tasks.ceph.mds.a:Stopped 2024-04-28T13:36:21.553 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-28T13:36:21.553 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:21.625 INFO:tasks.ceph.mds.c:Stopped 2024-04-28T13:36:21.625 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-28T13:36:21.625 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:21.657 INFO:tasks.ceph.mds.b:Stopped 2024-04-28T13:36:21.657 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-28T13:36:21.657 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:21.725 INFO:tasks.ceph.mds.d:Stopped 2024-04-28T13:36:21.725 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-28T13:36:21.725 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-28T13:36:21.725 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:21.800 INFO:tasks.ceph.osd.0:Stopped 2024-04-28T13:36:21.801 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-28T13:36:21.801 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:21.865 INFO:tasks.ceph.osd.1:Stopped 2024-04-28T13:36:21.866 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-28T13:36:21.866 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:21.946 INFO:tasks.ceph.osd.2:Stopped 2024-04-28T13:36:21.947 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-28T13:36:21.947 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:22.026 INFO:tasks.ceph.osd.3:Stopped 2024-04-28T13:36:22.027 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-28T13:36:22.027 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:22.097 INFO:tasks.ceph.osd.4:Stopped 2024-04-28T13:36:22.097 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-28T13:36:22.097 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:22.174 INFO:tasks.ceph.osd.5:Stopped 2024-04-28T13:36:22.174 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-28T13:36:22.174 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:22.263 INFO:tasks.ceph.osd.6:Stopped 2024-04-28T13:36:22.264 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-28T13:36:22.264 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:22.319 INFO:tasks.ceph.osd.7:Stopped 2024-04-28T13:36:22.319 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-28T13:36:22.320 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-28T13:36:22.320 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:22.414 INFO:tasks.ceph.mgr.y:Stopped 2024-04-28T13:36:22.415 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-28T13:36:22.415 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:22.489 INFO:tasks.ceph.mgr.x:Stopped 2024-04-28T13:36:22.489 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-28T13:36:22.489 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-28T13:36:22.489 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:22.533 INFO:tasks.ceph.mon.a:Stopped 2024-04-28T13:36:22.533 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-28T13:36:22.534 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:22.569 INFO:tasks.ceph.mon.b:Stopped 2024-04-28T13:36:22.570 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-28T13:36:22.570 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T13:36:22.622 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-04-28T13:36:22.622 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-04-28T13:36:22.622 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-04-28T13:36:22.623 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-28T13:36:22.623 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-28T13:36:22.623 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-28T13:36:22.656 INFO:tasks.ceph.mon.c:Stopped 2024-04-28T13:36:22.656 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-28T13:36:22.656 DEBUG:teuthology.orchestra.run.smithi196:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow request' | head -n 1 2024-04-28T13:36:22.760 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:36:22.760 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-28T13:36:22.986 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:36:22.986 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-28T13:36:23.117 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:36:23.117 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-28T13:36:23.248 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:36:23.248 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-28T13:36:23.411 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:36:23.411 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-28T13:36:23.701 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:36:23.701 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-28T13:36:23.831 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:36:23.832 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-28T13:36:23.987 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:36:23.988 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-28T13:36:24.108 INFO:tasks.ceph:Archiving mon data... 2024-04-28T13:36:24.110 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911/data/mon.a.tgz 2024-04-28T13:36:24.110 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-04-28T13:36:24.164 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.sjFqUa4XzB 2024-04-28T13:36:24.165 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.sjFqUa4XzB 2024-04-28T13:36:24.348 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.sjFqUa4XzB 2024-04-28T13:36:24.485 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.sjFqUa4XzB is 401KB 2024-04-28T13:36:24.549 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.sjFqUa4XzB 2024-04-28T13:36:24.598 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911/data/mon.b.tgz 2024-04-28T13:36:24.599 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-04-28T13:36:24.656 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.62ID5sCyXN 2024-04-28T13:36:24.657 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.62ID5sCyXN 2024-04-28T13:36:24.879 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.62ID5sCyXN 2024-04-28T13:36:25.006 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.62ID5sCyXN is 536KB 2024-04-28T13:36:25.107 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.62ID5sCyXN 2024-04-28T13:36:25.163 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911/data/mon.c.tgz 2024-04-28T13:36:25.163 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-04-28T13:36:25.212 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.UxEzC1iev8 2024-04-28T13:36:25.212 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.UxEzC1iev8 2024-04-28T13:36:25.403 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.UxEzC1iev8 2024-04-28T13:36:25.541 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.UxEzC1iev8 is 536KB 2024-04-28T13:36:25.648 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.UxEzC1iev8 2024-04-28T13:36:25.704 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-28T13:36:25.704 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-28T13:36:25.712 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-28T13:36:25.852 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-28T13:36:25.910 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-28T13:36:25.911 INFO:tasks.ceph:Archiving crash dumps... 2024-04-28T13:36:25.912 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911/remote/smithi196/crash 2024-04-28T13:36:25.913 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-28T13:36:25.999 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911/remote/smithi203/crash 2024-04-28T13:36:26.000 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-28T13:36:26.089 INFO:tasks.ceph:Shutting down logrotate 2024-04-28T13:36:26.089 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:36:26.097 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T13:36:26.224 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-28T13:36:26.291 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-28T13:36:26.379 INFO:tasks.ceph:Compressing logs... 2024-04-28T13:36:26.379 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-28T13:36:26.387 DEBUG:teuthology.orchestra.run.smithi203:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-28T13:36:26.464 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44931.log 2024-04-28T13:36:26.465 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-28T13:36:26.466 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.47390.log 2024-04-28T13:36:26.466 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33676.log 2024-04-28T13:36:26.467 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51602.log 2024-04-28T13:36:26.467 INFO:teuthology.orchestra.run.smithi196.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44931.log.gz 2024-04-28T13:36:26.468 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.40861.log 2024-04-28T13:36:26.468 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.2.47390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47871.log 2024-04-28T13:36:26.468 INFO:teuthology.orchestra.run.smithi196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.47390.log.gz 2024-04-28T13:36:26.470 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.51602.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51602.log.gz 2024-04-28T13:36:26.470 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52105.log 2024-04-28T13:36:26.470 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.47871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.47386.log 2024-04-28T13:36:26.470 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47871.log.gz 2024-04-28T13:36:26.471 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.33676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40649.log 2024-04-28T13:36:26.471 INFO:teuthology.orchestra.run.smithi203.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33676.log.gz 2024-04-28T13:36:26.471 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.3.40861.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-28T13:36:26.471 INFO:teuthology.orchestra.run.smithi203.stderr: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-28T13:36:26.471 INFO:teuthology.orchestra.run.smithi203.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.40861.log.gz 2024-04-28T13:36:26.472 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.40649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40649.log.gz 2024-04-28T13:36:26.472 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40306.log 2024-04-28T13:36:26.475 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33596.log 2024-04-28T13:36:26.475 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.40306.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40306.log.gz 2024-04-28T13:36:26.475 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-28T13:36:26.475 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.33596.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-28T13:36:26.476 INFO:teuthology.orchestra.run.smithi203.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33596.log.gz 2024-04-28T13:36:26.476 INFO:teuthology.orchestra.run.smithi203.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-28T13:36:26.476 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-28T13:36:26.477 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52411.log 2024-04-28T13:36:26.477 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52105.log.gz 2024-04-28T13:36:26.477 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.2.47386.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.47386.log.gz 2024-04-28T13:36:26.477 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47787.log 2024-04-28T13:36:26.477 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52411.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52411.log.gz 2024-04-28T13:36:26.477 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46339.log 2024-04-28T13:36:26.478 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.47787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38930.log 2024-04-28T13:36:26.478 INFO:teuthology.orchestra.run.smithi196.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47787.log.gz 2024-04-28T13:36:26.478 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.46339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49414.log 2024-04-28T13:36:26.478 INFO:teuthology.orchestra.run.smithi196.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46339.log.gz 2024-04-28T13:36:26.478 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.38930.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38930.log.gz 2024-04-28T13:36:26.478 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47156.log 2024-04-28T13:36:26.481 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-28T13:36:26.481 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33425.log 2024-04-28T13:36:26.481 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33636.log 2024-04-28T13:36:26.481 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.33425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39684.log 2024-04-28T13:36:26.481 INFO:teuthology.orchestra.run.smithi203.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33425.log.gz 2024-04-28T13:36:26.482 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47913.log 2024-04-28T13:36:26.483 INFO:teuthology.orchestra.run.smithi196.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49414.log.gz 2024-04-28T13:36:26.483 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.47156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42966.log 2024-04-28T13:36:26.483 INFO:teuthology.orchestra.run.smithi196.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47156.log.gz 2024-04-28T13:36:26.483 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.47913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42756.log 2024-04-28T13:36:26.483 INFO:teuthology.orchestra.run.smithi196.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47913.log.gz 2024-04-28T13:36:26.483 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46255.log 2024-04-28T13:36:26.483 INFO:teuthology.orchestra.run.smithi196.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42966.log.gz 2024-04-28T13:36:26.484 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39818.log 2024-04-28T13:36:26.484 INFO:teuthology.orchestra.run.smithi196.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42756.log.gz 2024-04-28T13:36:26.488 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.33636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33476.log 2024-04-28T13:36:26.488 INFO:teuthology.orchestra.run.smithi203.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33636.log.gz 2024-04-28T13:36:26.488 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.3.39684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33556.log 2024-04-28T13:36:26.488 INFO:teuthology.orchestra.run.smithi203.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.39684.log.gz 2024-04-28T13:36:26.489 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.33476.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-28T13:36:26.489 INFO:teuthology.orchestra.run.smithi203.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33476.log.gz 2024-04-28T13:36:26.489 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.33556.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-28T13:36:26.489 INFO:teuthology.orchestra.run.smithi203.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33556.log.gz 2024-04-28T13:36:26.489 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-28T13:36:26.490 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.46255.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46255.log.gz 2024-04-28T13:36:26.491 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42252.log 2024-04-28T13:36:26.491 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42924.log 2024-04-28T13:36:26.491 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39818.log.gz 2024-04-28T13:36:26.491 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49582.log 2024-04-28T13:36:26.491 INFO:teuthology.orchestra.run.smithi196.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42252.log.gz 2024-04-28T13:36:26.491 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42924.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42924.log.gz 2024-04-28T13:36:26.491 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39476.log 2024-04-28T13:36:26.491 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42546.log 2024-04-28T13:36:26.491 INFO:teuthology.orchestra.run.smithi196.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49582.log.gz 2024-04-28T13:36:26.492 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41479.log 2024-04-28T13:36:26.492 INFO:teuthology.orchestra.run.smithi196.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39476.log.gz 2024-04-28T13:36:26.492 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42546.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.42546.log.gz 2024-04-28T13:36:26.492 INFO:teuthology.orchestra.run.smithi196.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46659.log 2024-04-28T13:36:26.492 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.0.41479.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41479.log.gz 2024-04-28T13:36:26.492 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44601.log 2024-04-28T13:36:26.497 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.46659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47529.log 2024-04-28T13:36:26.497 INFO:teuthology.orchestra.run.smithi196.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46659.log.gz 2024-04-28T13:36:26.497 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45311.log 2024-04-28T13:36:26.497 INFO:teuthology.orchestra.run.smithi196.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44601.log.gz 2024-04-28T13:36:26.497 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.47529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49666.log 2024-04-28T13:36:26.497 INFO:teuthology.orchestra.run.smithi196.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.47529.log.gz 2024-04-28T13:36:26.497 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40568.log 2024-04-28T13:36:26.498 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45311.log.gz 2024-04-28T13:36:26.498 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50301.log 2024-04-28T13:36:26.498 INFO:teuthology.orchestra.run.smithi196.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49666.log.gz 2024-04-28T13:36:26.498 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40568.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40568.log.gz 2024-04-28T13:36:26.498 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43161.log 2024-04-28T13:36:26.499 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40645.log 2024-04-28T13:36:26.499 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.1.40645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39688.log 2024-04-28T13:36:26.500 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.3.39688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40645.log.gz 2024-04-28T13:36:26.500 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40302.log 2024-04-28T13:36:26.500 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.40857.log 2024-04-28T13:36:26.500 INFO:teuthology.orchestra.run.smithi203.stderr: 0.0%/var/log/ceph/ceph-client.3.40857.log: -- replaced with /var/log/ceph/ceph-client.3.39688.log.gz 2024-04-28T13:36:26.502 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53058.log 2024-04-28T13:36:26.502 INFO:teuthology.orchestra.run.smithi196.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50301.log.gz 2024-04-28T13:36:26.502 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52453.log 2024-04-28T13:36:26.502 INFO:teuthology.orchestra.run.smithi196.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43161.log.gz 2024-04-28T13:36:26.502 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.53058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43050.log 2024-04-28T13:36:26.503 INFO:teuthology.orchestra.run.smithi196.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53058.log.gz 2024-04-28T13:36:26.503 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52453.log: gzip -5 87.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.52453.log.gz 2024-04-28T13:36:26.503 INFO:teuthology.orchestra.run.smithi196.stderr: -- /var/log/ceph/ceph-client.admin.43851.log 2024-04-28T13:36:26.505 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46827.log 2024-04-28T13:36:26.505 INFO:teuthology.orchestra.run.smithi196.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43050.log.gz 2024-04-28T13:36:26.506 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38888.log 2024-04-28T13:36:26.506 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43851.log.gz 2024-04-28T13:36:26.506 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.46827.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46827.log.gz 2024-04-28T13:36:26.506 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48320.log 2024-04-28T13:36:26.507 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.40302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40302.log.gz 2024-04-28T13:36:26.507 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39270.log 2024-04-28T13:36:26.507 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41699.log 2024-04-28T13:36:26.507 INFO:teuthology.orchestra.run.smithi203.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.40857.log.gz 2024-04-28T13:36:26.508 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.39270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39289.log 2024-04-28T13:36:26.509 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.38888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42378.log 2024-04-28T13:36:26.510 INFO:teuthology.orchestra.run.smithi196.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38888.log.gz 2024-04-28T13:36:26.510 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49330.log 2024-04-28T13:36:26.510 INFO:teuthology.orchestra.run.smithi196.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48320.log.gz 2024-04-28T13:36:26.510 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49372.log 2024-04-28T13:36:26.510 INFO:teuthology.orchestra.run.smithi196.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42378.log.gz 2024-04-28T13:36:26.510 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49330.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49330.log.gz -5 2024-04-28T13:36:26.510 INFO:teuthology.orchestra.run.smithi196.stderr: --verbose -- /var/log/ceph/ceph-client.0.41498.log 2024-04-28T13:36:26.513 INFO:teuthology.orchestra.run.smithi203.stderr: 0.0%/var/log/ceph/ceph-client.1.41699.log: gzip -- replaced with /var/log/ceph/ceph-client.1.39270.log.gz 2024-04-28T13:36:26.513 INFO:teuthology.orchestra.run.smithi203.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33385.log 2024-04-28T13:36:26.514 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.39289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.39289.log.gz 2024-04-28T13:36:26.514 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.33385.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33385.log.gz 2024-04-28T13:36:26.514 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48826.log 2024-04-28T13:36:26.515 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49372.log.gz 2024-04-28T13:36:26.515 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.0.41498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41498.log.gz 2024-04-28T13:36:26.515 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46093.log 2024-04-28T13:36:26.515 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48826.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-28T13:36:26.515 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48826.log.gz 2024-04-28T13:36:26.515 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.0.46093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46093.log.gz 2024-04-28T13:36:26.515 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40154.log 2024-04-28T13:36:26.516 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45225.log 2024-04-28T13:36:26.518 INFO:teuthology.orchestra.run.smithi203.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41699.log.gz 2024-04-28T13:36:26.518 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41703.log 2024-04-28T13:36:26.518 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.41703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33516.log 2024-04-28T13:36:26.520 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40442.log 2024-04-28T13:36:26.520 INFO:teuthology.orchestra.run.smithi196.stderr: 86.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-28T13:36:26.520 INFO:teuthology.orchestra.run.smithi196.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40154.log.gz 2024-04-28T13:36:26.520 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52789.log 2024-04-28T13:36:26.520 INFO:teuthology.orchestra.run.smithi196.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45225.log.gz 2024-04-28T13:36:26.520 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45397.log 2024-04-28T13:36:26.520 INFO:teuthology.orchestra.run.smithi196.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40442.log.gz 2024-04-28T13:36:26.521 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43767.log 2024-04-28T13:36:26.521 INFO:teuthology.orchestra.run.smithi196.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52789.log.gz 2024-04-28T13:36:26.521 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.41935.log 2024-04-28T13:36:26.521 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45397.log.gz 2024-04-28T13:36:26.524 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.33516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41703.log.gz 2024-04-28T13:36:26.524 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-28T13:36:26.524 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-28T13:36:26.525 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48664.log 2024-04-28T13:36:26.525 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43767.log.gz 2024-04-28T13:36:26.525 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.2.41935.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.2.41935.log.gz 2024-04-28T13:36:26.525 INFO:teuthology.orchestra.run.smithi196.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-28T13:36:26.525 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47637.log 2024-04-28T13:36:26.525 INFO:teuthology.orchestra.run.smithi196.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48664.log.gz 2024-04-28T13:36:26.525 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52327.log 2024-04-28T13:36:26.526 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.47637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52873.log 2024-04-28T13:36:26.528 INFO:teuthology.orchestra.run.smithi203.stderr: 88.2%/var/log/ceph/ceph-mds.d.log: -- replaced with /var/log/ceph/ceph-client.admin.33516.log.gz 2024-04-28T13:36:26.528 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-28T13:36:26.528 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52327.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47637.log.gz 2024-04-28T13:36:26.529 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49456.log 2024-04-28T13:36:26.529 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52327.log.gz 2024-04-28T13:36:26.529 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47090.log 2024-04-28T13:36:26.529 INFO:teuthology.orchestra.run.smithi196.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52873.log.gz 2024-04-28T13:36:26.529 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49456.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49456.log.gz 2024-04-28T13:36:26.529 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50473.log 2024-04-28T13:36:26.530 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.0.47090.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.0.47090.log.gz -5 2024-04-28T13:36:26.530 INFO:teuthology.orchestra.run.smithi196.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44685.log 2024-04-28T13:36:26.530 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50431.log 2024-04-28T13:36:26.531 INFO:teuthology.orchestra.run.smithi196.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50473.log.gz 2024-04-28T13:36:26.531 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49834.log 2024-04-28T13:36:26.532 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44685.log.gz 2024-04-28T13:36:26.533 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52243.log 2024-04-28T13:36:26.533 INFO:teuthology.orchestra.run.smithi196.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50431.log.gz 2024-04-28T13:36:26.534 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49834.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49834.log.gzgzip 2024-04-28T13:36:26.534 INFO:teuthology.orchestra.run.smithi196.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43008.log 2024-04-28T13:36:26.535 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39802.log 2024-04-28T13:36:26.535 INFO:teuthology.orchestra.run.smithi196.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52243.log.gz 2024-04-28T13:36:26.536 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40162.log 2024-04-28T13:36:26.536 INFO:teuthology.orchestra.run.smithi196.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43008.log.gz 2024-04-28T13:36:26.537 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50641.log 2024-04-28T13:36:26.537 INFO:teuthology.orchestra.run.smithi196.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39802.log.gz 2024-04-28T13:36:26.538 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38466.log 2024-04-28T13:36:26.538 INFO:teuthology.orchestra.run.smithi196.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40162.log.gz 2024-04-28T13:36:26.539 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45733.log 2024-04-28T13:36:26.540 INFO:teuthology.orchestra.run.smithi196.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50641.log.gz 2024-04-28T13:36:26.541 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.38466.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-28T13:36:26.541 INFO:teuthology.orchestra.run.smithi196.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38466.log.gz 2024-04-28T13:36:26.542 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45439.log 2024-04-28T13:36:26.542 INFO:teuthology.orchestra.run.smithi196.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45733.log.gz 2024-04-28T13:36:26.543 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40820.log 2024-04-28T13:36:26.544 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51738.log 2024-04-28T13:36:26.544 INFO:teuthology.orchestra.run.smithi196.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-28T13:36:26.544 INFO:teuthology.orchestra.run.smithi196.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45439.log.gz 2024-04-28T13:36:26.545 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51644.log 2024-04-28T13:36:26.545 INFO:teuthology.orchestra.run.smithi196.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40820.log.gz 2024-04-28T13:36:26.546 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.51738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38674.log 2024-04-28T13:36:26.546 INFO:teuthology.orchestra.run.smithi196.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51738.log.gz 2024-04-28T13:36:26.548 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.51644.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51644.log.gz 2024-04-28T13:36:26.548 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38972.log 2024-04-28T13:36:26.549 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.38674.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38674.log.gz 2024-04-28T13:36:26.549 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43245.log 2024-04-28T13:36:26.551 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.38972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46785.log 2024-04-28T13:36:26.551 INFO:teuthology.orchestra.run.smithi196.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38972.log.gz 2024-04-28T13:36:26.552 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43809.log 2024-04-28T13:36:26.552 INFO:teuthology.orchestra.run.smithi196.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43245.log.gz 2024-04-28T13:36:26.553 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.46785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49162.log 2024-04-28T13:36:26.553 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46785.log.gz 2024-04-28T13:36:26.554 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43809.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43809.log.gz 2024-04-28T13:36:26.554 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49708.log 2024-04-28T13:36:26.555 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45649.log 2024-04-28T13:36:26.555 INFO:teuthology.orchestra.run.smithi196.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49162.log.gz 2024-04-28T13:36:26.556 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52495.log 2024-04-28T13:36:26.556 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49708.log.gz 2024-04-28T13:36:26.557 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38758.log 2024-04-28T13:36:26.558 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45649.log.gz 2024-04-28T13:36:26.558 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52495.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-28T13:36:26.559 INFO:teuthology.orchestra.run.smithi196.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52495.log.gz 2024-04-28T13:36:26.559 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.38758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48952.log 2024-04-28T13:36:26.560 INFO:teuthology.orchestra.run.smithi196.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38758.log.gz 2024-04-28T13:36:26.561 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39475.log 2024-04-28T13:36:26.562 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33331.log 2024-04-28T13:36:26.562 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48952.log.gz 2024-04-28T13:36:26.563 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39475.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.45481.log 2024-04-28T13:36:26.563 INFO:teuthology.orchestra.run.smithi196.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39475.log.gz 2024-04-28T13:36:26.564 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.33331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48362.log 2024-04-28T13:36:26.565 INFO:teuthology.orchestra.run.smithi196.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.33331.log.gz 2024-04-28T13:36:26.566 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50345.log 2024-04-28T13:36:26.566 INFO:teuthology.orchestra.run.smithi196.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45481.log.gz 2024-04-28T13:36:26.567 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40652.log 2024-04-28T13:36:26.567 INFO:teuthology.orchestra.run.smithi196.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48362.log.gz 2024-04-28T13:36:26.568 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49288.log 2024-04-28T13:36:26.568 INFO:teuthology.orchestra.run.smithi196.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50345.log.gz 2024-04-28T13:36:26.569 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39098.log 2024-04-28T13:36:26.569 INFO:teuthology.orchestra.run.smithi196.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40652.log.gz 2024-04-28T13:36:26.570 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49288.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49288.log.gz 2024-04-28T13:36:26.570 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45141.log 2024-04-28T13:36:26.571 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46171.log 2024-04-28T13:36:26.572 INFO:teuthology.orchestra.run.smithi196.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39098.log.gz 2024-04-28T13:36:26.573 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40484.log 2024-04-28T13:36:26.573 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45141.log.gz 2024-04-28T13:36:26.574 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.46171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40904.log 2024-04-28T13:36:26.574 INFO:teuthology.orchestra.run.smithi196.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46171.log.gz 2024-04-28T13:36:26.575 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40484.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40484.log.gz 2024-04-28T13:36:26.575 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45057.log 2024-04-28T13:36:26.576 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40132.log 2024-04-28T13:36:26.576 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40904.log.gz 2024-04-28T13:36:26.577 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47703.log 2024-04-28T13:36:26.577 INFO:teuthology.orchestra.run.smithi196.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45057.log.gz 2024-04-28T13:36:26.578 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40132.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-28T13:36:26.578 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40132.log.gz 2024-04-28T13:36:26.579 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.47703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39819.log 2024-04-28T13:36:26.579 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47703.log.gz 2024-04-28T13:36:26.580 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39825.log 2024-04-28T13:36:26.582 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39819.log: 92.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-28T13:36:26.582 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46743.log 2024-04-28T13:36:26.582 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39819.log.gz 2024-04-28T13:36:26.583 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49971.log 2024-04-28T13:36:26.583 INFO:teuthology.orchestra.run.smithi196.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39825.log.gz 2024-04-28T13:36:26.584 INFO:teuthology.orchestra.run.smithi203.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-28T13:36:26.585 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.46743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52063.log 2024-04-28T13:36:26.585 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46743.log.gz 2024-04-28T13:36:26.586 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42336.log 2024-04-28T13:36:26.586 INFO:teuthology.orchestra.run.smithi196.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49971.log.gz 2024-04-28T13:36:26.587 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51560.log 2024-04-28T13:36:26.587 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52063.log.gz 2024-04-28T13:36:26.588 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46455.log 2024-04-28T13:36:26.588 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42336.log.gz 2024-04-28T13:36:26.589 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.51560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48200.log 2024-04-28T13:36:26.589 INFO:teuthology.orchestra.run.smithi196.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51560.log.gz 2024-04-28T13:36:26.590 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.46455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45355.log 2024-04-28T13:36:26.590 INFO:teuthology.orchestra.run.smithi196.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46455.log.gz 2024-04-28T13:36:26.591 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48200.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-28T13:36:26.591 INFO:teuthology.orchestra.run.smithi196.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48200.log.gz 2024-04-28T13:36:26.592 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45355.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-28T13:36:26.592 INFO:teuthology.orchestra.run.smithi196.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45355.log.gz 2024-04-28T13:36:26.593 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51476.log 2024-04-28T13:36:26.595 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52621.log 2024-04-28T13:36:26.596 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.51476.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26135.log 2024-04-28T13:36:26.597 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51476.log.gz 2024-04-28T13:36:26.597 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51127.log 2024-04-28T13:36:26.598 INFO:teuthology.orchestra.run.smithi196.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52621.log.gz 2024-04-28T13:36:26.599 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph.tmp-client.admin.26135.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26135.log.gz 2024-04-28T13:36:26.599 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42462.log 2024-04-28T13:36:26.600 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.0.51127.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51127.log.gz 2024-04-28T13:36:26.601 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50259.log 2024-04-28T13:36:26.602 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44973.log 2024-04-28T13:36:26.602 INFO:teuthology.orchestra.run.smithi196.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-28T13:36:26.603 INFO:teuthology.orchestra.run.smithi196.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42462.log.gz 2024-04-28T13:36:26.603 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39182.log 2024-04-28T13:36:26.603 INFO:teuthology.orchestra.run.smithi196.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50259.log.gz 2024-04-28T13:36:26.604 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44973.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44973.log.gz 2024-04-28T13:36:26.604 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42882.log 2024-04-28T13:36:26.605 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39182.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-28T13:36:26.606 INFO:teuthology.orchestra.run.smithi196.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39182.log.gz 2024-04-28T13:36:26.606 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50599.log 2024-04-28T13:36:26.607 INFO:teuthology.orchestra.run.smithi196.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42882.log.gz 2024-04-28T13:36:26.608 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44055.log 2024-04-28T13:36:26.609 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44307.log 2024-04-28T13:36:26.609 INFO:teuthology.orchestra.run.smithi196.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50599.log.gz 2024-04-28T13:36:26.610 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52976.log 2024-04-28T13:36:26.611 INFO:teuthology.orchestra.run.smithi196.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44055.log.gz 2024-04-28T13:36:26.611 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50217.log 2024-04-28T13:36:26.612 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44307.log.gz 2024-04-28T13:36:26.613 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53016.log 2024-04-28T13:36:26.613 INFO:teuthology.orchestra.run.smithi196.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52976.log.gz 2024-04-28T13:36:26.614 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46497.log 2024-04-28T13:36:26.615 INFO:teuthology.orchestra.run.smithi196.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50217.log.gz 2024-04-28T13:36:26.615 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.53016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50683.log 2024-04-28T13:36:26.616 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53016.log.gz 2024-04-28T13:36:26.617 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.46497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49876.log 2024-04-28T13:36:26.617 INFO:teuthology.orchestra.run.smithi196.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46497.log.gz 2024-04-28T13:36:26.618 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48404.log 2024-04-28T13:36:26.618 INFO:teuthology.orchestra.run.smithi196.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50683.log.gz 2024-04-28T13:36:26.619 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46581.log 2024-04-28T13:36:26.619 INFO:teuthology.orchestra.run.smithi196.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49876.log.gz 2024-04-28T13:36:26.621 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47198.log 2024-04-28T13:36:26.621 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48404.log.gz 2024-04-28T13:36:26.622 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.46581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45183.log 2024-04-28T13:36:26.622 INFO:teuthology.orchestra.run.smithi196.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46581.log.gz 2024-04-28T13:36:26.623 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.47198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39482.log 2024-04-28T13:36:26.623 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47198.log.gz 2024-04-28T13:36:26.625 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44181.log 2024-04-28T13:36:26.625 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45183.log.gz 2024-04-28T13:36:26.626 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42714.log 2024-04-28T13:36:26.626 INFO:teuthology.orchestra.run.smithi196.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39482.log.gz 2024-04-28T13:36:26.627 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46213.log 2024-04-28T13:36:26.627 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44181.log.gz 2024-04-28T13:36:26.628 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52537.log 2024-04-28T13:36:26.629 INFO:teuthology.orchestra.run.smithi196.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42714.log.gz 2024-04-28T13:36:26.630 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.46213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50389.log 2024-04-28T13:36:26.630 INFO:teuthology.orchestra.run.smithi196.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46213.log.gz 2024-04-28T13:36:26.631 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39823.log 2024-04-28T13:36:26.631 INFO:teuthology.orchestra.run.smithi196.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52537.log.gz 2024-04-28T13:36:26.632 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50557.log 2024-04-28T13:36:26.632 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50389.log.gz 2024-04-28T13:36:26.633 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39489.log 2024-04-28T13:36:26.634 INFO:teuthology.orchestra.run.smithi196.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39823.log.gz 2024-04-28T13:36:26.634 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40694.log 2024-04-28T13:36:26.635 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50557.log.gz 2024-04-28T13:36:26.636 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52747.log 2024-04-28T13:36:26.636 INFO:teuthology.orchestra.run.smithi196.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39489.log.gz 2024-04-28T13:36:26.637 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40400.log 2024-04-28T13:36:26.637 INFO:teuthology.orchestra.run.smithi196.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40694.log.gz 2024-04-28T13:36:26.638 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46701.log 2024-04-28T13:36:26.638 INFO:teuthology.orchestra.run.smithi196.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52747.log.gz 2024-04-28T13:36:26.639 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38716.log 2024-04-28T13:36:26.640 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40400.log.gz 2024-04-28T13:36:26.641 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.46701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38547.log 2024-04-28T13:36:26.641 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46701.log.gz 2024-04-28T13:36:26.642 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.38716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46297.log 2024-04-28T13:36:26.642 INFO:teuthology.orchestra.run.smithi196.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38716.log.gz 2024-04-28T13:36:26.643 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.38547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40988.log 2024-04-28T13:36:26.644 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38547.log.gz 2024-04-28T13:36:26.645 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.46297.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-28T13:36:26.645 INFO:teuthology.orchestra.run.smithi196.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46297.log.gz 2024-04-28T13:36:26.646 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48278.log 2024-04-28T13:36:26.647 INFO:teuthology.orchestra.run.smithi196.stderr: 91.0%/var/log/ceph/ceph-mon.a.log: -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz 2024-04-28T13:36:26.647 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49204.log 2024-04-28T13:36:26.649 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39056.log 2024-04-28T13:36:26.649 INFO:teuthology.orchestra.run.smithi196.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48278.log.gz 2024-04-28T13:36:26.650 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49204.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49204.log.gz 2024-04-28T13:36:26.650 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49624.log 2024-04-28T13:36:26.652 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39056.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39056.log.gz 2024-04-28T13:36:26.652 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38632.log 2024-04-28T13:36:26.653 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49624.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49624.log.gz 2024-04-28T13:36:26.653 INFO:teuthology.orchestra.run.smithi203.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-28T13:36:26.654 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52579.log 2024-04-28T13:36:26.655 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.38632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49540.log 2024-04-28T13:36:26.655 INFO:teuthology.orchestra.run.smithi196.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38632.log.gz 2024-04-28T13:36:26.656 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42840.log 2024-04-28T13:36:26.656 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52579.log.gz 2024-04-28T13:36:26.658 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49540.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49540.log.gz 2024-04-28T13:36:26.659 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42840.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42840.log.gz 2024-04-28T13:36:26.659 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42798.log 2024-04-28T13:36:26.660 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48032.log 2024-04-28T13:36:26.663 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50051.log 2024-04-28T13:36:26.663 INFO:teuthology.orchestra.run.smithi196.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42798.log.gz 2024-04-28T13:36:26.664 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48032.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48032.log.gz 2024-04-28T13:36:26.664 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50051.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50051.log.gz 2024-04-28T13:36:26.665 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48910.log 2024-04-28T13:36:26.666 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43683.log 2024-04-28T13:36:26.668 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51822.log 2024-04-28T13:36:26.668 INFO:teuthology.orchestra.run.smithi196.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48910.log.gz 2024-04-28T13:36:26.669 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43683.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43683.log.gz 2024-04-28T13:36:26.670 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.51822.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51822.log.gz 2024-04-28T13:36:26.671 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40526.log 2024-04-28T13:36:26.672 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38802.log 2024-04-28T13:36:26.673 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48784.log 2024-04-28T13:36:26.674 INFO:teuthology.orchestra.run.smithi196.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40526.log.gz 2024-04-28T13:36:26.675 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.38802.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38802.log.gz 2024-04-28T13:36:26.676 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48784.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48784.log.gz 2024-04-28T13:36:26.677 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50935.log 2024-04-28T13:36:26.678 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44097.log 2024-04-28T13:36:26.679 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44888.log 2024-04-28T13:36:26.680 INFO:teuthology.orchestra.run.smithi196.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50935.log.gz 2024-04-28T13:36:26.681 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44097.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44097.log.gz 2024-04-28T13:36:26.682 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44888.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44888.log.gz 2024-04-28T13:36:26.682 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50174.log 2024-04-28T13:36:26.684 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39824.log 2024-04-28T13:36:26.685 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51123.log 2024-04-28T13:36:26.685 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50174.log.gz 2024-04-28T13:36:26.687 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39824.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39824.log.gz 2024-04-28T13:36:26.687 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.0.51123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51123.log.gz 2024-04-28T13:36:26.688 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46902.log 2024-04-28T13:36:26.689 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44846.log 2024-04-28T13:36:26.690 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.46902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52369.log 2024-04-28T13:36:26.691 INFO:teuthology.orchestra.run.smithi196.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46902.log.gz 2024-04-28T13:36:26.692 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44846.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44846.log.gz 2024-04-28T13:36:26.693 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52369.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52369.log.gz 2024-04-28T13:36:26.693 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48868.log 2024-04-28T13:36:26.695 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44349.log 2024-04-28T13:36:26.696 INFO:teuthology.orchestra.run.smithi203.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-28T13:36:26.697 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48868.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38589.log 2024-04-28T13:36:26.697 INFO:teuthology.orchestra.run.smithi196.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48868.log.gz 2024-04-28T13:36:26.698 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52663.log 2024-04-28T13:36:26.698 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44349.log.gz 2024-04-28T13:36:26.699 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.38589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40358.log 2024-04-28T13:36:26.700 INFO:teuthology.orchestra.run.smithi196.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38589.log.gz 2024-04-28T13:36:26.701 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39224.log 2024-04-28T13:36:26.701 INFO:teuthology.orchestra.run.smithi196.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52663.log.gz 2024-04-28T13:36:26.702 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45267.log 2024-04-28T13:36:26.702 INFO:teuthology.orchestra.run.smithi196.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40358.log.gz 2024-04-28T13:36:26.704 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39224.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52705.log 2024-04-28T13:36:26.704 INFO:teuthology.orchestra.run.smithi196.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39224.log.gz 2024-04-28T13:36:26.705 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45267.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45267.log.gz 2024-04-28T13:36:26.705 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42588.log 2024-04-28T13:36:26.706 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.41939.log 2024-04-28T13:36:26.707 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52705.log.gz 2024-04-28T13:36:26.708 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49036.log 2024-04-28T13:36:26.708 INFO:teuthology.orchestra.run.smithi196.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-28T13:36:26.708 INFO:teuthology.orchestra.run.smithi196.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42588.log.gz 2024-04-28T13:36:26.709 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.2.41939.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.41939.log.gzgzip 2024-04-28T13:36:26.709 INFO:teuthology.orchestra.run.smithi196.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49078.log 2024-04-28T13:36:26.710 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44013.log 2024-04-28T13:36:26.710 INFO:teuthology.orchestra.run.smithi196.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49036.log.gz 2024-04-28T13:36:26.711 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49078.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49078.log.gz -5 2024-04-28T13:36:26.711 INFO:teuthology.orchestra.run.smithi196.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49498.log 2024-04-28T13:36:26.712 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41126.log 2024-04-28T13:36:26.713 INFO:teuthology.orchestra.run.smithi196.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44013.log.gz 2024-04-28T13:36:26.713 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44559.log 2024-04-28T13:36:26.714 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49498.log.gz 2024-04-28T13:36:26.715 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.41126.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41126.log.gz 2024-04-28T13:36:26.715 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42168.log 2024-04-28T13:36:26.716 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39826.log 2024-04-28T13:36:26.716 INFO:teuthology.orchestra.run.smithi196.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44559.log.gz 2024-04-28T13:36:26.717 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42168.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42168.log.gz 2024-04-28T13:36:26.717 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40736.log 2024-04-28T13:36:26.718 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39308.log 2024-04-28T13:36:26.718 INFO:teuthology.orchestra.run.smithi203.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-28T13:36:26.719 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39826.log: /var/log/ceph/ceph-client.admin.40736.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39826.log.gz 2024-04-28T13:36:26.719 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40736.log.gz 2024-04-28T13:36:26.719 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50767.log 2024-04-28T13:36:26.720 INFO:teuthology.orchestra.run.smithi203.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-28T13:36:26.721 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40778.log 2024-04-28T13:36:26.721 INFO:teuthology.orchestra.run.smithi196.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39308.log.gz 2024-04-28T13:36:26.722 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48580.log 2024-04-28T13:36:26.722 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50767.log.gz 2024-04-28T13:36:26.723 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45015.log 2024-04-28T13:36:26.723 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40778.log.gz 2024-04-28T13:36:26.724 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48580.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48580.log.gz 2024-04-28T13:36:26.724 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50725.log 2024-04-28T13:36:26.725 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51518.log 2024-04-28T13:36:26.726 INFO:teuthology.orchestra.run.smithi196.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45015.log.gz 2024-04-28T13:36:26.727 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50725.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50725.log.gz 2024-04-28T13:36:26.727 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42504.log 2024-04-28T13:36:26.727 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.51518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52285.log 2024-04-28T13:36:26.728 INFO:teuthology.orchestra.run.smithi196.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51518.log.gz 2024-04-28T13:36:26.729 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43491.log 2024-04-28T13:36:26.729 INFO:teuthology.orchestra.run.smithi196.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42504.log.gz 2024-04-28T13:36:26.730 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52285.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33434.log 2024-04-28T13:36:26.730 INFO:teuthology.orchestra.run.smithi196.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52285.log.gz 2024-04-28T13:36:26.731 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42126.log 2024-04-28T13:36:26.732 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43491.log.gz 2024-04-28T13:36:26.732 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.33434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45607.log 2024-04-28T13:36:26.732 INFO:teuthology.orchestra.run.smithi196.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33434.log.gz 2024-04-28T13:36:26.734 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46089.log 2024-04-28T13:36:26.734 INFO:teuthology.orchestra.run.smithi196.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42126.log.gz 2024-04-28T13:36:26.735 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45607.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45607.log.gz 2024-04-28T13:36:26.735 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44139.log 2024-04-28T13:36:26.736 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.0.46089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50869.log 2024-04-28T13:36:26.736 INFO:teuthology.orchestra.run.smithi196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46089.log.gz 2024-04-28T13:36:26.737 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47595.log 2024-04-28T13:36:26.737 INFO:teuthology.orchestra.run.smithi196.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44139.log.gz 2024-04-28T13:36:26.738 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43329.log 2024-04-28T13:36:26.739 INFO:teuthology.orchestra.run.smithi203.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-28T13:36:26.739 INFO:teuthology.orchestra.run.smithi196.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50869.log.gz 2024-04-28T13:36:26.740 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.47595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42294.log 2024-04-28T13:36:26.740 INFO:teuthology.orchestra.run.smithi196.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47595.log.gz 2024-04-28T13:36:26.741 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43329.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43329.log.gz 2024-04-28T13:36:26.741 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33065.log 2024-04-28T13:36:26.742 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42294.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42294.log.gz 2024-04-28T13:36:26.742 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49750.log 2024-04-28T13:36:26.743 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.33065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48446.log 2024-04-28T13:36:26.744 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49750.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.33065.log.gz 2024-04-28T13:36:26.745 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51864.log 2024-04-28T13:36:26.745 INFO:teuthology.orchestra.run.smithi196.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49750.log.gz 2024-04-28T13:36:26.746 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39810.log 2024-04-28T13:36:26.746 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48446.log.gz 2024-04-28T13:36:26.747 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.51864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39266.log 2024-04-28T13:36:26.747 INFO:teuthology.orchestra.run.smithi196.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51864.log.gz 2024-04-28T13:36:26.748 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51906.log 2024-04-28T13:36:26.749 INFO:teuthology.orchestra.run.smithi196.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39810.log.gz 2024-04-28T13:36:26.750 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46539.log 2024-04-28T13:36:26.750 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39266.log.gz 2024-04-28T13:36:26.750 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.51906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45691.log 2024-04-28T13:36:26.751 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51906.log.gz 2024-04-28T13:36:26.752 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.46539.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46539.log.gz 2024-04-28T13:36:26.752 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52193.log 2024-04-28T13:36:26.753 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39472.log 2024-04-28T13:36:26.754 INFO:teuthology.orchestra.run.smithi196.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45691.log.gz 2024-04-28T13:36:26.754 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52193.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52193.log.gz 2024-04-28T13:36:26.754 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46413.log 2024-04-28T13:36:26.755 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38370.log 2024-04-28T13:36:26.756 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39472.log: /var/log/ceph/ceph-client.admin.46413.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39472.log.gz 2024-04-28T13:36:26.756 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39140.log 2024-04-28T13:36:26.757 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46413.log.gz 2024-04-28T13:36:26.757 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.38370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52915.log 2024-04-28T13:36:26.758 INFO:teuthology.orchestra.run.smithi196.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38370.log.gz 2024-04-28T13:36:26.759 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39140.log: gzip -5 88.8% --verbose -- replaced with /var/log/ceph/ceph-client.admin.39140.log.gz -- 2024-04-28T13:36:26.759 INFO:teuthology.orchestra.run.smithi196.stderr: /var/log/ceph/ceph-client.admin.43407.log 2024-04-28T13:36:26.760 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44433.log 2024-04-28T13:36:26.761 INFO:teuthology.orchestra.run.smithi196.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52915.log.gz 2024-04-28T13:36:26.761 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43407.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43407.log.gz 2024-04-28T13:36:26.761 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42072.log 2024-04-28T13:36:26.762 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45523.log 2024-04-28T13:36:26.762 INFO:teuthology.orchestra.run.smithi203.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-28T13:36:26.763 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42072.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44433.log.gz 2024-04-28T13:36:26.763 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42072.log.gz 2024-04-28T13:36:26.763 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40149.log 2024-04-28T13:36:26.764 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49792.log 2024-04-28T13:36:26.765 INFO:teuthology.orchestra.run.smithi196.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45523.log.gz 2024-04-28T13:36:26.765 INFO:teuthology.orchestra.run.smithi203.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-28T13:36:26.766 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40149.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40149.log.gz 2024-04-28T13:36:26.766 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44265.log 2024-04-28T13:36:26.766 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42672.log 2024-04-28T13:36:26.767 INFO:teuthology.orchestra.run.smithi196.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49792.log.gz 2024-04-28T13:36:26.768 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44265.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44265.log.gz 2024-04-28T13:36:26.768 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39486.log 2024-04-28T13:36:26.768 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43203.log 2024-04-28T13:36:26.769 INFO:teuthology.orchestra.run.smithi203.stderr: 2024-04-28T13:36:26.769 INFO:teuthology.orchestra.run.smithi203.stderr:real 0m0.333s 2024-04-28T13:36:26.769 INFO:teuthology.orchestra.run.smithi203.stderr:user 0m1.620s 2024-04-28T13:36:26.769 INFO:teuthology.orchestra.run.smithi203.stderr:sys 0m0.274s 2024-04-28T13:36:26.771 INFO:teuthology.orchestra.run.smithi196.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42672.log.gz 2024-04-28T13:36:26.771 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42210.log 2024-04-28T13:36:26.771 INFO:teuthology.orchestra.run.smithi196.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39486.log.gz 2024-04-28T13:36:26.771 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49246.log 2024-04-28T13:36:26.772 INFO:teuthology.orchestra.run.smithi196.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43203.log.gz 2024-04-28T13:36:26.772 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44765.log 2024-04-28T13:36:26.772 INFO:teuthology.orchestra.run.smithi196.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42210.log.gz 2024-04-28T13:36:26.773 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43287.log 2024-04-28T13:36:26.773 INFO:teuthology.orchestra.run.smithi196.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49246.log.gz 2024-04-28T13:36:26.774 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38846.log 2024-04-28T13:36:26.774 INFO:teuthology.orchestra.run.smithi196.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44765.log.gz 2024-04-28T13:36:26.775 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47745.log 2024-04-28T13:36:26.776 INFO:teuthology.orchestra.run.smithi196.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43287.log.gz 2024-04-28T13:36:26.776 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.38846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43449.log 2024-04-28T13:36:26.776 INFO:teuthology.orchestra.run.smithi196.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38846.log.gz 2024-04-28T13:36:26.777 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.47745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49120.log 2024-04-28T13:36:26.778 INFO:teuthology.orchestra.run.smithi196.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47745.log.gz 2024-04-28T13:36:26.779 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43449.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48706.log 2024-04-28T13:36:26.779 INFO:teuthology.orchestra.run.smithi196.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43449.log.gz 2024-04-28T13:36:26.780 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.49120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52148.log 2024-04-28T13:36:26.780 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49120.log.gz 2024-04-28T13:36:26.781 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48706.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.48706.log.gz 2024-04-28T13:36:26.781 INFO:teuthology.orchestra.run.smithi196.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52831.log 2024-04-28T13:36:26.782 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48158.log 2024-04-28T13:36:26.783 INFO:teuthology.orchestra.run.smithi196.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52148.log.gz 2024-04-28T13:36:26.784 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.52831.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52831.log.gz 2024-04-28T13:36:26.784 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42630.log 2024-04-28T13:36:26.784 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44643.log/var/log/ceph/ceph-client.admin.48158.log: 2024-04-28T13:36:26.785 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42630.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48158.log.gz 2024-04-28T13:36:26.786 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51205.log 2024-04-28T13:36:26.786 INFO:teuthology.orchestra.run.smithi196.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42630.log.gz 2024-04-28T13:36:26.787 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43725.log 2024-04-28T13:36:26.787 INFO:teuthology.orchestra.run.smithi196.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44643.log.gz 2024-04-28T13:36:26.788 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.51205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50809.log 2024-04-28T13:36:26.788 INFO:teuthology.orchestra.run.smithi196.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51205.log.gz 2024-04-28T13:36:26.789 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43725.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-28T13:36:26.790 INFO:teuthology.orchestra.run.smithi196.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43725.log.gz 2024-04-28T13:36:26.790 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50809.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.43575.log -- replaced with /var/log/ceph/ceph-client.admin.50809.log.gz 2024-04-28T13:36:26.791 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-04-28T13:36:26.792 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48074.log 2024-04-28T13:36:26.793 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43575.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.43575.log.gz -5 2024-04-28T13:36:26.793 INFO:teuthology.orchestra.run.smithi196.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40084.log 2024-04-28T13:36:26.794 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48074.log: gzip -5 --verbose -- 92.5% /var/log/ceph/ceph-client.admin.43971.log -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-28T13:36:26.794 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-04-28T13:36:26.794 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40084.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48074.log.gz 2024-04-28T13:36:26.795 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43533.log 2024-04-28T13:36:26.795 INFO:teuthology.orchestra.run.smithi196.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40084.log.gz 2024-04-28T13:36:26.797 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47094.log 2024-04-28T13:36:26.797 INFO:teuthology.orchestra.run.smithi196.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43971.log.gz 2024-04-28T13:36:26.798 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43533.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.43533.log.gz 2024-04-28T13:36:26.798 INFO:teuthology.orchestra.run.smithi196.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41631.log 2024-04-28T13:36:26.798 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.0.47094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48116.log 2024-04-28T13:36:26.799 INFO:teuthology.orchestra.run.smithi196.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47094.log.gz 2024-04-28T13:36:26.800 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.41631.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40161.log 2024-04-28T13:36:26.800 INFO:teuthology.orchestra.run.smithi196.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41631.log.gz 2024-04-28T13:36:26.801 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50132.log 2024-04-28T13:36:26.801 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48116.log.gz 2024-04-28T13:36:26.802 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45099.log 2024-04-28T13:36:26.802 INFO:teuthology.orchestra.run.smithi196.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40161.log.gz 2024-04-28T13:36:26.804 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48622.log 2024-04-28T13:36:26.804 INFO:teuthology.orchestra.run.smithi196.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50132.log.gz 2024-04-28T13:36:26.804 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45817.log 2024-04-28T13:36:26.805 INFO:teuthology.orchestra.run.smithi196.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45099.log.gz 2024-04-28T13:36:26.806 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45565.log 2024-04-28T13:36:26.806 INFO:teuthology.orchestra.run.smithi196.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48622.log.gz 2024-04-28T13:36:26.807 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45883.log 2024-04-28T13:36:26.808 INFO:teuthology.orchestra.run.smithi196.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45817.log.gz 2024-04-28T13:36:26.808 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51247.log 2024-04-28T13:36:26.809 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45565.log.gz 2024-04-28T13:36:26.809 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45775.log 2024-04-28T13:36:26.810 INFO:teuthology.orchestra.run.smithi196.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45883.log.gz 2024-04-28T13:36:26.811 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.51247.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51247.log.gz 2024-04-28T13:36:26.811 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51289.log 2024-04-28T13:36:26.812 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51331.log 2024-04-28T13:36:26.812 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.45775.log: /var/log/ceph/ceph-client.admin.51289.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45775.log.gz 2024-04-28T13:36:26.813 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40946.log 2024-04-28T13:36:26.813 INFO:teuthology.orchestra.run.smithi196.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51289.log.gz 2024-04-28T13:36:26.814 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.51331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38271.log 2024-04-28T13:36:26.815 INFO:teuthology.orchestra.run.smithi196.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.51331.log.gz 2024-04-28T13:36:26.815 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40946.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40946.log.gz 2024-04-28T13:36:26.815 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40862.log 2024-04-28T13:36:26.816 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42420.log 2024-04-28T13:36:26.817 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.38271.log: /var/log/ceph/ceph-client.admin.40862.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38271.log.gz 2024-04-28T13:36:26.818 INFO:teuthology.orchestra.run.smithi196.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40862.log.gz 2024-04-28T13:36:26.818 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47452.log 2024-04-28T13:36:26.818 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.42420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51780.log 2024-04-28T13:36:26.820 INFO:teuthology.orchestra.run.smithi196.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42420.log.gz 2024-04-28T13:36:26.820 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.47452.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47452.log.gz 2024-04-28T13:36:26.820 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40148.log 2024-04-28T13:36:26.821 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44475.log 2024-04-28T13:36:26.822 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.51780.log: /var/log/ceph/ceph-client.admin.40148.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51780.log.gz 2024-04-28T13:36:26.822 INFO:teuthology.orchestra.run.smithi196.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40148.log.gz 2024-04-28T13:36:26.822 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44223.log 2024-04-28T13:36:26.823 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39490.log 2024-04-28T13:36:26.824 INFO:teuthology.orchestra.run.smithi196.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44475.log.gz 2024-04-28T13:36:26.824 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44223.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44223.log.gz 2024-04-28T13:36:26.824 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48994.log 2024-04-28T13:36:26.825 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5/var/log/ceph/ceph-client.admin.39490.log: --verbose -- /var/log/ceph/ceph-client.admin.40125.log 2024-04-28T13:36:26.826 INFO:teuthology.orchestra.run.smithi196.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39490.log.gz 2024-04-28T13:36:26.826 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50515.log 89.4% 2024-04-28T13:36:26.827 INFO:teuthology.orchestra.run.smithi196.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48994.log.gz 2024-04-28T13:36:26.827 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.40125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39487.log 2024-04-28T13:36:26.828 INFO:teuthology.orchestra.run.smithi196.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40125.log.gz 2024-04-28T13:36:26.829 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.50515.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50515.log.gz 2024-04-28T13:36:26.829 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44517.log 2024-04-28T13:36:26.829 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43929.log 2024-04-28T13:36:26.830 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44517.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39487.log.gz 2024-04-28T13:36:26.831 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40610.log 2024-04-28T13:36:26.831 INFO:teuthology.orchestra.run.smithi196.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44517.log.gz 2024-04-28T13:36:26.832 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.43929.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43929.log.gz 2024-04-28T13:36:26.832 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48520.log 2024-04-28T13:36:26.833 INFO:teuthology.orchestra.run.smithi196.stderr:gzip/var/log/ceph/ceph-client.admin.40610.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39014.log 2024-04-28T13:36:26.834 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.48520.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40610.log.gz 2024-04-28T13:36:26.835 INFO:teuthology.orchestra.run.smithi196.stderr:gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48520.log.gz 2024-04-28T13:36:26.835 INFO:teuthology.orchestra.run.smithi196.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44391.log 2024-04-28T13:36:26.836 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.39014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47829.log 2024-04-28T13:36:26.836 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39014.log.gz 2024-04-28T13:36:26.837 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.44391.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44391.log.gz 2024-04-28T13:36:26.838 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.47829.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47829.log.gz 2024-04-28T13:36:26.840 INFO:teuthology.orchestra.run.smithi196.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-28T13:36:26.860 INFO:teuthology.orchestra.run.smithi196.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-28T13:36:26.861 INFO:teuthology.orchestra.run.smithi196.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-28T13:36:27.023 INFO:teuthology.orchestra.run.smithi196.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-28T13:36:27.026 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-04-28T13:36:27.026 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m0.593s 2024-04-28T13:36:27.027 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m1.370s 2024-04-28T13:36:27.027 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.952s 2024-04-28T13:36:27.028 INFO:tasks.ceph:Archiving logs... 2024-04-28T13:36:27.028 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911/remote/smithi196/log 2024-04-28T13:36:27.029 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-28T13:36:27.536 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911/remote/smithi203/log 2024-04-28T13:36:27.537 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-28T13:36:27.626 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-04-28T13:36:27.626 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-04-28T13:36:27.626 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-04-28T13:36:27.626 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-28T13:36:27.627 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-28T13:36:27.627 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-28T13:36:27.823 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-28T13:36:27.849 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-28T13:36:27.849 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-28T13:36:27.857 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-28T13:36:27.987 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-28T13:36:27.996 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-28T13:36:27.996 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-28T13:36:28.004 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-28T13:36:28.050 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-04-28T13:36:28.057 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-04-28T13:36:28.061 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-28T13:36:28.061 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-04-28T13:36:28.061 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv01.front.sepia.ceph.com 3 6 377 64 +2003us[+2010us] +/- 60ms 2024-04-28T13:36:28.061 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 63 -643us[ -643us] +/- 63ms 2024-04-28T13:36:28.061 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 -1716us[-1716us] +/- 72ms 2024-04-28T13:36:28.061 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T13:36:28.066 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-28T13:36:28.066 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-04-28T13:36:28.066 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 +2002us[+2007us] +/- 60ms 2024-04-28T13:36:28.066 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 63 -628us[ -628us] +/- 63ms 2024-04-28T13:36:28.067 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 -1686us[-1680us] +/- 72ms 2024-04-28T13:36:28.067 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T13:36:28.068 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-28T13:36:28.077 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-28T13:36:28.077 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-28T13:36:28.086 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-28T13:36:28.267 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-28T13:36:28.465 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-28T13:36:28.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:36:28.558 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-28T13:36:28.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:36:28.654 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-28T13:36:28.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-28T13:36:28.686 INFO:teuthology.task.internal:Duration was 833.330954 seconds 2024-04-28T13:36:28.686 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-28T13:36:28.696 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-28T13:36:28.696 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-28T13:36:28.703 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-28T13:36:28.844 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-28T13:36:28.851 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-28T13:36:29.228 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-28T13:36:29.229 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-04-28T13:36:29.229 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-28T13:36:29.292 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2024-04-28T13:36:29.292 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-28T13:36:29.366 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-28T13:36:29.366 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-28T13:36:29.374 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-28T13:36:29.454 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-28T13:36:29.454 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-28T13:36:29.459 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-28T13:36:29.633 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-28T13:36:29.643 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-28T13:36:29.643 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-28T13:36:29.729 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-28T13:36:29.815 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-28T13:36:29.825 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-28T13:36:29.833 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-28T13:36:29.928 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-04-28T13:36:29.929 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2024-04-28T13:36:29.971 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-28T13:36:30.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:36:30.021 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-28T13:36:30.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:36:30.077 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-28T13:36:30.087 INFO:teuthology.task.internal:Transferring archived files... 2024-04-28T13:36:30.087 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911/remote/smithi196 2024-04-28T13:36:30.087 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-28T13:36:30.183 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911/remote/smithi203 2024-04-28T13:36:30.183 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-28T13:36:30.274 INFO:teuthology.task.internal:Removing archive directory... 2024-04-28T13:36:30.274 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-28T13:36:30.282 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-28T13:36:30.336 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-28T13:36:30.348 INFO:teuthology.task.internal:Not uploading archives. 2024-04-28T13:36:30.348 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-28T13:36:30.357 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-28T13:36:30.358 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-28T13:36:30.363 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-28T13:36:30.412 INFO:teuthology.orchestra.run.smithi196.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 28 13:36 /home/ubuntu/cephtest 2024-04-28T13:36:30.420 INFO:teuthology.orchestra.run.smithi203.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 28 13:36 /home/ubuntu/cephtest 2024-04-28T13:36:30.423 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-28T13:36:30.433 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-28T13:36:30.453 INFO:teuthology.nuke:Checking targets against current locks 2024-04-28T13:36:30.480 DEBUG:teuthology.nuke:shortname: smithi196 2024-04-28T13:36:30.480 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-28T13:36:30.504 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-28 13:11:16.106168', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI7L8TE5So2DQv96t6FnSiAvpZaOROivbIf2fyRlxXKTgmq3DTpY0WPzfNamw+B3wfidZAzU0uOrvF6H1I2X2g0='} 2024-04-28T13:36:30.507 DEBUG:teuthology.nuke:shortname: smithi203 2024-04-28T13:36:30.507 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-28T13:36:30.532 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676911', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-28 13:11:16.107648', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHYTvuYQNVm4jN95gDka/WmqvB4XshGdGOq29jZP/IqzYmL0Ee4E/59D4XGQeznrW/74dP6qqCG/Y7eMwSIaH5w='} 2024-04-28T13:36:30.562 INFO:teuthology.orchestra.console.smithi196:Power off 2024-04-28T13:36:30.562 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-28T13:36:30.586 DEBUG:teuthology.orchestra.console.smithi196:power off output: Chassis Power Control: Down/Off 2024-04-28T13:36:30.586 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:36:30.602 INFO:teuthology.orchestra.console.smithi203:Power off 2024-04-28T13:36:30.602 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-28T13:36:30.611 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-04-28T13:36:30.623 DEBUG:teuthology.orchestra.console.smithi203:power off output: Chassis Power Control: Down/Off 2024-04-28T13:36:30.623 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:36:30.643 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2024-04-28T13:36:34.612 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:36:34.645 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:36:34.722 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-04-28T13:36:34.756 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2024-04-28T13:36:38.724 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:36:38.757 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:36:38.835 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-04-28T13:36:38.866 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off 2024-04-28T13:36:38.936 INFO:teuthology.orchestra.console.smithi196:Power off completed 2024-04-28T13:36:38.967 INFO:teuthology.orchestra.console.smithi203:Power off completed 2024-04-28T13:36:39.073 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/quiesce} duration: 833.330953836441 failure_reason: 'Test failure: test_quiesce_find (tasks.cephfs.test_quiesce.TestQuiesce)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=50b72d1600b845d0924df4aa241bdae9 status: fail success: false 2024-04-28T13:36:39.073 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-28T13:36:39.159 INFO:teuthology.run:FAIL